Mocking the datetime library in pypy raises TypeError when comparing 2 mocked objects

Question

I'm doing some tests for my python library which is supposed to work on all python versions - 27, 33, 34, 35, 36 and pypy. In my tests I'm mocking the datetime library and they all work, except when I run them in pypy (see why I bolded that earlier? such a plot twist!).

Here's a MCVE for the trouble I'm having:

import mock
import datetime as dtl
mk = mock.Mock(wraps=dtl.datetime)
p = mock.patch('datetime.datetime', mk)
p.start()
from datetime import datetime
d1 = datetime.now()
d2 = datetime.now()
print d1 == d2

In all python versions the last line returns False. In pypy the last line throws:

>>>> d1 == d2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pypy/lib_pypy/datetime.py", line 1764, in __eq__
    if isinstance(other, datetime):
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types

I dug through pypy's source code trying to understand the problem, and why on earth are there differences between it and other python versions, and I found the following:

# Comparisons of datetime objects with other.

def __eq__(self, other):
    if isinstance(other, datetime):
        return self._cmp(other) == 0
    elif hasattr(other, "timetuple") and not isinstance(other, date):
        return NotImplemented
    else:
        return False

This kind of makes sense. The datetime class is now a Mock object - and python insists that the second argument to isinstance will be a type. Neato.

So I decided to look at cpython's implementation of datetime, and surprise surprise:

# Comparisons of datetime objects with other.

def __eq__(self, other):
    if isinstance(other, datetime):
        return self._cmp(other, allow_mixed=True) == 0
    elif not isinstance(other, date):
        return NotImplemented
    else:
        return False

The same exact verification is done here, yet it does not raise anything ¯\_(ツ)_/¯.

I'll add that this happens in python 2.7:

>>> d = datetime.now()
>>> d
datetime.datetime(2017, 9, 7, 9, 31, 50, 838155)
>>> d == d
True
>>> datetime
<Mock id='139788521555024'>
>>> isinstance(d, datetime)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types

There are 2 questions I'm struggling with:

1. Why on earth does it work for cpython?
2. How can I mock the datetime object in a way that this will work without reimplementing __eq__ or any other magic method :)

Much obliged!

Answer 1

That's a real problem in python and a good example why using isinstanceof is a bad practice.

The solution is to not patch with a mock, but rather with a class inheriting from datetime.datetime. But, because you will inherit from datetime and still want to check the type based on the real datetime class we will need to override the isinstance check to be on the real class.

import datetime as dtl
import mock

real_datetime_class = dtl.datetime

class DatetimeSubclassMeta(type):
    """Datetime mock metaclass to check instancechek to the real class."""

    @classmethod
    def __instancecheck__(mcs, obj):
        return isinstance(obj, real_datetime_class)

class BaseMockedDatetime(real_datetime_class):
    """Mock class to cover datetime class."""

MockedDatetime = DatetimeSubclassMeta('datetime',
                                      (BaseMockedDatetime,),
                                      {})

p = mock.patch('datetime.datetime', MockedDatetime)
p.start()


from datetime import datetime
d = datetime.now()

isinstance(d, datetime)

• Solution is based on: http://blog.xelnor.net/python-mocking-datetime/

Answer 2

As for "why", it probably has to do with datetime being implemented in C in CPython, and pure python in PyPy, and some subtle differences between a Python class object and a builtin C one.