Python mock.patch doesn't patch the correct import

Question

Code

def test_get_network_info(self):
    with open(dirname(abspath(__file__)) + '/files/fake_network_info.txt', 'r') as mock_network_info:
        with patch('subprocess.check_output', Mock(return_value=mock_network_info.read())):
            self.assertEqual('192.168.1.100', get_network_info()[0])
            self.assertEqual('255.255.255.0', get_network_info()[1])
            self.assertEqual('192.168.1.0', get_network_info()[2])

Error

======================================================================
ERROR: test_get_network_info (tests.test_tools.ToolsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/Documents/overseer/app/tests/test_tools.py", line 21, in test_get_network_info
    with patch('subprocess.check_output', Mock(return_value=mock_network_info.read())):
  File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1268, in __enter__
    original, local = self.get_original()
  File "/usr/local/lib/python2.7/dist-packages/mock.py", line 1242, in get_original
    "%s does not have the attribute %r" % (target, name)
AttributeError: <module 'subprocess' from '/usr/local/lib/python2.7/dist-packages/twill/other_packages/subprocess.pyc'> does not have the attribute 'check_output'

What I understand

My understanding of the problem is that mock is trying to mock twill's subprocess module instead of the python one.

Questions

1. Am I doing something wrong ?

2. How can I specify that I want to patch the python subprocess module and not the twill's one ? (that may have been imported earlier in the test suite)**

3. Is there another way to patch the subprocess module ?

What I tried

• I tried with patch('tools.subprocess.check_output', ...

Doesn't work.

• I tired to use a decorator ...

Doesn't work either

• I tired to patch directly the subprocess module subprocess.check_output = Mock( ...

Works but it's not good since it doesn't undo the patching.

Some more informations

If I run just this test and no other tests, it works because twill's subprocess module never got imported. But as soon as I run a test using twill, the above test will fail.

Here is the twill's version of subprocess wich looks like it has been copy pasted from an old version of python. It doesn't have any check_output function and that's why the test fails.

Twill's package comes from the Flask-Testing plugin which I use extensively. I submitted an issue on github here.

I hope someone from the lovely python community can help. :)

Answer 1

See my comment up there, due to bad practices in twill, the proper way would be to either fix twill, which may take some work, or move away to something else, but since you now heavily depend on Flask-Testing, it's not a cheap move either.

So this leaves us with a dirty trick: make sure to import subprocess anywhere before twill is imported. Internally, this will add a reference to the right subprocess module in sys.modules. Once a module is loaded, all subsequents import won't look anymore in sys.path but just use the reference already cached in sys.modules.

Unfortunately this is maybe not the end of the problem. Apparently twill uses a patched version of subprocess for some reason ; and those patches won't be available for him, since the plain built-in subprocess will be loaded instead. It's very likely it'll crash or behave in an unexpected way. If it's the case, well ... back to the suggestions above.