Why does unittest's `mock.patch.start` re-run the function in which the patcher is started?

Question

Let's say we have two files:

to_patch.py

from unittest.mock import patch

def patch_a_function():
    print("Patching!")
    patcher = patch("to_be_patched.function")
    patcher.start()
    print("Done patching!")

to_be_patched.py

from to_patch import patch_a_function

def function():
    pass

patch_a_function()
function()

And we run python -m to_be_patched. This will output:

Patching!
Patching!

1. Why isn't Done patching! ever printed?
2. Why is Patching! printed twice?

I've narrowed the answer to (2) down; the call to patch.start seems to trigger patch_a_function again. I suspect this is because it's imported in to_be_patched.py, but am not sure why the function itself would run for a second time. Similarly, I'm not sure why the Done patching! line is not reached in either of the calls to patch_a_function. patcher.start() can't be blocking, because the program exits nicely instead of hanging there... right?

Edit: Huh. It looks like no one can reproduce Done patching! not being printed (which was honestly the main difficulty)—so I guess that's just a my-side problem

Answer 1

1. Why isn't Done patching! ever printed?

Can not reproduce.

$ python -m to_be_patched
Patching!
Patching!
Done patching!
Done patching!

2. Why is Patching! printed twice?

Your module gets imported twice. If you add print(__name__) into the file to_be_patched.py it will be clear:

from to_patch import patch_a_function

print(f"{__name__=}")

def function():
    pass

patch_a_function()
function()  # note: this line doesn't actually do anything, and could be commented out

Result:

$ python -m to_be_patched
__name__='__main__'
Patching!
__name__='to_be_patched'
Patching!
Done patching!
Done patching!

When you use python -m to_be_patched your module to_be_patched will be loaded as top-level code, i.e. the module __name__ will be "__main__".

When mock.patch is used, mock will first import the patch target. When given a patch target as a string like "to_be_patched.function" mock will use importlib, via pkgutil.resolve_name, to find the correct namespace in which to patch. This method loads the target module with __name__ as "to_be_patched", it's not the top-level code environment. Although it's the same underlying .py file being loaded, there is a cache miss in sys.modules, because of the name mismatch: "__main__" != "to_be_patched".

The function patch_a_function now has dual identities and exists in the module __main__ as well as the module to_be_patched, so what you're seeing is each one getting called. The first call triggers the second call, by the double-import mechanism described.

Answer 2

The problem that occurs here is not related to patcher.start(), but due to circular imports and calling patch_a_function from the top level of a script.

When you run to_be_patched it calls patch_a_function.

Now in patch_a_function's patcher = patch("to_be_patched.function") needs to imports to_be_patched and when it's called it calls patch_a_function again.

This time when patch("to_be_patched.function") is called, it doesn't reimport the to_be_patched because it was already imported, so from this point script continues as expected.

You could describe steps as:

 1. CALL patch_a_function
 2. print("Patching!")
 3. CALL `patch("to_be_patched.function")`
 4. `patch` CALLS __import__ of `to_be_patched`
 5. `to_be_patched` is not imported yet so import module
 6. During the import CALL patch_a_function
 7. print("Patching!")
 8. CALL `patch("to_be_patched.function")`
 9. `patch` CALLS __import__ of `to_be_patched`
 10. `to_be_patched` is already imported
 11. Finish the rest of second time called 
 12. CALL patcher.start()
 13. print("Done patching!")
 14. CALL function() (still it is done since we import the module) 
 15. CALL patcher.start()
 16. print("Done patching!")
 17. CALL function()

If you want to resolve the problem in to_be_patched call functions in if __name__ == '__main__' statement:

from to_patch import patch_a_function

def function():
    pass

if __name__ == '__main__':
    patch_a_function()
    function()

Here's more info about top-level code environment: https://docs.python.org/3/library/__main__.html