How to do mocking/monkey patching in huey tasks?

Question

I want to test a huey task, and need to patch requests.get.

# huey_tasks.py

from huey import RedisHuey

huey = RedisHuey()

@huey.task()
def function():
    import requests
    print(requests.get('http://www.google.com'))

File that runs tests:

import huey_tasks

@patch('requests.get')
def call_patched(fake_get):
    fake_get.return_value = '1'
    huey_tasks.function()

Launch huey_consumer: huey_tasks.huey -w 10 -l logs/huey.log
Run test, however patching didn't have any effect.

[2016-01-24 17:01:12,053] INFO:requests.packages.urllib3.connectionpool:Worker-1:Starting new HTTP connection (1): www.google.com
[2016-01-24 17:01:12,562] INFO:requests.packages.urllib3.connectionpool:Worker-1:Starting new HTTP connection (1): www.google.com.sg
<Response[200]>

If I remove the @huey.task() decorator, patching works and 1 gets printed.

So how should I test huey tasks? After all, I can't remove the decorator every time, has to be a better way.

Answer 1

OK, finally found a way to test

# huey_tasks.py

def _function():
    import requests
    print(requests.get('http://www.google.com'))

function = huey.task()(_function)
import huey_tasks

The important part is to define actual task function first then decorate it. Note that huey.task is a decorator that needs arguments.

@patch('requests.get')
def test_patched(fake_get):
    fake_get.return_value = '1'
    huey_tasks._function()

Directly run test code without launching huey_consumer.

Answer 2

If I read correctly this is your problem

• Huey tasks run in a separate consumer process

• Unit tests run in their own process

Process cannot mock or patch another. Either

• Make your code paths so that you do not need to mock patch consumer process... don't call tasks directly, but make them functions which you can expose and patch

• Run huey inside your test process using threading