Retry does not work when running with run_until_complete

Question

First, I running the code like that, and the retry is working properly.

# -*- coding:utf-8 -*-
from retrying import retry
import asyncio
import time
num = 0;

def retry_if_result_none(result):
    return result is None

@retry(retry_on_result=retry_if_result_none) 
def get_result():
    global num;
    num += 1;
    if num < 10:
        print('Retry.....');
        return None;
    else:
        return True;
    time.sleep(1);

def call():
    end = get_result();
    if end:
        print('ok');
    else:
        print('over')

if __name__ == '__main__':
    call();

Output:
Retry.....
Retry.....
Retry.....
Retry.....
Retry.....
Retry.....
Retry.....
Retry.....
Retry.....
ok

Second, I edit the code like that, and running again, but receive difference result.

# -*- coding:utf-8 -*-
from retrying import retry
import asyncio
import time
num = 0;

def retry_if_result_none(result):
#    print("retry_if_result_none") 
    return result is None

@retry(retry_on_result=retry_if_result_none) 
async def get_result():
    global num;
    num += 1;
    if num < 10:
        print('Retry.....');
        return None;
    else:
        return True;
    time.sleep(1);

async def call():
    end = await get_result();
    if end:
        print('ok');
    else:
        print('over')

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(call())

Output:    
Retry.....
over

As the show,the retry doesn't work in the second code. The difference is that I put the call() in loop.run_until_complete method, How to resolve this issue?

Answer 1

The relevant difference is that, in your second code snipped, you no longer decorate a function, but a coroutine. They return only after being executed in an event loop.

Adding a debug print for result to your check function and running it with your synchronous code example shows the expected result:

def retry_if_result_none(result):
    print(result)
    return result is None

Retry.....
None
Retry.....
None
True # Note: I have set the condition to num < 3
ok

If you do the same with your asynchronous version, you see the problem:

<coroutine object get_result at 0x0322EF90>
Retry.....
over

So result is actually the couroutine itself, not its result. Hence your retry_if_result_none function returns False and the retry loop is terminated after the first iteration.

It's basically a timing issue. Your synchronous decorator is not in sync (pun very much intended) with the asynchronous execution of the coroutine in the event loop.

You'll have to use an asynchronous decorator to be able to await the result of the coroutine. I have adopted this basic but functional asnyc retry decorator to make its decision based on the return value of your function, like the one from retrying does.

Note that the inner wrapper function is a coroutine that awaits the result of the decorated coroutine get_result.

def tries(func):
    def func_wrapper(f):
        async def wrapper(*args, **kwargs):
            while True:
                try:
                    if func(await f(*args, **kwargs)):
                        continue
                    else:
                        break
                except Exception as exc:
                    pass
            return True
        return wrapper
    return func_wrapper

@tries(retry_if_result_none)
async def get_result():
    [...]

Using this on your asynchronous code yields the expected output:

Retry.....
None
Retry.....
None
[...]
Retry.....
None
True
ok

The rest of your code has not been altered except for switching the decorator on get_result and the mentioned print statement in the retry_if_result_none function.