How to patch @backoff decorator in Python

Question

Say you have:

Class MyClass:
  @backoff.on_exception(backoff.expo, Exception, max_tries=20)
  def my_method():
    do_something()

For the purposes of testing my_method, you only want to retry 2 times. I have not found a way to patch, wrap, etc. that backoff.on_exception. I am wondering if anyone else can help? Thanks

Answer 1

One option would be to use Runtime Configuration for the max_tries parameter. The documentation says "decorator functions can be passed callables which are evaluated at runtime to obtain the value." You could set an environment variable that the callable checks to determine what max_tries should be set to. It might be as simple as checking an existing ENVIRONMENT variable, if you always do your testing locally. Or you could create a new variable TESTING that is False by default, but switched to True by your tests.

Then your decorator would look like:

class MyClass:
    @backoff.on_exception(backoff.expo, Exception, max_tries=lookup_max_tries)
    def my_method():
        do_something()

And the lookup function would check the environment variable:

def lookup_max_tries():
    if os.getenv("ENVIRONMENT") == "local":
        return 2
    else:
        return 20

Answer 2

Instead of using the @ symbol directly above the my_method method, you can achieve the same result by creating a separate method and applying the backoff.on_exception decorator to it. This decorator takes three arguments: the backoff strategy (backoff.expo in this case), the exception to catch (Exception), and the maximum number of retries (max_tries=2).

Here's the modified code:

import backoff

class MyClass:

    def my_method(self):
        do_something()

# Instantiate MyClass
instantiated_myclass = MyClass()

# Create a backoff decorator with specified parameters
backoff_decorator = backoff.on_exception(backoff.expo, Exception, max_tries=2)

# Apply the decorator to my_method and call the resulting decorated method
backed_off_method = backoff_decorator(instantiated_myclass.my_method)
backed_off_method()

This way, you achieve the same effect as using the @backoff.on_exception decorator directly above the my_method method, but now you have more control over when and how the backoff is applied on an individual method call basis.