How to avoid name collisions in python decorators functions

Question

I would like to write a python decorator so that a function raising an exception will be run again until either it succeeds, or it reaches the maximum number of attempts before giving up.

Like so :

def tryagain(func):
    def retrier(*args,**kwargs,attempts=MAXIMUM):
        try:
            return func(*args,**kwargs)
        except Exception as e:
            if numberofattempts > 0:
                logging.error("Failed. Trying again")
                return retrier(*args,**kwargs,attempts=attempts-1)
            else:
                logging.error("Tried %d times and failed, giving up" % MAXIMUM)
                raise e
    return retrier

My problem is that I want a guarantee that no matter what names the kwargs contain, there cannot be a collision with the name used to denote the number of attempts made.

however this does not work when the function itself takes attempts as a keyword argument

@tryagain
def other(a,b,attempts=c):
    ...
    raise Exception

other(x,y,attempts=z)

In this example,if other is run, it will run z times and not MAXIMUM times (note that for this bug to happen, the keyword argument must be explicitly used in the call !).

Answer 1

You can specify decorator parameter, something along the lines of this:

import logging

MAXIMUM = 5

def tryagain(attempts=MAXIMUM):
    def __retrier(func):
        def retrier(*args,**kwargs):
            nonlocal attempts
            while True:
                try:
                    return func(*args,**kwargs)
                except Exception as e:
                    attempts -= 1
                    if attempts > 0:
                        print('Failed, attempts left=', attempts)
                        continue
                    else:
                        print('Giving up')
                        raise
        return retrier
    return __retrier


@tryagain(5)                              # <-- this specifies number of attempts
def fun(attempts='This is my parameter'): # <-- here the function specifies its own `attempts` parameter, unrelated to decorator
    raise Exception(attempts)

fun()

Answer 2

Instead of an argument, get the number of retry attempts from a function attribute.

def tryagain(func):
    def retrier(*args,**kwargs):
        retries = getattr(func, "attempts", MAXIMUM)
        while retries + 1 > 0:
            try:
                return func(*args, **kwargs)
            except Exception as e:
                logging.error("Failed. Trying again")
                last_exception = e
            retries -= 1
        else:
            logging.error("Tried %d times and failed, giving up", retries)
            raise last_exception

    return retrier

@tryagain
def my_func(...):
    ...

my_func.attempts = 10
my_func()  # Will try it 10 times

To make MAXIMUM something you can specify when you call decorate the function, change the definition to

def tryagain(maximum=10):
    def _(f):
        def retrier(*args, **kwargs):
            retries = getattr(func, "attempts", maximum)
            while retries + 1 > 0:
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    logging.error("Failed. Trying again")
                    last_exception = e
                retries -= 1
            else:
                logging.error("Tried %d times and failed, giving up", retries)
                raise last_exception
        return retrier
    return _

Although there's still a risk of a name collision with the attempts attributes, the fact that function attributes are rarely used makes it more reasonable to document tryagain as not working with functions with a pre-existing attempts attribute.

(I leave it as an exercise to modify tryagain to take an attribute name to use as an argument:

@tryagain(15, 'max_retries')
def my_func(...):
    ...

so that you can choose an unused name at decoration time. For that matter, you can also use an argument to tryagain as the name of a keyword argument to add to my_func.)