Callback protocol for functions with different kwargs

Question

I have a function that takes a callback function as a parameter:

def function(arg1: int, callback):
    ...

I am trying to add a type hint for the callback function. Now, the callback functions that are passed to this function have the same positional args but the kwargs can be completely different (both in type and name).

def function1(arg1: int, arg2: str, **kwargs1):
    ...

def function2(arg1: int, arg2: str, **kwargs2):
    ...

I am trying to write a Callback Protocol that fits these 2 functions, but so far I don't see a way to make this work when the kwargs are different. Is there a way to create a unified Callback protocol in this case?

Answer 1

You can create this Callback protocol:

from typing_extensions import Protocol


class Callback(Protocol):
    def __call__(self, arg1: int, arg2: str, **kwargs): ...

and type hint callback as Callback:

def function(arg1: int, callback: Callback): ...

Now, among these three function calls:

def function1(arg1: int, arg2: str, **kwargs1): ...
def function2(arg1: int, arg2: str, **kwargs2): ...
def function3(arg1: int): ...

function(0, function1) # Success.
function(0, function2) # Success.
function(0, function3) # Error.

mypy will report success for the first two, but report an error for the third one because of the missing arg2.

The key thing to realize here is that the name of the keyword-arguments parameter does not actually matter. If I take function2 and rename arg1 to arg_1, mypy would complain about that. This is because the public interface of function2 has changed in a backwards-incompatible way. For example, I would have to modify arg1=0 at any call site to arg_1=0.

But if I take the same function and rename kwargs2 to kwargs_2, the public interface does not actually change! This is because it was impossible to ever explicitly refer to the kwargs / kwargs1 / kwargs2 parameter at a call site in the first place. Take this example:

function2(arg1=0, arg2="", kwargs2={})

If I were to dump kwargs2 from inside the function, it would actually have the key "kwargs2" mapped to {}, instead of being an empty dictionary itself. This shows that it is impossible to rely on the name of the keyword-arguments parameter. mypy can therefore allow different names when checking if the public interface of function2 matches that of a Callback.

As for the type, kwargs, kwargs1 and kwargs2 all have a type of Dict[str, Any] left annotated. Since you're consistent across all 3, mypy reports success here, while also enabling you to take advantage of Any for keyword arguments specific to function1 or function2.