Which calls in Python may not call `__call__`?

Question

The answer to my question may depend on the interpreter for the code although I'm not sure. If it does, then I would be happy to hear about any widely used Python interpreter, especially CPython perhaps.

Consider the following (uninteresting) example of a callable object c.

class C:
    def __call__(self):
        pass

c = C()

Calling this c will call c.__call__, which may then use its own __call__ method in turn. However, in general, it's clear that not every callable object c always uses c.__call__ when it's called in a chain of calls such as considered.

To give an example, call of the callable(!) object get := vars(cls)["__get__"], where cls is the class types.WrapperDescriptorType, can't always rely on get.__call__ since get happens to be of type cls and so does call := vars(cls)["__call__"]! (If get.__get__ were needed for getting get.__call__, then getting it would have involved some call of get like get(get, get, cls) at some point. Actually, get.__call__ would be got not as call.__get__(get, cls) but apparently as get(call, get, cls).)

Question. Given a callable object c, how can one know whether call of it always relies on c.__call__?

I'd like a way which can be expected to work for all near future versions of Python, with the more robust way wanted more strongly.

Answer 1

You shouldn't ever need to check whether calling an object uses __call__. That's good, because trying to check is really awkward.

If you're worried about Python ignoring a __call__ method you write, you don't need to worry. Unless you try to set __call__ on an instance or something (it needs to be on the class), Python will respect your __call__ methods. The cases where Python internally bypasses __call__ are cases you'd only need to worry about if you're writing C extensions or working on the interpreter core itself.

---

You've correctly recognized that not all callables can be called by calling their __call__ method, because trying to do so would involve recursion with no base case. Aside from the __get__ example you used, there's also the problem of, how do you call a __call__ method? Call its __call__ method? How would you call that?

The primary way this is resolved in the CPython implementation is that, at C level, __call__ isn't actually the primary hook for the function call operator. The primary hook is a tp_call slot in every type object's memory layout, holding a function pointer to the C-level function that handles the call operator for this type.

For types that don't implement __call__, tp_call is NULL. For types with __call__ written in Python, tp_call holds a pointer to slot_tp_call, a C function that searches the type's MRO for a Python-level __call__ method and calls that.

But for types with __call__ written in C, tp_call holds a pointer to a C function that directly implements the type's call functionality. In this case, there is no need to go through a Python-level __call__ method, and no need to invoke further __call__ methods in the process of finding and calling that method.

(There's also a tp_vectorcall slot some types use for more efficient handling, and a bunch of places where CPython special-cases certain types and hardcodes the right handling instead of going through tp_call.)

Types with a tp_call that works this way still have a __call__ method. In this case, __call__ is usually set to an instance of types.WrapperDescriptorType wrapping the tp_call function pointer, where calling the __call__ method delegates to the function pointer.

---

It would be extremely unusual for a program to ever need to check whether calling a type bypasses __call__, because one of tp_call or __call__ is always supposed to delegate to the other. It would take very unusual issues for a type's tp_call and __call__ to behave differently from each other - for example, memory corruption, or an interpreter core bug, and in most such cases, the interpreter state would be broken enough that your check wouldn't do much good.

If you really wanted to check anyway, the most robust check would probably be to compare the class's tp_call pointer against slot_tp_call, but neither tp_call nor slot_tp_call is exposed at Python level. slot_tp_call is even static at C level, so you wouldn't even be able to write typeobj->tp_call == &slot_tp_call in a C extension. You'd have to retrieve the slot_tp_call pointer from the tp_call slot of a type you know uses slot_tp_call.

Less robust checks would be things like assuming tp_call will delegate to __call__ if and only if a type's __call__ is an ordinary Python function object:

import types
if isinstance(type(obj).__call__, types.FunctionType):
    ...