Weird Behavior of __new__

Question

I have encountered a peculiar behavior with the __new__ method in Python and would like some clarification on its functionality in different scenarios. Let me illustrate with two unrelated classes, A and B, and provide the initial code:

class A:
    def __new__(cls, *args, **kwargs):
        return super().__new__(B)

    def __init__(self, param):
        print(param)


class B:
    def __init__(self, param):
        print(param)


if __name__ == '__main__':
    a = A(1)

In this case, no output is generated, and neither A's __init__ nor B's __init__ is called.

However, when I modify the code to make B a child of A:

......
class B(A):
......

Suddenly, B's __init__ is invoked, and it prints 1.

I am seeking clarification on how this behavior is occurring. In the first case, if I want to invoke B's __init__ explicitly, I find myself resorting to the following modification:

class A:
    def __new__(cls, *args, **kwargs):
        obj = super().__new__(B)
        obj.__init__(*args, **kwargs)
        return obj

Can someone explain why the initial code behaves as it does and why making B a child of A alters the behavior? Additionally, how does the modified code explicitly calling B's __init__ achieve the desired outcome and not without it?

Answer 1

TL;DR super().__new__(B) is not the same as B().

---

As @jonsharpe pointed out in a comment, __init__ is only called when an instance of cls is returned. But this implicit call to __init__ is handled* by type.__call__, not the call to __new__ itself before it returns.

For example, you can imagine type.__call__ is implemented as something like

def __call__(cls, *args, **kwargs):
    obj = cls.__new__(cls, *args, **kwargs)
    if isinstance(obj, cls):
        obj.__init__(*args, **kwargs)
    return obj

So when you call A(1) in the first case, this translates to A.__call__(1), which gets implemented as type.__call__(A, 1).

Inside __call__, cls is bound to A, so the first line is equivalent to

obj = A.__new__(A, 1)

Inside A.__new__, you don't pass cls as the first argument to super().new, but B, so obj gets assigned an instance of B. That object is not an instance of A, so obj.__init__ never gets called, and then the instance is returned.

When you make B a subclass of A, now obj is an instance of A, so obj.__init__ gets called.

---

* I believe this to be the case; everything I've ever tried is consistent with this model, but I haven't delved deeply enough into CPython to confirm.

For example, if you change A's metaclass to

class MyMeta(type):
    def __call__(cls, *args, **kwargs):
        return cls.__new__(cls, *args, **kwargs)

then B.__init__ fails to be called, even when B is a subclass of A.

The sentence in the documentation,

If __new__() is invoked during object construction and it returns an instance of cls, then the new instance’s __init__() method will be invoked like __init__(self[, ...]), where self is the new instance and the remaining arguments are the same as were passed to the object constructor.

is vague about what exactly "during object construction" means and who is invoking __init__. My understanding of type.__call__ conforms to this, but alternate implementations may be allowed. That would seem to make defining any __call__ method in a custom metaclass that doesn't return super().__call__(*args, **kwargs) a dicey proposition.