How to write decorator without syntactic sugar in python?

Question

This question is rather specific, and I believe there are many similar questions but not exactly like this.

I am trying to understand syntactic sugar. My understanding of it is that by definition the code always can be written in a more verbose form, but the sugar exists to make it easier for humans to handle. So there is always a way to write syntactic sugar "without sugar" so to speak?

With that in mind, how precisely do you write a decorator without syntactic sugar? I understand it's basically like:

# With syntactic sugar
@decorator
def foo():
    pass
    
# Without syntactic sugar
def foo():
    pass
foo = decorator(foo)

Except from PEP 318

Current Syntax

The current syntax for function decorators as implemented in Python 2.4a2 is:

@dec2
@dec1
def func(arg1, arg2, ...):
    pass

This is equivalent to:

def func(arg1, arg2, ...):
    pass
func = dec2(dec1(func))

without the intermediate assignment to the variable func. (emphasis mine)

In the example I gave above, which is how the syntactic sugar is commonly explained, there is an intermediate assignment. But how does the syntactic sugar work without the intermediate assignment? A lambda function? But I also thought they could only be one line? Or is the name of the decorated function changed? It seems like that could possibly conflict with another method if the user created one coincidentally with that same name. But I don't know which is why I'm asking.

To give a specific example, I'm thinking of how a property is defined. Since when defining a property's setter method, it cannot work if the setter method is defined as that would destroy the property.

class Person:
    def __init__(self, name):
        self.name = name

    @property
    def name(self):
        return self._name
    # name = property(name)
    # This would work
    
    @name.setter
    def name(self, value):
        self._name = value.upper()
    # name = name.setter(name)
    # This would not work as name is no longer a property but the immediately preceding method 

Answer 1

def func(arg1, arg2, ...):
    pass
func = dec2(dec1(func))

In the example [...] there is an intermediate assignment. But how does the syntactic sugar work without the intermediate assignment?

Actually, the "non syntactic sugar" version, as you call it, is not exactly the same as using the decorator syntax, with an @decorator:

As you noted, when using the @ notation, the initial function name is never assigned a variable: the only assignment that takes place is for the resolved decorator.

So:

@deco
def func():
    ...

What is assigned to func in the globals() scope is the value returned by the call to deco.

While in:

def func():
    ...
func = deco(func)

First func is assigned to the raw function, and just as the func=deco(func) line is executed the former is shadowed by the decorated result.

The same apples for cascading decorators: only the final output, of the topmost decorator, is ever assigned to a variable name.

And, as well, the name used when using the @ syntax is taken from the source code - the name used in the def statement: if one of the decorators happen to modify the function __name__ attribute, that has no effect in the assigned name for the decorated function.

These differences are just implementation details, and derive of the way things work - I am not sure if they are on the language specs, but for those who have a certain grasp on the language, (1) they feel so natural, no one would dare implementing it differently, and (2) they actually make next to no difference - but for code that'd be tracing the program execution (a debugger, or some code using the auditing capabilities of the language (https://docs.python.org/3/library/audit_events.html )).

Despite this not being in other language specs, note however that the difference that the decorator syntax does not make the intermediate assignment is written down in PEP 318. Lacking other references, what is in the PEP is the law:

This is equivalent to: [example without the @ syntax], though without the intermediate creation of a variable named func.

For sake of completeness, it is worth noting that from Python 3.10 (maybe 3.9), the syntax restriction that limited which expressions could be used as decorators after the @ was lifted, superseding the PEP text: any valid expression which evaluates to a callable can be used now.

what about property ?

class Person:
    ...
    @property
    def name(self):
        ...
    
    @name.setter
    def name(self, value):
        ...

What takes place in this example, is that name.setter is a callable, called with the setter method (def name(self, value):) defined bellow, as usual - but what happens is that it returns a property object. (Not the same as @name - a new property, but for the purpose of understanding what takes place, it could even return the same object).

So that code is equivalent to:

class Person:
    ...
    def name(self):
        ...
    name = property(name)  # Creates a property with the getter only
    
    def name_setter(self, value):  # Indeed: this can't be called simply `name`: it would override the property created above
         ...

    name = name.setter(name_setter)  # Creates a new property, copying what already was set in the previous "name" and adding a setter.
    del name_setter  # optionally: removes the explicit setter from the class namespace, as it will always be called via the property. 

In fact, while property was created before the decorator syntax (IRCC, in Python 2.2 - decorator syntax came out in Python 2.4) - it was not initially used as a decorator. The way one used to define properties in Python 2.2 times was:

class Person:
    ...
    def name_getter(self):
        ...
    def name_getter(self):
        ...
    name = property(name_getter, name_setter)

    del name_getter, name_setter # optional

It was only in Python 2.5 (or later) they made the clever ".setter" ".getter" and ".deleter" methods to properties so that it could be entirely defined using the decorator syntax.

Note that for properties with only a getter, @property would work from Python 2.3 on, as it would just take the single parameter supplied by the decorator syntax to be the getter. But it was not extendable to have setter and deleter afterwards.

Answer 2

You can use the dis module (python3 -m dis Person.py) to see how CPython implements this.

First, the (built-in) name property is pushed to the stack, followed by the code object resulting from compelling the body of the def name statement.

  5          16 LOAD_NAME                4 (property)

  6          18 LOAD_CONST               2 (<code object name at 0x108d19960, file "tmp.py", line 5>)

Next, the code object is turned into a function, but that function is not yet bound to a name.

             20 MAKE_FUNCTION            0

Finally, property is called on the function, and the resulting instance is bound to the name name.

  5          22 PRECALL                  0
             26 CALL                     0

  6          36 STORE_NAME               5 (name)

Next, we do the same thing for the setter. Instead of putting the name property on the stack, though, we put the result of evaluating name.setter on the stack.

 11          38 LOAD_NAME                5 (name)
             40 LOAD_ATTR                6 (setter)

Then we treat the new name function as before: loading a code object and making a function, but not yet binding it to any name. (Note that at this point, we neither need to the old value of the name name, but we don't overwrite it yet, either.)

 12          50 LOAD_CONST               3 (<code object name at 0x108c173c0, file "tmp.py", line 11>)
             52 MAKE_FUNCTION            0

Call the bound method on the new function, and bind the result back to name, finally replacing the old value.

 11          54 PRECALL                  0
             58 CALL                     0

 12          68 STORE_NAME               5 (name)