When does Python create a name record in the local scope

Question

This question is not about solving the issue, but rather about understanding the inner workings of the Python (2.7) compiler part.

See the code below. It is a standard decorator function with arguments stripped of all the fancy stuff.

def deco(basedir):
  def wrap(f):
    def newf(*args, **kwargs):
      if basedir == "":          # local or nested?
        basedir="empty"          # local scope
      print basedir              # local or nested?
      return f(*args, **kwargs)
    return newf
  return wrap

@deco(basedir="full")
def test1(a):
  print a

if __name__ == "__main__":
  test1("test1")

When you test it, it fails with UnboundLocalError: local variable 'basedir' referenced before assignment on the if basedir == "": line. Please notice that in this example the body of the conditional (the assignment) was never executed.

But when you replace the if with the following code snippet it suddenly works.

if basedir == "":
  base = "empty"
else:
  base = basedir
print base

I would expect the failing case to create a local variable basedir when the interpreter encounters the assignment, but use the parent's scope in the if statement. But it seems to me that when there is the basedir= assignment present anywhere in the code, it tries to use a local variable in the if as well.

So can anybody please explain to me how the Python compiler creates names in this case? Is the local scope is prepared in advance during compilation time? And when the compiler sees assignment to a local variable does it pre-initialize the name with an undefined value?

Do you know if this behaviour is documented anywhere?

I know a similar situation was solved here before, but I am not after solution. I would like to see the explanation and ideally a pointer to the documentation.

Related to: "local variable referenced before assignment" — only functions?

Answer 1

The problem is that the Python compiler can't determine basedir's proper scope. Scoping in Python is determined statically. But basedir="empty" assignment statement makes basedir variable as a local variable for newf function. And as a result you cannot access a local variable before its first assignment.

And as for your "fix" just you removed assignment of basedir="empty" and basedir isn't more local for newf but it uses from deco function.

Answer 2

In Python 2 you can only set variables globally or locally, not nested:

global_variable_that_is_global = 1
global_variable_that_is_only_used_locally = 2

def function():
    nested_variable_that_can_not_be_set_by_inner_functions = 3
    def inner_function():
        global global_variable_that_is_global
        global_variable_that_is_global = "changed"
        global_variable_that_is_only_used_locally = "changed"
        nested_variable_that_can_not_be_set_by_inner_functions = "changed"

    def get_nested_variable():
        return nested_variable_that_can_not_be_set_by_inner_functions

    return inner_function, get_nested_variable

When you execute this you can try this out:

>>> inner_function, get_nested_variable = function()
>>> get_nested_variable()
3
>>> inner_function()
>>> get_nested_variable()
3
>>> global_variable_that_is_only_used_locally
2
>>> global_variable_that_is_global
'changed'

Therefore and in contrast to Python 2 there is a nonlocal keyword in Python 3:

>>> def function():
    nonlocal_variable = 1
    def inner_function():
        nonlocal nonlocal_variable
        nonlocal_variable = "changed"
    def get_nonlocal_variable():
        return nonlocal_variable
    return inner_function, get_nonlocal_variable

>>> inner_function, get_nonlocal_variable = function()
>>> get_nonlocal_variable()
1
>>> inner_function()
>>> get_nonlocal_variable()
'changed'