How to Avoid Leaking of Python Assignment Expressions in Comprehensions

Question

In the Effective Python book, the author recommends using assignment expressions to avoid redundancy in comprehensions, for example:

def fun(i):
    return 2 * i

result = {x: y for x in [0, 1, 2, 3] if (y := fun(x)) > 3}

instead of

result = {x: fun(x) for x in [0, 1, 2, 3] if fun(x) > 3}

result has the value {2: 4, 3: 6}.

The author states that

If a comprehension uses the walrus operator in the value part of the comprehension and doesn’t have a condition, it’ll leak the loop variable into the containing scope. [...] It’s better not to leak loop variables, so I recommend using assignment expressions only in the condition part of a comprehension.

However, in the example above, y is set to 6 at the end of the program. So, the variable in the assignment expression leaked, although it is defined in the condition.

The same thing happens for list comprehensions:

>>> _ = [(x, leak) for x in range(4) if (leak := 2 * x) > 3]
>>> leak
6

And even for generator expressions:

>>> it = ((x, leak) for x in range(4) if (leak := 2 * x) > 3)
>>> next(it)
(2, 4)
>>> leak
4
>>> next(it)
(3, 6)
>>> leak
6

What am I missing? Is there any way to avoid leaking in assignment expressions in comprehensions at all?

Answer 1

In Python, it's impossible not to leak loop variables

Unlike other languages such as C or Java, Python has no separate scope within if and for blocks. So when you use the := operator in an if statement, a for loop, or a list comprehension, the assigned variable will be in scope throughout the remainder of the function or class definition. This also means that after every for loop, the loop variable will still be in scope and contain the value of the loop's last iteration.

I disagree with the author of of Effective Python if he thinks that's a bad thing. "Leaking" loop variables can be very useful! Consider the following example:

while line := f.readLine():
    if 'Kilian' in line:
        break

print('This is the first line that contains your name: ', line)

However, there is one exception to this rule: implicit assignments made in list comprehensions have their own scope:

>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined

This exception is probably where your confusion stems from. It's a special case only, and not applicable when using the := inside a list comprehension.