Python: Two objects seem to be referencing the same variable even though I do not have any explicit connection between them

Question

I am trying to change an octet based on the mask I have. Lets say I have IP: 194.216.35.54 and mask- 27, I need to change the 4th octet (3 when counted from 0). I am trying to change the values in the work_octets variable. But, somehow, the values in the network_octets are also changing. Why is this happening? both network_octets and work_octets are two different lists created by looping over the values and slicing the ip_dict_list. How is it that network_octets getting changed- can any one please help me understand what am I doing wrong here?

Note: I can deepcopy and change this behavior, but I need to know what is it that I am doing wrong here. Any explanation in this regard is highly appreciated.

Code:

octet = 3
ip_dict_list = dc(self.ip_dict_list) # looks like this: [{1: {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 16: 0, 32: 0, 8: 0}}, {2: {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 16: 0, 32: 0, 8: 0}}, {3: {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 16: 0, 32: 0, 8: 0}}, {4: {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 16: 0, 32: 0, 8: 0}}]

vals = [v for i in ip_dict_list for k, v in i.items()]
network_octets = vals[:octet]
work_octets = vals[octet:]
ip_list = ip.split('.')
iof = int(ip_list[octet])  # ip octet in focus (54)
for i in range(len(work_octets)):
    for ob in self.bit_placement:  # loop over the list [128, 64, 32, 16, 8, 4, 2, 1]
        # 32 < 54
        if ob <= iof:
            print "Iof: {}, Ob: {}".format(iof, ob)
            iof = iof - ob  # 54-32 = 22; 22-16 = 6; 6-4: 2; 2-2 = 0 (done)
            work_octets[i][ob] = 1  # {32: 1, 16: 0, 8: 0, 4: 0, 2: 0, 1:0 }
            if iof == 0:
                break

Iof: 54, Ob: 32
Iof: 22, Ob: 16
Iof: 6, Ob: 4
Iof: 2, Ob: 2
print work_octets # as expected
[{128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}]

Now network_octets also got changed and I expect it to remain the same - not as expected

print network_octets 
[{128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}, {128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}, {128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}]

This should be unchanged and should look like this:

network_octets
[{128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 8: 0, 16: 0, 32: 0}, {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 8: 0, 16: 0, 32: 0}, {128: 0, 64: 0, 2: 0, 4: 0, 1: 0, 8: 0, 16: 0, 32: 0}]

The variable vals is also changing after the for loop:
vals
[{128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}, {128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}, {128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}, {128: 0, 64: 0, 2: 1, 4: 1, 1: 0, 8: 0, 16: 1, 32: 1}]

I expect only the work_octets to have the changes in the values of the dict element and the network_octets and vals should remain same. But all the variables are getting modified after the for loop

Answer 1

There is a lot of code, and your snippet is not executable as is, so it is a bit difficult to give definitive answers. Nevertheless, I think the problem lies in how you created your lists.

If I understood your code correctly, network_octets and work_octets are two lists of dictionaries. But those are the same dictionaries, so even if you have different lists (i.e. network_octets is work_octets will be False), their contents are the same (i.e. network_octets[0] is work_octets[0] will be True). Therefore, when you assign work_octets[i][ob], you also touch the dictionary in network_octets[i]. You can confirm that by printing ip_dict_list, which should have changed as well.

We can reproduce the problem with a more minimal example:

source = [{0: 10, 1: 20}, {0: 30, 1: 40}, {0: 50, 1: 60}]
l1 = source[:2]
l2 = source[:2]

# Prints 'True'
print(l1[0] is l2[0])

# Here we're effectively touching `l1[0][0]` *and* `l2[0][0]`
l1[0][0] = 100

# Prints '[{0: 100, 1: 20}, {0: 30, 1: 40}]'
print(l2)

You don't necessarily need to deep copy everything. You can focus only on the objects you're modifying. For instance, in my example it would boil down to the following:

# ...
l1[0] = dict(l1[0])
l1[0][0] = 100

# Prints '[{0: 100, 1: 20}, {0: 30, 1: 40}]'
print(l1)

# Prints '[{0: 10, 1: 20}, {0: 30, 1: 40}]'
print(l2)

Be careful in your code, as the comments suggest your lists actually contain dictionaries of dictionaries. Hence, the aforementioned problem would also apply if you only shallow-copy the outer ones. A deep-copy will surely be safer, but also more expensive if it can be of any concern.

You can try to print-debug with is (like I did in my examples) or id to make sure dictionaries are different before you alter them:

# Prints '4406947912 4406947912'
# Obviously the numbers could be different every time you run your code.
print(id(l1[0]), id(l2[0]))