Measuring the allocated memory with tracemalloc

Question

The Python module tracemalloc offers a detailed insight into the allocated memory of the program. For example, one use case is to record the current and peak memory usage:

import tracemalloc

tracemalloc.start()

# ...code...

current, peak = tracemalloc.get_traced_memory()

If we now wanted the peak to be reset, the documentation suggests to use tracemalloc.reset_peak(). However, this function was only added in Python 3.9, and I was wondering whether I could achieve the same effect with tracemalloc.clear_traces()?

My use case is sth like this:

for i in range(10): 
     # do sth
     current, peak = tracemalloc.get_traced_memory()
     print('Current and peak memory usage: {} {}'.format(current, peak))
     # clear peak memory 

So for every i in the for-loop, I do sth and want to measure only the memory of what I created. The peak should be only per index, not for the global run, that's why I want to clear it.

EDIT: To test the difference between reset_peak() and clear_traces(), I tested this program:

tracemalloc.start()
current_memories = []
peak_memories = []
for i in range(10): 
    a = list(range(100000))
    current, peak = tracemalloc.get_traced_memory()
    current_memories.append(current/(1024*1024))
    peak_memories.append(peak/(1024*1024))
    tracemalloc.reset_peak()
    # tracemalloc.clear_traces()
    del current, peak
print('Average current memory [MB]: {}, average peak memory [MB]: {} +/- {}'.format(
      round(np.mean(current_memories), 4), round(np.mean(peak_memories), 4), 
      round(np.std(peak_memories), 4))
)

When I test clear_traces(), the output is:

Average current memory [MB]: 3.4273, average peak memory [MB]: 3.4274 +/- 0.0019

When I instead use reset_peak(), I obtain:

Average current memory [MB]: 3.4313, average peak memory [MB]: 6.5156 +/- 1.0273

Why is there a difference between the two methods in the amount of memory they display?

Answer 1

It seems impossible to mimic reset_peak before python3.9. reset_peak is a C function whose code is peak = current, and the memory counters are C variables, so it is not possible to modify them in python.

With clear_traces all previous allocations are forgotten:

>>> import tracemalloc
>>> tracemalloc.start()
>>> a = list(range(1000))
>>> tracemalloc.get_traced_memory()
(37619, 47929)
>>> tracemalloc.clear_traces()
>>> tracemalloc.get_traced_memory()
(8691, 19001)