A module to profile peak memory usage of Python code

Question

Currently, I tried to use the memory_profiler module to get the used memory like the following code:

from memory_profiler import memory_usage
memories=[]
def get_memory(mem,ended):
  if ended:
    highest_mem=max(mem)
    print highest_mem
  else:
  memories.append(mem)

def f1():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something
def f2():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something

#main
f1()
f2()
ended=True
get_memory(memory_usage(),ended) #code end

>>>#output
# output 
# highest memory 

however, it did not successfully execute. It got stuck when ended=True and sent the value of memory_usage() and ended to the function of get_memory. It did not show any error as well., just waiting for long long time, then I force to stop executing. Anyone knows the better way or the solution?

Answer 1

An easy way to use memory_usage to get the peak / maximum memory from a block of code is to first put that code in a function, and then pass that function - without the () call - to memory_usage() as the proc argument:

from memory_profiler import memory_usage

def myfunc():
  # code
  return
    
mem = max(memory_usage(proc=myfunc))

print("Maximum memory used: {} MiB".format(mem))

Other arguments allow you to collect timestamps, return values, pass arguments to myfunc, etc. The docstring seems to be the only complete source for documentation on this: https://github.com/fabianp/memory_profiler/blob/master/memory_profiler.py

https://github.com/fabianp/memory_profiler/blob/4089e3ed4d5c4197925a2df8393d4cbfca745ae5/memory_profiler.py#L244

Answer 2

I mainly use Heapy because it's really easy to use.

Just type the following code where you want to test for memory usage.

from guppy import hpy
hp = hpy()
print hp.heap()