Profiling memory usage on App Engine

Question

How can I profile memory (RAM) usage on my App Engine app? I'm trying to address errors related to exceeding the instance memory limit. I've tried these things and, so far, they don't work or don't provide what I need.

• Appstats. This doesn't provide memory usage details.
• Apptrace. It hasn't been updated since 2012 and depends on a deprecated version of the SDK. Doesn't work out of the box.
• Appengine-profiler. Doesn't provide memory stats.
• Gae-mini-profiler, which uses cProfile. Doesn't provide memory stats.
• guppy. After downloading and installing the library's code in my app's folder, running guppy.hpy() fails with ImportError: No module named heapyc
• resource. Not part of the SDK's version of python, so I can't use it.

Am I wrong about any of the above? The top-rated answer (not the accepted one) on this question says that there is no way to monitor memory usage on App Engine. That can't be true. Can it?

EDIT

I can confirm that GAE mini profiler does the job. After installation, I could change the settings in the UI to "sampling with memory" and then see this readout:

Thanks to all the contributors!

Answer 1

GAE Mini Profiler does provide memory stats if you use the sampling profiler and set memory_sample_rate nonzero; at each snapshot it will tell you the memory that was in use. You will want to turn the sample frequency way down as the memory sample takes a few ms to execute.

Edit: the way it gets the memory stats is from the GAE runtime API which is deprecated, but still worked as of last I knew; I'm not sure if there's a good replacement.

Answer 2

To add to Ben's answer, as of 16th November 2015, despite being deprecated, the Google App Engine runtime API still works. There isn't an official replacement from Google yet.

from google.appengine.api.runtime import runtime
import logging

logging.info(runtime.memory_usage())

This will output memory usage statistics, where the numbers are expressed in MB. For example:

current: 464.0859375
average1m: 464
average10m: 379.575