Using Stackless Python to save the state of a large running program?

Question

Given a large (4.5 GB codebase) python testing framework whose execution involves many tens of files, many of which are not directly pickle-able, is it possible to wrap initial execution of the program in a one line function, create a Stackless tasklet around that function, and, during execution, pickle the tasklet as a way of saving the whole program's state? What is the limit of Stackless' tasklet pickling capabilities?

Answer 1

This is indeed a possibility offered by Stackless Pickling

One of the main features of Stackless is its ability to pickle and unpickle tasklets. That means that a running program inside a tasklet can be persistently stored to a file or string. Later, it can be restored again and can continue to run at the point where it was previously halted. This need not be on the same machine!:

So to our question "is it possible" the answer is "yes".

As a matter of "how to do it", I think the link above provide a concrete example. Try it and post an other question if it doesn't work. Given the size of your code base (4.5GB of Python source files is rather huge!), maybe you will reach the limits of Stackless?

Answer 2

To be more concrete: Stackless adds pickling support to a number of built in elements, such as execution frames and modules and other runtime objects However, code, such as classes, functions and modules, are all pickled by name. What this means is that on the other machine, the same objects must be accessable through the import mechanism.
In other words, the pickled execution state will contain the current local variables and all that, but the contents of code objects, or modules will not be picled. These need to be accessible by name when the state is unpickled.