Dill dump_session: Making Jupyter-session pickle-able again

Question

I like to use dill.dump_session to save my notebook state. However, I often have unpicklable objects loaded (dask-clusters, keras models). Is there any way I can remove them from my environment and then use dump_session()?

I tried to find the offending object with dill.detect.errors:

(with globals() and locals() )

problemDict={}
exceptions=["Out","get_ipython","exit","quit"]

for name, obj in globals().items():
    if name in exceptions or name[0]=="_"::
        continue
    if dill.detect.errors(obj) is not None:
        print(name)
        problemDict[name]=obj

I deleted the two offending objects (and the summary dict afterwards) with

for name, obj in problemDict.items():
    print(name)
    del(globals()[name])

and they are gone form my namespace, but I still get the same error:

TypeError: cannot pickle 'tensorflow.python._tf_stack.StackSummary' object

Could the keras-model still "hide" somewhere? Do the imports already make the session unpicklable? Unfortunately, the dill.dump_session source code does not really help me much. (https://github.com/uqfoundation/dill/blob/master/dill/_dill.py#L343)

Edit: the "dill.detect.trace(False)"-output did also not help. No idea what to do with:

...
# T4
2021-04-14 14:12:34,726 # T4
# D2
2021-04-14 14:12:34,727 # D2
# D2
2021-04-14 14:12:34,728 # D2
D2: <dict object at 0x7ffa9030a6c0>
2021-04-14 14:12:34,728 D2: <dict object at 0x7ffa9030a6c0>
D2: <dict object at 0x7ffa90305b80>
2021-04-14 14:12:34,736 D2: <dict object at 0x7ffa90305b80>
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
...

Any help appreciated!

Answer 1

While the original notebook is long closed (could not solve it), I had a similar issue. Again, some variable was not pickable, and seemed to reside in some ipython variable after del. Manually deleting "Out"/"_oh"-keys/items did not help, but

%reset out

made the session finally pickable again!

God, I really hope this was the underlying problem...