"could not load glyph" in multiprocessing

2k Views Asked by At

I'm running into an issue using pathos multiprocessing that runs figure saving function.

from pathos.multiprocessing import ProcessingPool as Pool
datasets = Pool().map(model_handler, analysisParams)

which throws

  File ".../lib/python3.6/site-packages/matplotlib/backends/backend_pdf.py", line 2029, in draw_text
    font.set_text(s, 0.0, flags=LOAD_NO_HINTING)
RuntimeError: In set_text: could not load glyph
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "wrapper.py", line 410, in <module>
    datasets = Pool().map(model_handler, analysisParams)
  File ".../lib/python3.6/site-packages/pathos/multiprocessing.py", line 137, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File ".../lib/python3.6/site-packages/multiprocess/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File ".../lib/python3.6/site-packages/multiprocess/pool.py", line 608, in get
    raise self._value
RuntimeError: In set_text: could not load glyph
"""

My current understanding is that this occurs because glyph is being called simultaneously by multiple processes. The exception only happens when the number of parallel executions gets >3, which is consistent with that idea. I see that one person's work around was simply to call savefig repeatedly until the command went through, but the parallelized function contains a ton of plots and I'd rather not need to wrap each one in a try statement. Does anyone have an idea about how avoid this exception when plotting figures in parallel? Thanks!!

EDIT Minimal example per comment request.

from pathos.multiprocessing import ProcessingPool as Pool

def model_handler(analysisParam):
    import matplotlib
    matplotlib.use('agg')
    import matplotlib.pyplot as plt

    figout = plt.figure(figsize=(1, 1))
    axes = figout.add_subplot(1,1,1)
    axes.scatter(range(1000), range(1000))
    figout.savefig('dummyfig{}.pdf'.format(analysisParam), format='pdf')
    return analysisParam

datasets_serial = []
for j in range(20):
    datasets_serial.append(model_handler(j))

datasets = Pool(20).map(model_handler, range(20))

Interestingly, if I replace the dataset_serial loop with dataset_serial = map(model_handler, range(20)), the pathos parallel pool executes with no problem, but if I convert the serial map object to a list (print(list(dataset_serial))), the glyph exception in the parallel pool comes back.

0

There are 0 best solutions below