NetLogo on HPC (Slurm) without BehaviorSpace

103 Views Asked by At

I want to change a little bit my workflow for running NetLogo on a HPC using Slurm. For context, I run around 360 simulations in parallel, each one can take from 5 to 7 days (I know, not efficient) and they write some outputs at the end of X ticks. I post-process all these outputs after with Python.

I currently use JobArrays, but since the cluster I'm using has nodes with only 48 cores I have to create 9 BehaviorSpaces by hand (I know, I could use XML files).

I have been reading and it seems that in order to run multi-nodes experiments I have to be careful. A few alternatives I have been thinking:

  1. pynetlogo + mpi4pi
  1. pynetlogo + ipyparallel, it would be similar to this example https://pynetlogo.readthedocs.io/en/latest/_docs/SALib_ipyparallel.html

I have no experience with MPI or ipyparallel, but I have experience with Python, NetLogo and a little with Slurm.

I considered netlogo-headless but as far as I know that works with BehaviorSpaces and that is exactly what I want to avoid.

0

There are 0 best solutions below