Cannot call Keras from Slurm

Question

I want to use Keras on a cluster using Slurm as the job engine.

If I open a terminal and run the following commands, everything works fine:

$python
>>> import tensorflow
>>> import keras

However, if I place import tensorflow and import keras in a Python file that I then call from slurm :

srun [bunch of parameters for my cluster] python mypythonfile.py

Then I get the following error: ImportError: No module named keras.

Is there something specific to do when using Keras in a cluster with Slurm?

Answer 1

I'm reiterating my comment just to show that this question has been answered:

---

It's common to module load xxxx where xxxx is a different Python installation than the default. You usually stick this in your .bash_profile or a similar file to make sure that you have the Python version you want, always available.

When you submit a job with Slurm, it doesn't call your .bash_profile. It just executes the script. You need to make sure that loading your Python distribution is part of that script.

Answer 2

I also met this problem and solved it recently. As the above answer mentioned, slurm command won't excute .bash_profile and the reason that you can import kears directly in python is the environment setting sentence in the .bash_profile. As a result, I added export PATH="/n/home01/username/miniconda3/bin:$PATH" into my sbatch file and then everything worked.