Execute bash script that loads environment modules (Intel fortran compiler) from within Python

102 Views Asked by At

I have a Python script that calls three instances of a program written in Fortran to be run in parallel. I managed to get this set up running with the following code snippet, where f"{sim}.out" is the compiled program:

from subprocess import Popen, PIPE

def parallel_execution(simulations: list):
    cmds = [[f'cd /path/to/script; ./{sim}.out'] for sim in sims]
    procs = [Popen(cmd, stdout=PIPE, stderr=PIPE, shell=true) for cmd in cmds]
    for proc in procs:
        proc.wait()

For the compiled Fortran code, I have to use the Intel Fortran Compiler which I load with environment modules prior to launching the Python script from the terminal:

source /usr/share/modules/init/bash
module load intel/2016

It all works out quite well. However, I now wanted to be able to run the whole set-up from within PyCharm console. For this, I wrote a bash script which activates the Intel Compiler and then executes the Fortran code:

#!/bin/bash

# initialze environment modules and load intel compiler
source /usr/share/modules/init/bash
module load intel/2016

# run fortran code
cd /path/to/fortran/code
./fortran_code.out

When I run this script from the terminal, it works like a charm. However, when trying to run the bash script instead of the compiled Fortran code within the parallel execution (i.e. first code snipped), it does nothing at all (neither starting the Python script from the terminal nor from within Pycharm). I then tried to run one bash script at a time with os.system('./bash_script.sh') and it throws "error (76)" and says something within the lines of "buffer overflow detected".

What am I doing wrong? Why is it not possible to run the bash script from within Python while manually starting it from the terminal works like a charm?

I appreciate any help!

0

There are 0 best solutions below