Using line_profiler/kernprof with mpiexec

Question

I'm trying to do some line-by-line profiling of a Python program that uses mpi4py using the kernprof (https://github.com/rkern/line_profiler) profiler.

If I simply run the script with in the normal way:

kernprof -l -v mpi_program.py

everything seems to work, but course only a single instance (rank=0) of the MPI program is run. I therefore want to run it with mpiexec to get more processes running, but if I run the profiler inside mpiexec the programs just runs and no profiling information is produced:

mpiexec -l kernprof -l -v mpi_program.py

Does anyone have any experiance with running kernprofon an MPI program?

Answer 1

Did you manually decorate any functions? I have run kernprof with mpi4py programs and also had empty results unless I specified particular functions with @profile.