Mvapich2 (with IB) strange info after computing

Question

I'm running example programs (hello-world & pi) with mvapich2 2.2. After computing I see info that looks like debug (in error thread).

output thread

Process 2 of 5 is on 37eff7e817ee
Process 0 of 5 is on cb1479885879
Process 4 of 5 is on 6511b189f06b
Process 1 of 5 is on 6511b189f06b
Process 3 of 5 is on cb1479885879
pi is approximately 3.1415926544231230, Error is 0.0000000008333298
wall clock time = 0.051757

errors thread

[2] 40 at [0x0000000001d8dd28], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1123]
[2] 136 at [0x0000000001d8dbf8], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1110]
[4] 40 at [0x0000000000df3868], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1123]
[4] 136 at [0x0000000000df3738], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1110]
[3] 40 at [0x0000000001bf32c8], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1123]
[3] 136 at [0x0000000001bf3198], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1110]
[0] 40 at [0x00000000012ebda8], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1123]
[0] 136 at [0x00000000013029b8], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1110]
[1] 40 at [0x0000000001514788], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1123]
[1] 136 at [0x0000000001514658], pid/ch3/channels/mrail/src/gen2/rdma_iba_priv.c[1110]

Environment:

• CentOS 7;
• Mellanox MT25204;
• mvapich2-2.2 (installed from yum).

screenshot

How can I disable it?

Answer 1

These prints are caused by memory leaks inside the MPI library or the application itself. It also shows that you are using a debug build of MVAPICH2. If you built MVAPICH2 yourself, remove the options --enable-g=all --enable-fast=none --disable-fast from configure. This will also improve your performance.