Error when running VASP 5.3.5 on Cray XC30

Question

When running VASP 5.3.5 on a Cray XC30 I see the following error:

Rank 4 [Mon Jun  1 14:58:35 2015] [c1-0c0s2n0] Fatal error in PMPI_Allgatherv:  Invalid buffer pointer, error stack:
PMPI_Allgatherv(1235): MPI_Allgatherv(sbuf=0x49ead730, scount=1536,  MPI_DOUBLE_COMPLEX, rbuf=0x49e95730, rcounts=0x494cdba0, displs=0x494d7360, 
MPI_DOUBLE_COMPLEX, comm=0x84000007) failed
PMPI_Allgatherv(1183): Buffers must not be aliased. Consider using MPI_IN_PLACE or setting MPICH_NO_BUFFER_ALIAS_CHECK

that does not exist with an executable that was compiled 6 months ago. Any ideas what is going on here?

I am using GCC 4.9.2 with Cray MPICH2 7.1.1, Cray LibSci 13.0.1 and FFTW 3.3.4.1.

Answer 1

This is due to a change in the MPICH2 library that now strictly enforces non-overlapping buffers in MPI collectives.

Ideally, the code should be modified so that it is compliant with the MPI standard by either modifying the calls so that the buffers do not overlap or by specifying MPI_IN_PLACE in the MPI call.

However, you can revert to the same behaviour as for previous versions of Cray MPICH2 by setting:

export MPICH_NO_BUFFER_ALIAS_CHECK=1

before running the code (in bash, you would usually do this in your job submission script before the aprun call to VASP).

Note this issue is also seen with VASP 4 as well as VASP 5.