coarray fortran array doesn't get updated

Question

I'm just learning to use coarry with Fortran. I have a very simple program. I have an array of length 9 that I want to distribute over 3 processes, do some calculations, and merge them back to a single array (basic MPI_scatter/MPI_gather like problem).

program name
   implicit none
   integer:: arr(9), local(3)[*]
   integer:: i, j, k, iz

   arr = [(i, i = 1, 9)]

   local(:)[1] = arr(1:3)
   local(:)[2] = arr(4:6)
   local(:)[3] = arr(7:9)

   iz = this_image()
   local = local*iz

   sync all
    
   if(iz == 1) then
      arr(1:3) = local(:)[1]
      arr(4:6) = local(:)[2]
      arr(7:9) = local(:)[3]
      write(*,'(*(i3))')arr
   endif

end program name

I'm compiling this with

gfortran -fcorray=lib -lcaf_mpi

and executing with this

mpirun -np 3 ./a.out

This should a print output like this

 1 2 3 8 10 12 21 24 27

But, If I run the executable multiple times (without recompilation) sometimes it shows multiple results, like 1 2 3 4 5 6 21 24 27 etc. that is the values do not get updated with the calculation.

What I'm doing wrong here? How to fix this problem

Answer 1

Welcome to the wonderful world of shared memory programming!

You have a race condition. It is possible that one of the images is so far behind the others that it resets the (remote) data after the other image performs local = local*iz. Here's a way to fix it, being extra careful that only 1 process ever sets up a given part of the data:

ijb@ijb-Latitude-5410:~/work/stack$ cat caf.f90
program name
   implicit none
   integer:: arr(9), local(3)[*]
   integer:: i, j, k, iz

   iz = this_image()

   ! Only one image should ever write to a given memory location
   ! between synchronisation points. In a real code each different image
   ! would set up different parts of the array
   If( iz == 1 ) Then
      arr = [(i, i = 1, 9)]
      local(:)[1] = arr(1:3)
      local(:)[2] = arr(4:6)
      local(:)[3] = arr(7:9)
   End If

   ! Make sure the array is fully set up before you use it
   sync all

   local = local*iz

   sync all
    
   if(iz == 1) then
      arr(1:3) = local(:)[1]
      arr(4:6) = local(:)[2]
      arr(7:9) = local(:)[3]
      write(*,'(*(i3))')arr
   endif

end program name
ijb@ijb-Latitude-5410:~/work/stack$ mpif90 -std=f2018 -fcheck=all -Wall -Wextra -O -g -fcoarray=lib  caf.f90 -lcaf_openmpi
caf.f90:4:17:

    4 |    integer:: i, j, k, iz
      |                 1
Warning: Unused variable ‘j’ declared at (1) [-Wunused-variable]
caf.f90:4:20:

    4 |    integer:: i, j, k, iz
      |                    1
Warning: Unused variable ‘k’ declared at (1) [-Wunused-variable]
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27
ijb@ijb-Latitude-5410:~/work/stack$ mpirun -np 3 ./a.out
  1  2  3  8 10 12 21 24 27