Is there a way to sort the rank in Mpi by ascending and descending order

Question

I am trying to display the Hello World message by ascending and descending order of it rank

I read about the bitonic sort but couldn't understand how to implement it,

int[] datalist = new int[8];
MPI.Init(args);

int rank = MPI.COMM_WORLD.Rank();
int size = MPI.COMM_WORLD.Size();

System.out.println("Hello World from <"+rank+"> of total "+size+" processes");

MPI.Finalize();

I get the output from this coding but didn't how to output it by sorting it, I really need help cause I am still new with the mpi stuff

Answer 1

The question is not very clear. It may be interpreted in two ways:

1. You would like to see the printed rank in ascending or descending order. Any MPI library, including MPJ Express, follows the Single Program Multiple Data (SPMD) model. This essentially means that multiple copies of your program will be executed. The number of copies depend upon how many parallel processes you specified while executing the program with mpjrun switch (using -np switch). It is not possible for MPJ Express to print this line in any particular order since it has no control over execution order of parallel copies of this program. So the output will always be non-deterministic.

2. You would like to see the data in datalist array to be sorted in an ascending or descending order. Again, for that you will need Gather() or Reduce() operation. Currently your program is making N copies of datalist array (assuming that you started N parallel processes).

Hope this helps.

Answer 2

By default, the output of this Hello World program is non-deterministic. But one can force this program to output Hello World in order, i.e. by enforcing a sequential execution order of MPI processes.

Explanation: [assuming that you are familiar with SPMD programming model] Let's look at the execution order of this program.

Step 1: The process with Rank 0 will reach the print statement first and will get the output channel to print it. All other processes will enter else-if and have to wait at the Recv function call. Note: Recv is a blocking call and require a matching Send. Please refer to a complete MPI tutorial for a comprehensive explanation!

Step 2: The Process with Rank 0 will send a message to the process with Rank 1 (rank+1). Now Rank 1 process comes out of the blocking Recv as a matching Send is posted and will get the next turn to print the output. After that, it will send a message to the next process (rank+1) to give it a turn to print.

Next Steps: At each step, a process in else-if will get a matching Send and will come out of the blocking Recv and print Hello World and sends a message to the next rank to allow it to print. Finally, the last else statement is the corner case where the last worker prints the output and don't send a message. The execution is completed after that.

int[] datalist = new int[8];
MPI.Init(args);

int rank = MPI.COMM_WORLD.Rank();
int size = MPI.COMM_WORLD.Size();

int buff[] = new int [1];
buff[0] = rank;
int tag = 1001;
if (rank == 0){ 
    System.out.println("Hello World from <"+rank+"> of total "+size+" processes");
    MPI.COMM_WORLD.Send(buff, 0, 1, MPI.INT, rank+1, tag);
}
else if (rank < size-1){
    MPI.COMM_WORLD.Recv(buff, 0, 1, MPI.INT, rank-1, tag);
    System.out.println("Hello World from <"+rank+"> of total "+size+" processes");
    MPI.COMM_WORLD.Send(buff, 0, 1, MPI.INT, rank+1, tag);
}
else{
    MPI.COMM_WORLD.Recv(buff, 0, 1, MPI.INT, rank-1, tag);
    System.out.println("Hello World from <"+rank+"> of total "+size+" processes");

}

MPI.Finalize();

For 4 processes the output will always be:

Hello World from <0> of total 4 processes
Hello World from <1> of total 4 processes
Hello World from <2> of total 4 processes
Hello World from <3> of total 4 processes