References:

• https://curc.readthedocs.io/en/latest/programming/MPI-C.html
• https://mpitutorial.com/tutorials/mpi-broadcast-and-collective-communication/
• https://mpitutorial.com/tutorials/mpi-scatter-gather-and-allgather/

Group operators are very useful for MPI. They allow for swaths of data to be distributed from a root process to all other available processes, or data from all processes can be collected at one process (consider processes grouped by a communicator).

All collective communications are blocking, and all processes execute the same collective communication routine, which can function as a “send” or can function as a “receive”;

Broadcast (MPI_Bcast):

Broadcast function, using this you can send data to all processes grouped by a communicator.

It is possible to create a broadcast function using only MPI_Send and MPI_Recv but, with a default implementation, it will be pretty inefficient, the MPI_Bcast uses a tree approach:

In this illustration, process zero starts with the data and sends it to process one. From that point on, process one now helps the root process by forwarding data to process three. During the second stage, two network connections are being used at the same time. Network utilization doubles at each subsequent stage of tree communication until all processes have received data.

Function prototype:

int MPI_Bcast(
	void *buffer, //Address of message to be sent or message receiver
	int count, //Quantity of elements pointed/sent
	MPI_Datatype datatype, //Type of message data
	int root, //Rank of the process that will send the message
	MPI_Comm comm //Communicator
);

If the process is the root, the buffer must point to the message to be sent. Otherwise, in all other processes, the buffer must point to the message receiver.

Although the root process and receiver processes do different jobs, they all call the same MPI_Bcast function.

Usage example:

#include <mpi.h>
#include <iostream>
using namespace std;

void printArray(int* array, int length);

int main(int argc, char* argv[]) {
  int rank, size;

  MPI_Init(&argc, &argv);
  MPI_Comm_size(MPI_COMM_WORLD, &size);
  MPI_Comm_rank(MPI_COMM_WORLD, &rank);

  int dataLength = 3;
  int* data = new int[dataLength];

  if (rank == 0) {
    for(int ind=0 ; ind < dataLength ; ind++) {
      data[ind] = (ind+1)*1000;
    }
  }

  MPI_Bcast(data, 3, MPI_INT, 0, MPI_COMM_WORLD);
  if (rank != 0) {
    cout << "Showing data from process " << rank << ": ";
    printArray(data, dataLength);
    cout << endl;
  }

  MPI_Finalize();
  return 0;
}

void printArray(int* array, int length) {
  cout << "{ ";

  for(int ind=0 ; ind<length ; ind++) {
    cout << array[ind];
    if(ind != length - 1) cout << ", ";
  }

  cout << " }";
}

As all other processes receives the data from root (process 0), their data will be equal to the root:

Scatter (MPI_Scatter):