Skip to main content

Exchanging data

Point-to-Point and Collective Communication in MPI​

Now that you understand the basics of communicators, rank, and size, it's time to introduce how MPI processes exchange data.

MPI offers two main types of communication:

  • Point-to-point: communication between two specific processes.

  • Collective: communication involving all processes in a communicator.

In this tutorial, we’ll explore three key functions:

  • send: sends data to another process

  • recv: receives data from another process

  • bcast: broadcasts data from one process to all others

Point-to-Point Communication: send and recv​

These functions are the building blocks of MPI messaging. Let’s start with an example.

Python file: mpi_point_to_point.py​

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

if rank == 0:
data = "Hello from rank 0"
comm.send(data, dest=1)
print(f"[Rank 0] Sent message to rank 1")

elif rank == 1:
received = comm.recv(source=0)
print(f"[Rank 1] Received message: '{received}'")

SLURM script: mpi_point_to_point.sh​

#!/bin/bash
#SBATCH --job-name=mpi_for_beginner_point_to_point
#SBATCH --output=mpi_for_beginner_point_to_point.txt
#SBATCH --ntasks=2
#SBATCH --nodes=1
#SBATCH --partition=std

module load python/3.12.5

srun python mpi_point_to_point.py

Output​

mpi_result_03

This example illustrates how rank 0 sends a message to rank 1, and rank 1 waits to receive it. These are blocking operations, meaning recv() will wait until the expected message arrives.

Collective Communication: bcast​

Sometimes, you want one process (often rank 0) to share data with all others. That’s where bcast comes in.

Python file: mpi_broadcast.py​

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

# Only rank 0 sets the data
if rank == 0:
data = {"value": 42}
else:
data = None

# Broadcast the data to all processes
data = comm.bcast(data, root=0)
print(f"Rank {rank} received data: {data}")

SLURM script: mpi_broadcast.sh​

#!/bin/bash
#SBATCH --job-name=mpi_for_beginner_bcast
#SBATCH --output=mpi_for_beginner_bcast.txt
#SBATCH --ntasks=4
#SBATCH --nodes=1
#SBATCH --partition=std

module load python/3.12.5

srun python mpi_broadcast.py

Output​

mpi_result_04

Each process receives the exact same object from rank 0.

Summary​

  • Use send and recv when only two processes need to communicate.

  • Use bcast when a single process needs to share data with all others.