Comment on page
MPI4PY
This page documents how to use the MPI for Python package within a Conda environment.
The installation of mpi4py will be discussed in the following sections. This section provides an example of how mpi4py would be used in a python script after such an installation.
To use MPI in a python script through
mpi4py
, you must first import it using the following code:from mpi4py import MPI
Here is an example python script
mpi4pytest.py
that uses MPI:from mpi4py import MPI
import sys
def print_hello(rank, size, name):
msg = "Hello World! I am process {0} of {1} on {2}.\n"
sys.stdout.write(msg.format(rank, size, name))
if __name__ == "__main__":
size = MPI.COMM_WORLD.Get_size()
rank = MPI.COMM_WORLD.Get_rank()
name = MPI.Get_processor_name()
print_hello(rank, size, name)
The file
mpi4pytest.py
can be found at /gpfs/runtime/softwareexamples/mpi4py/$ module load anaconda/2020.02
$ conda create -n my_env
$ conda activate my_env
Once you have activated your conda environment, run the following commands to install
mpi4py
:$ module load mpi/mpi/openmpi_4.0.5_gcc_10.2_slurm20
$ module load gcc/10.2 cuda/11.1.1
$ pip install python=3.7 mpi4py
You may change the python version in the pip command.
To check that the installation process was a success you can run
$ python -c "import mpi4py"
If no errors result from running the command, the installation has worked correctly.
Here is an example batch job script
mpi4pytest_conda.sh
that uses mpi4pytest.py
and the conda environment setup:#!/bin/bash
#SBATCH -N 2
#SBATCH --mem=1G
module load anaconda/2020.02
source /gpfs/runtime/opt/anaconda/2020.02/etc/profile.d/conda.sh
conda activate my_env
module load mpi/mpi/openmpi_4.0.5_gcc_10.2_slurm20 gcc/10.2 cuda/11.1.1
srun --mpi=pmix python mpi4pytest.py
The example script above runs the python script on two nodes by using the
#SBATCH -N 2
command. For more information on #SBATCH
options, see our documentation.Start by creating and activating a python virtual environment:
$ module load python/2.7.16
$ virtualenv my_env
$ source my_env/bin/activate
Once you have activated your conda environment, run the following command to install
mpi4py
:$ python -m pip install mpi4py
$ deactivate
Here is an example batch job script
mpi4pytest_virtualenv.sh
and the python virtual environment setup:#!/bin/bash
#SBATCH -N 2
#SBATCH --mem=1G
module load python/2.7.16
module load mpi/mpi/openmpi_4.0.5_gcc_10.2_slurm20 gcc/10.2 cuda/11.1.1
source my_env/bin/activate
srun --mpi=pmix python mpi4pytest.py
Last modified 2yr ago