In January 2023, Oscar will be migrating to use Slurm version 22.05.7.
Slurm version 22.05.7
improves security and speed,
supports boths PMI2 and PMIX, and
provides REST APIs
allows users to prioritize their jobs via scontrol top <job_id>
While most applications will be unaffected by these changes, applications built to make use of MPI may need to be rebuilt to work properly. To help facilitate this, we are providing users who use MPI-based applications (either through Oscar's module system or built by users) with advanced access to a test cluster running the new version of Slurm. Instructions for accessing the test cluster, building MPI-based applications, and submitting MPI jobs using the new Slurm, are provided below.
Please note - some existing modules of MPI-based applications will be deprecated and removed from the system as part of this upgrade. A list of modules that will no longer be available to users following the upgrade is given at the bottom of the page.
Request access to the Slurm 22.05.7 test cluster (email support@ccv.brown.edu)
Connect to Oscar via either SSH or Open OnDemand (instructions below)
Build your application using the new MPI applications listed below
Submit your job
Users must contact support@ccv.brown.edu to obtain access to the test cluster in order to submit jobs using Slurm 22.05.7.
Connect to Oscar using the ssh command in a terminal window
From Oscar's command line, connect to the test cluster using the command ssh node1947
From the node1947 command line, submit your jobs (either interactive or batch) as follows:
For CPU-only jobs: interact -q image-test
For GPU jobs: interact -q gpu
Include the following line within your batch script and then submit using the sbatch command, as usual
For CPU-only jobs: #SBATCH -p image-test
For GPU jobs: #SBATCH -p gpu
Open a web browser and connect to poodcit2.services.brown.edu
Login with your Oscar username and password
Start a session using the Advanced Desktop App
Select the gpu partition and click the launch button.
Only the Advanced Desktop App will connect to the test cluster
The Advanced Desktop App must connect to the gpu partition
If the "Current Module Version" for an application is blank, a new version is built for the application.
abaqus
2021.1_intel17
2021_slurm22_a
ambertools
amber22
boost
1.69
1.69_openmpi_4.0.7_gcc_10.2_slurm22
CharMM
CharMM/c47b1_slurm20
CharMM/c47b1
cp2k
2022.2
dedalus
2.1905
2.1905_openmpi_4.05_gcc_10.2_slurm20
2.1905_openmpi_4.0.7_gcc_10.2_slurm22
esmf
8.4.0b12
8.4.0_openmpi_4.0.7_gcc_10.2_slurm22
fftw
3.3.6
3.3.8
3.3.6_openmpi_4.0.7_gcc_10.2_slurm22
3.3.10_slurm22
global_arrays
5.8_openmpi_4.0.5_gcc_10.2_slurm20
5.8_openmpi_4.0.7_gcc_10.2_slurm22
gpaw
21.1.0_hpcx_2.7.0_gcc_10.2_slurm20
21.1.0_openmpi_4.0.5_gcc_10.2_slurm20
21.1.0a_openmpi_4.0.5_gcc_10.2_slurm20
21.1.0_openmpi_4.0.7_gcc_10.2_slurm22
21.1.0_openmpi_4.0.7_gcc_10.2_slurm22
21.1.0_openmpi_4.0.7_gcc_10.2_slurm22
gromacs
2018.2
gromacs/2018.2_mvapich2-2.3.5_gcc_10.2_slurm22
hdf5
1.10.8_mvapich2_2.3.5_gcc_10.2_slurm22
1.10.8_openmpi_4.0.7_gcc_10.2_slurm22
1.10.8_openmpi_4.0.7_intel_2020.2_slurm22
1.12.2_openmpi_4.0.7_intel_2020.2_slurm22
ior
3.3.0
lammps
29Sep21_openmpi_4.0.5_gcc_10.2_slurm20
29Sep21_openmpi_4.0.7_gcc_10.2_slurm22
meme
5.3.0
5.3.0_slurm22
Molpro
2021.3.1
2021.3.1_openmpi_4.0.7_gcc_10.2_slurm22
mpi
hpcx_2.7.0_gcc_10.2_slurm20
mvapich2-2.3.5_gcc_10.2_slurm20
hpcx_2.7.0_gcc_10.2_slurm22
mvapich2-2.3.5_gcc_10.2_slurm22
openmpi_4.0.7_gcc_10.2_slurm22
openmpi_4.0.7_intel_2020.2_slurm22
mpi4py
3.1.4_py3.9.0_slurm22
netcdf
4.7.4_gcc_10.2_hdf5_1.10.5
4.7.4_intel_2020.2_hdf5_1.12.0
4.7.4_gcc_10.2_hdf5_1.10.8_slurm22
4.7.4_gcc_10.2_hdf5_1.12.2_slurm22
netcdf4-python
1.6.2
osu-mpi
5.6.3_openmpi_4.0.7_gcc_10.2
petsc
petsc/3.18.2_openmpi_4.0.7_gcc_10.2_slurm22
pnetcdf
1.12.3
1.12.3_openmpi_4.0.7_gcc_10.2_slurm22
qmcpack
3.9.2_hpcx_2.7.0_gcc_10.2_slurm20
3.9.2_openmpi_4.0.0_gcc_8.3_slurm20
3.9.2_openmpi_4.0.0_gcc_8.3_slurm20_complex
3.9.2_openmpi_4.0.1_gcc
3.9.2_openmpi_4.0.4_gcc
3.9.2_openmpi_4.0.5_intel_2020.2_slurm20
3.9.2_openmpi_4.0.7_gcc_10.2_slurm22
quantumespresso
6.4_openmpi_4.0.0_gcc_8.3_slurm20
6.4_openmpi_4.0.5_intel_2020.2_slurm20
7.0_openmpi_4.0.5_intel_2020.2_slurm20
6.4_openmpi_4.0.7_gcc_10.2_slurm22
6.4_openmpi_4.0.7_intel_2020.2_slurm22
7.0_openmpi_4.0.7_gcc_10.2_slurm22
vasp
5.4.1
5.4.1_mvapich2-2.3.5_intel_2020.2_slurm20
5.4.4
5.4.4_intel
5.4.4_mvapich2-2.3.5_intel_2020.2_slurm20
5.4.4_openmpi_4.0.5_gcc_10.2_slurm20
5.4.4a
6.1.1_ompi405_yqi27
6.1.1_openmpi_4.0.5_intel_2020.2_yqi27_slurm20
6.1.1_yqi27
6.3.0_cfgoldsm
6.3.2_avandewa
5.4.1_slurm22
5.4.4_slurm22
5.4.4_openmpi_4.0.7_gcc_10.2_slurm22
6.1.1_ompi407_yqi27_slurm22
6.3.0_cfgoldsm_slurm22
6.3.2_avandewa_slurm22
wrf
4.2.1_hpcx_2.7.0_intel_2020.2_slurm20
We recommend using following MPI modules to build your custom applications:
GCC based OpenMPI
mpi/openmpi_4.0.7_gcc_10.2_slurm22
Intel based OpenMPI
mpi/openmpi_4.0.7_intel_2020.2_slurm22
MVAPICH
mpi/mvapich2-2.3.5_gcc_10.2_slurm22
Mellanox HPC-X
mpi/hpcx_2.7.0_gcc_10.2_slurm22
module load mpi/openmpi_4.0.7_gcc_10.2_slurm22
module load gcc/10.2 cuda/11.7.1
CC=mpicc CXX=mpicxx ./configure --prefix=/path/to/install/dir
module load mpi/openmpi_4.0.7_gcc_10.2_slurm22
module load gcc/10.2 cuda/11.7.1
cmake -DCMAKE_C_COMPILER=mpicc DCMAKE_CXX_COMPILER=mpicxx ..
A new module might be available for a deprecated application module. Please search the table above to check if a new module is available for an application.
abaqus
2017
2021
2021.1
6.12sp2
abinit
9.6.2
abyss
2.1.1
ambertools
amber16
amber16-gpu
amber17
amber17_lic
amber21
bagel
1.2.2
boost
1.55
1.57
1.68
1.44.0
1.62.0-intel
1.63.0
1.75.0_openmpi_4.0.5_intel_2020.2_slurm20
1.76.0_hpcx_2.7.0_gcc_10.2_slurm20
1.76.0_hpcx_2.7.0_intel_2020.2_slurm20
cabana
1
1.1
1.1_hpcx_2.7.0_gcc_10.2_slurm20
campari
3.0
cesm
1.2.1
1.2.2
2.1.1
cp2k
7.1
7.1_mpi
8.1.0
9.1.0
dacapo
2.7.16_mvapich2_intel
dalton
2018
2018.0_mvapich2-2.3.5_intel_2020.2_slurm20
dice
1
esmf
7.1.0r
8.0.0
8.0.0b
8.1.0b11
8.1.9b17
8.3.0
8.3.1b05
fenics
2017.1
2018.1.0
ffte
6.0
6.0/mpi
fftw
2.1.5
2.1.5_slurm2020
2.1.5-double
3.3.8a
gerris
1
global_arrays
5.6.1
5.6.1_i8
5.6.1_openmpi_2.0.3
gpaw
1.2.0
1.2.0_hpcx_2.7.0_gcc
1.2.0_mvapich2-2.3a_gcc
20.10_hpcx_2.7.0_intel_2020.2_slurm20
20.10.0_hpcx_2.7.0_intel_2020.2_slurm20
gromacs
2016.6
2020.1
2018.2_gpu
2018.2_hpcx_2.7.0_gcc_10.2_slurm20
2020.1_hpcx_2.7.0_gcc_10.2_slurm20
2020.4_gpu
2020.4_gpu_hpcx_2.7.0_gcc_10.2_slurm20
2020.4_hpcx_2.7.0_gcc_10.2_slurm20
2020.6_plumed
2021.5_plumed
hande
1.1.1
1.1.1_64
1.1.1_debug
hdf5
1.10.0
1.10.1_parallel
1.10.5
1.10.5_fortran
1.10.5_mvapich2-2.3.5_intel_2020.2_slurm20
1.10.5_openmpi_3.1.3_gcc
1.10.5_openmpi_3.1.6_gcc
1.10.5_openmpi_4.0.0_gcc
1.10.5_openmpi_4.0.5_gcc_10.2_slurm20
1.10.5_parallel
1.10.7_hpcx_2.7.0_intel_2020.2_slurm20
1.10.7_openmpi_4.0.5_gcc_10.2_slurm20
1.10.7_openmpi_4.0.5_intel_2020.2_slurm20
1.12.0_hpcx_2.7.0_intel_2020.2
1.12.0_hpcx_2.7.0_intel_2020.2_slurm20
1.12.0_openmpi_4.0.5_intel_2020.2_slurm20
hnn
1.0
hoomd
2.9.0
horovod
0.19.5
ior
3.0.1
3.3.0
lammps
17-Nov-16
11-Aug-17
16-Mar-18
22-Aug-18
7-Aug-19
11Aug17_serial
29Oct20_hpcx_2.7.0_intel_2020.2
29Oct20_openmpi_4.0.5_gcc_10.2_slurm20
medea
3.2.3.0
meme
5.0.5
meshlab
20190129_qt59
Molpro
2019.2
2020.1
2012.1.15
2015_gcc
2015_serial
2018.2_ga
2019.2_ga
2020.1_ga
2020.1_openmpi_4.0.5_gcc_10.2_slurm20
2021.3.1_openmpi_4.0.5_gcc_10.2_slurm20
mpi4py
3.0.1_py3.6.8
multinest
3.1
n2p2
1.0.0
2.0.0
2.0.0_hpcx
namd
2.11-multicore
2.13b1-multicore
netcdf
3.6.3
4.4.1.1_gcc
4.4.1.1_intel
4.7.0_intel2019.3
4.7.4_gcc8.3
nwchem
7
6.8-openmpi
7.0.2_mvapich2-2.3.5_intel_2020.2_slurm20
7.0.2_openmpi_4.0.5_intel_2020.2_slurm20
7.0.2_openmpi_4.1.1_gcc_10.2_slurm20
openfoam
4.1
7
4.1-openmpi_3.1.6_gcc_10.2_slurm20
4.1a
7.0_hpcx_2.7.0_gcc_10.2_slurm20
openmpi
openmpi_4.0.5_gcc_10.2_slurm20
Openmpi wth Intel compilers
openmpi_4.0.5_intel_2020.2_slurm20
orca
4.0.1.2
4.1.1
4.2.1
5.0.0
5.0.1
osu-mpi
5.3.2
paraview
5.1.0
5.1.0_yurt
5.4.1
5.6.0_no_scalable
5.6.0_yurt
5.8.0
5.8.0_mesa
5.8.0_release
5.8.1_openmpi_4.0.5_intel_2020.2_slurm20
5.9.0
5.9.0_ui
paris
1.1.3
petsc
3.14.2_hpcx_2.7.0_intel_2020.2_slurm20
3.14.2_mpich3.3a3_intel_2020.2
3.7.5
3.7.7
3.8.3
phyldog
1.0
plumed
2.7.2
2.7.5
pmclib
1.1
polychord
1
2
polyrate
17C
potfit
20201014
0.7.1
prophet
augustegm_1.2
pstokes
1.0
pymultinest
2.9
qchem
5.0.2
5.0.2-openmpi
qmcpack
3.10.0_hpcx_2.7.0_intel_2020.2_slurm20
3.10.0_openmpi_4.0.5_intel_2020.2_slurm20
3.7.0
3.9.1
3.9.1_openmpi_3.1.6
quantumespresso
6.1
6.4
6.5
6.6
6.4_hpcx_2.7.0_intel_2020.02_slurm20
6.4_hpcx_2.7.0_intel_2020.2_slurm20
6.4_openmpi_4.0.5_intel_slurm20
6.4.1
6.5_openmpi_4.0.5_intel_slurm20
6.6_openmpi_4.0.5_intel_2020.2_slurm20
6.7_openmpi_4.0.5_intel_2020.2_slurm20
relion
3.1.3
rotd
2014-11-15_mvapich2
scalasca
2.3.1_intel
scorep
3.0_intel_mvapich2
siesta
3.2
4.1
sprng
5
su2
7.0.2
trilinos
12.12.1
vtk
7.1.1
8.1.0
wrf
3.6.1
4.2.1_hpcx_2.7.0_intel_2020.2_slurm20