# Migration of MPI Apps to Slurm 22.05.7

In January 2023, Oscar will be migrating to use Slurm version 22.05.7.

{% hint style="info" %}
Slurm version 22.05.7

* improves security and speed,
* supports boths PMI2 and PMIX, and
* provides REST APIs
* allows users to prioritize their jobs via scontrol top \<job\_id>
  {% endhint %}

While most applications will be unaffected by these changes, applications built to make use of MPI may need to be rebuilt to work properly. To help facilitate this, we are providing users who use MPI-based applications (either through Oscar's module system or built by users) with advanced access to a test cluster running the new version of Slurm. Instructions for accessing the test cluster, building MPI-based applications, and submitting MPI jobs using the new Slurm, are provided below.

Please note - some existing modules of MPI-based applications will be deprecated and removed from the system as part of this upgrade. A list of modules that will no longer be available to users following the upgrade is given at the bottom of the page.

## Instructions for Testing Applications with Slurm 22.05.7

1. Request access to the Slurm 22.05.7 test cluster (email **<support@ccv.brown.edu>**)
2. Connect to Oscar via either **SSH** or **Open OnDemand** (instructions below)
3. Build your application using the new MPI applications listed below
4. Submit your job

{% hint style="danger" %}
Users must contact **<support@ccv.brown.edu>** to obtain access to the test cluster in order to submit jobs using Slurm 22.05.7.
{% endhint %}

### Connecting via SSH

1. Connect to Oscar using the `ssh` command in a terminal window
2. From Oscar's command line, connect to the test cluster using the command `ssh node1947`
3. From the node1947 command line, submit your jobs (either interactive or batch) as follows:

{% tabs %}
{% tab title="Interactive job" %}

* For CPU-only jobs: `interact -q image-test`
* For GPU jobs: `interact -q gpu`
  {% endtab %}

{% tab title="Batch job" %}
Include the following line within your batch script and then submit using the `sbatch` command, as usual

* For CPU-only jobs: `#SBATCH -p image-test`
* For GPU jobs: `#SBATCH -p gpu`
  {% endtab %}
  {% endtabs %}

### Connecting via Open OnDemand

1. Open a web browser and connect to **poodcit2.services.brown.edu**
2. Login with your Oscar username and password
3. Start a session using the Advanced Desktop App
4. Select the **gpu** partition and click the **launch** button.

{% hint style="info" %}

* Only the Advanced Desktop App will connect to the test cluster
* The Advanced Desktop App must connect to the ***gpu*** partition
  {% endhint %}

## MPI Applications

### Migrated or New Modules

{% hint style="info" %}
If the "Current Module Version" for an application is blank, a new version is built for the application.
{% endhint %}

| Application         | Current Module Version                                                                                                                                                                                                                                                                                                                                                                                  | Migrated or New Module Version                                                                                                                                                                                                 |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **abaqus**          | <ul><li>2021.1\_intel17</li></ul>                                                                                                                                                                                                                                                                                                                                                                       | <ul><li>2021\_slurm22\_a</li></ul>                                                                                                                                                                                             |
| **ambertools**      |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>amber22</li></ul>                                                                                                                                                                                                      |
| **boost**           | <ul><li>1.69</li></ul>                                                                                                                                                                                                                                                                                                                                                                                  | <ul><li>1.69\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                     |
| **CharMM**          | <ul><li>CharMM/c47b1\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                 | <ul><li>CharMM/c47b1</li></ul>                                                                                                                                                                                                 |
| **cp2k**            |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>2022.2</li></ul>                                                                                                                                                                                                       |
| **dedalus**         | <ul><li>2.1905</li><li>2.1905\_openmpi\_4.05\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                              | <ul><li>2.1905\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                   |
| **esmf**            | <ul><li>8.4.0b12</li></ul>                                                                                                                                                                                                                                                                                                                                                                              | <ul><li>8.4.0\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                    |
| **fftw**            | <ul><li>3.3.6</li><li>3.3.8</li></ul>                                                                                                                                                                                                                                                                                                                                                                   | <ul><li>3.3.6\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>3.3.10\_slurm22</li></ul>                                                                                                                                            |
| **global\_arrays**  | <ul><li>5.8\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                               | <ul><li>5.8\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                      |
| **gpaw**            | <ul><li>21.1.0\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>21.1.0\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li><li>21.1.0a\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                        | <ul><li>21.1.0\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>21.1.0\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>21.1.0\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                             |
| **gromacs**         | <ul><li>2018.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                | <ul><li>gromacs/2018.2\_mvapich2-2.3.5\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                           |
| **hdf5**            |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>1.10.8\_mvapich2\_2.3.5\_gcc\_10.2\_slurm22</li><li>1.10.8\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>1.10.8\_openmpi\_4.0.7\_intel\_2020.2\_slurm22</li><li>1.12.2\_openmpi\_4.0.7\_intel\_2020.2\_slurm22</li></ul> |
| **ior**             |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>3.3.0</li></ul>                                                                                                                                                                                                        |
| **lammps**          | <ul><li>29Sep21\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                           | <ul><li>29Sep21\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                  |
| **meme**            | <ul><li>5.3.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                 | <ul><li>5.3.0\_slurm22</li></ul>                                                                                                                                                                                               |
| **Molpro**          | <ul><li>2021.3.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                              | <ul><li>2021.3.1\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                 |
| **mpi**             | <ul><li>hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>mvapich2-2.3.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                            | <ul><li>hpcx\_2.7.0\_gcc\_10.2\_slurm22</li><li>mvapich2-2.3.5\_gcc\_10.2\_slurm22</li><li>openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>openmpi\_4.0.7\_intel\_2020.2\_slurm22</li></ul>                                         |
| **mpi4py**          |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>3.1.4\_py3.9.0\_slurm22</li></ul>                                                                                                                                                                                      |
| **netcdf**          | <ul><li>4.7.4\_gcc\_10.2\_hdf5\_1.10.5</li><li>4.7.4\_intel\_2020.2\_hdf5\_1.12.0</li></ul>                                                                                                                                                                                                                                                                                                             | <ul><li>4.7.4\_gcc\_10.2\_hdf5\_1.10.8\_slurm22</li><li>4.7.4\_gcc\_10.2\_hdf5\_1.12.2\_slurm22</li></ul>                                                                                                                      |
| **netcdf4-python**  |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>1.6.2</li></ul>                                                                                                                                                                                                        |
| **osu-mpi**         |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>5.6.3\_openmpi\_4.0.7\_gcc\_10.2</li></ul>                                                                                                                                                                             |
| **petsc**           |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>petsc/3.18.2\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                             |
| **pnetcdf**         | <ul><li>1.12.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                | <ul><li>1.12.3\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                   |
| **qmcpack**         | <ul><li>3.9.2\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>3.9.2\_openmpi\_4.0.0\_gcc\_8.3\_slurm20</li><li>3.9.2\_openmpi\_4.0.0\_gcc\_8.3\_slurm20\_complex</li><li>3.9.2\_openmpi\_4.0.1\_gcc</li><li>3.9.2\_openmpi\_4.0.4\_gcc</li><li>3.9.2\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li></ul>                                                                                                         | <ul><li>3.9.2\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                                                                                                                    |
| **quantumespresso** | <ul><li>6.4\_openmpi\_4.0.0\_gcc\_8.3\_slurm20</li><li>6.4\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>7.0\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                        | <ul><li>6.4\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>6.4\_openmpi\_4.0.7\_intel\_2020.2\_slurm22</li><li>7.0\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li></ul>                                                                  |
| **vasp**            | <ul><li>5.4.1</li><li>5.4.1\_mvapich2-2.3.5\_intel\_2020.2\_slurm20</li><li>5.4.4</li><li>5.4.4\_intel</li><li>5.4.4\_mvapich2-2.3.5\_intel\_2020.2\_slurm20</li><li>5.4.4\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li><li>5.4.4a</li><li>6.1.1\_ompi405\_yqi27</li><li>6.1.1\_openmpi\_4.0.5\_intel\_2020.2\_yqi27\_slurm20</li><li>6.1.1\_yqi27</li><li>6.3.0\_cfgoldsm</li><li>6.3.2\_avandewa</li></ul> | <ul><li>5.4.1\_slurm22</li><li>5.4.4\_slurm22</li><li>5.4.4\_openmpi\_4.0.7\_gcc\_10.2\_slurm22</li><li>6.1.1\_ompi407\_yqi27\_slurm22</li><li>6.3.0\_cfgoldsm\_slurm22</li><li>6.3.2\_avandewa\_slurm22</li></ul>             |
| **wrf**             |                                                                                                                                                                                                                                                                                                                                                                                                         | <ul><li>4.2.1\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                   |

### To build custom applications:

We recommend using following MPI modules to build your custom applications:

<table><thead><tr><th>MPI</th><th width="386.3333333333333">Oscar Module</th></tr></thead><tbody><tr><td><strong>GCC based OpenMPI</strong></td><td>mpi/openmpi_4.0.7_gcc_10.2_slurm22</td></tr><tr><td><strong>Intel based OpenMPI</strong></td><td>mpi/openmpi_4.0.7_intel_2020.2_slurm22</td></tr><tr><td><strong>MVAPICH</strong></td><td>mpi/mvapich2-2.3.5_gcc_10.2_slurm22</td></tr><tr><td><strong>Mellanox HPC-X</strong></td><td>mpi/hpcx_2.7.0_gcc_10.2_slurm22</td></tr></tbody></table>

{% tabs %}
{% tab title="GNU Configure Example" %}
`module load mpi/openmpi_4.0.7_gcc_10.2_slurm22`

`module load gcc/10.2 cuda/11.7.1`

`CC=mpicc CXX=mpicxx ./configure --prefix=/path/to/install/dir`
{% endtab %}

{% tab title="CMAKE Configure Example" %}
`module load mpi/openmpi_4.0.7_gcc_10.2_slurm22`

`module load gcc/10.2 cuda/11.7.1`

`cmake -DCMAKE_C_COMPILER=mpicc DCMAKE_CXX_COMPILER=mpicxx ..`
{% endtab %}
{% endtabs %}

### Deprecated Modules

{% hint style="info" %}
A new module might be available for a deprecated application module. Please search the table above to check if a new module is available for an application.
{% endhint %}

| Application                 | Deprecated Module                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| abaqus                      | <ul><li>2017</li><li>2021</li><li>2021.1</li><li>6.12sp2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| abinit                      | <ul><li>9.6.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| abyss                       | <ul><li>2.1.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| ambertools                  | <ul><li>amber16</li><li>amber16-gpu</li><li>amber17</li><li>amber17\_lic</li><li>amber21</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| bagel                       | <ul><li>1.2.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| boost                       | <ul><li>1.55</li><li>1.57</li><li>1.68</li><li>1.44.0</li><li>1.62.0-intel</li><li>1.63.0</li><li>1.75.0\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>1.76.0\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>1.76.0\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                              |
| cabana                      | <ul><li>1</li><li>1.1</li><li>1.1\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| campari                     | <ul><li>3.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| cesm                        | <ul><li>1.2.1</li><li>1.2.2</li><li>2.1.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| cp2k                        | <ul><li>7.1</li><li>7.1\_mpi</li><li>8.1.0</li><li>9.1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| dacapo                      | <ul><li>2.7.16\_mvapich2\_intel</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| dalton                      | <ul><li>2018</li><li>2018.0\_mvapich2-2.3.5\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| dice                        | <ul><li>1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| esmf                        | <ul><li>7.1.0r</li><li>8.0.0</li><li>8.0.0b</li><li>8.1.0b11</li><li>8.1.9b17</li><li>8.3.0</li><li>8.3.1b05</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| fenics                      | <ul><li>2017.1</li><li>2018.1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| ffte                        | <ul><li>6.0</li><li>6.0/mpi</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| fftw                        | <ul><li>2.1.5</li><li>2.1.5\_slurm2020</li><li>2.1.5-double</li><li>3.3.8a</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| gerris                      | <ul><li>1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| global\_arrays              | <ul><li>5.6.1</li><li>5.6.1\_i8</li><li>5.6.1\_openmpi\_2.0.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| gpaw                        | <ul><li>1.2.0</li><li>1.2.0\_hpcx\_2.7.0\_gcc</li><li>1.2.0\_mvapich2-2.3a\_gcc</li><li>20.10\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>20.10.0\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| gromacs                     | <ul><li>2016.6</li><li>2020.1</li><li>2018.2\_gpu</li><li>2018.2\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>2020.1\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>2020.4\_gpu</li><li>2020.4\_gpu\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>2020.4\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li><li>2020.6\_plumed</li><li>2021.5\_plumed</li></ul>                                                                                                                                                                                                                                                                                                                          |
| hande                       | <ul><li>1.1.1</li><li>1.1.1\_64</li><li>1.1.1\_debug</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| hdf5                        | <ul><li>1.10.0</li><li>1.10.1\_parallel</li><li>1.10.5</li><li>1.10.5\_fortran</li><li>1.10.5\_mvapich2-2.3.5\_intel\_2020.2\_slurm20</li><li>1.10.5\_openmpi\_3.1.3\_gcc</li><li>1.10.5\_openmpi\_3.1.6\_gcc</li><li>1.10.5\_openmpi\_4.0.0\_gcc</li><li>1.10.5\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li><li>1.10.5\_parallel</li><li>1.10.7\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>1.10.7\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li><li>1.10.7\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>1.12.0\_hpcx\_2.7.0\_intel\_2020.2</li><li>1.12.0\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>1.12.0\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li></ul> |
| hnn                         | <ul><li>1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| hoomd                       | <ul><li>2.9.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| horovod                     | <ul><li>0.19.5</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| ior                         | <ul><li>3.0.1</li><li>3.3.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| lammps                      | <ul><li>17-Nov-16</li><li>11-Aug-17</li><li>16-Mar-18</li><li>22-Aug-18</li><li>7-Aug-19</li><li>11Aug17\_serial</li><li>29Oct20\_hpcx\_2.7.0\_intel\_2020.2</li><li>29Oct20\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| medea                       | <ul><li>3.2.3.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| meme                        | <ul><li>5.0.5</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| meshlab                     | <ul><li>20190129\_qt59</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Molpro                      | <ul><li>2019.2</li><li>2020.1</li><li>2012.1.15</li><li>2015\_gcc</li><li>2015\_serial</li><li>2018.2\_ga</li><li>2019.2\_ga</li><li>2020.1\_ga</li><li>2020.1\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li><li>2021.3.1\_openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                           |
| mpi4py                      | <ul><li>3.0.1\_py3.6.8</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| multinest                   | <ul><li>3.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| n2p2                        | <ul><li>1.0.0</li><li>2.0.0</li><li>2.0.0\_hpcx</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| namd                        | <ul><li>2.11-multicore</li><li>2.13b1-multicore</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| netcdf                      | <ul><li>3.6.3</li><li>4.4.1.1\_gcc</li><li>4.4.1.1\_intel</li><li>4.7.0\_intel2019.3</li><li>4.7.4\_gcc8.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| nwchem                      | <ul><li>7</li><li>6.8-openmpi</li><li>7.0.2\_mvapich2-2.3.5\_intel\_2020.2\_slurm20</li><li>7.0.2\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>7.0.2\_openmpi\_4.1.1\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| openfoam                    | <ul><li>4.1</li><li>7</li><li>4.1-openmpi\_3.1.6\_gcc\_10.2\_slurm20</li><li>4.1a</li><li>7.0\_hpcx\_2.7.0\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| openmpi                     | <ul><li>openmpi\_4.0.5\_gcc\_10.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| Openmpi wth Intel compilers | <ul><li>openmpi\_4.0.5\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| orca                        | <ul><li>4.0.1.2</li><li>4.1.1</li><li>4.2.1</li><li>5.0.0</li><li>5.0.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| osu-mpi                     | <ul><li>5.3.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| paraview                    | <ul><li>5.1.0</li><li>5.1.0\_yurt</li><li>5.4.1</li><li>5.6.0\_no\_scalable</li><li>5.6.0\_yurt</li><li>5.8.0</li><li>5.8.0\_mesa</li><li>5.8.0\_release</li><li>5.8.1\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>5.9.0</li><li>5.9.0\_ui</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                    |
| paris                       | <ul><li>1.1.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| petsc                       | <ul><li>3.14.2\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>3.14.2\_mpich3.3a3\_intel\_2020.2</li><li>3.7.5</li><li>3.7.7</li><li>3.8.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| phyldog                     | <ul><li>1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| plumed                      | <ul><li>2.7.2</li><li>2.7.5</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| pmclib                      | <ul><li>1.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| polychord                   | <ul><li>1</li><li>2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| polyrate                    | <ul><li>17C</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| potfit                      | <ul><li>20201014</li><li>0.7.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| prophet                     | <ul><li>augustegm\_1.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| pstokes                     | <ul><li>1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| pymultinest                 | <ul><li>2.9</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| qchem                       | <ul><li>5.0.2</li><li>5.0.2-openmpi</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| qmcpack                     | <ul><li>3.10.0\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>3.10.0\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>3.7.0</li><li>3.9.1</li><li>3.9.1\_openmpi\_3.1.6</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| quantumespresso             | <ul><li>6.1</li><li>6.4</li><li>6.5</li><li>6.6</li><li>6.4\_hpcx\_2.7.0\_intel\_2020.02\_slurm20</li><li>6.4\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li><li>6.4\_openmpi\_4.0.5\_intel\_slurm20</li><li>6.4.1</li><li>6.5\_openmpi\_4.0.5\_intel\_slurm20</li><li>6.6\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li><li>6.7\_openmpi\_4.0.5\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                  |
| relion                      | <ul><li>3.1.3</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| rotd                        | <ul><li>2014-11-15\_mvapich2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| scalasca                    | <ul><li>2.3.1\_intel</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| scorep                      | <ul><li>3.0\_intel\_mvapich2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| siesta                      | <ul><li>3.2</li><li>4.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| sprng                       | <ul><li>5</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| su2                         | <ul><li>7.0.2</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| trilinos                    | <ul><li>12.12.1</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| vtk                         | <ul><li>7.1.1</li><li>8.1.0</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| wrf                         | <ul><li>3.6.1</li><li>4.2.1\_hpcx\_2.7.0\_intel\_2020.2\_slurm20</li></ul>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |