1. Module System#

1.1. Where can I find installed software?#

Most end-user software on Hydra is available via modules that can be dynamically loaded by the user. That’s why it might seems that the software in Hydra is very limited, but it is not the case, we provide more than 2500 modules. To check the full list of installed software, type:

module av

Use the spider option to list all available versions of a given software package:

List all modules of Python#
module spider Python

Use the help option to list all extensions included in a given module:

List all extensions in the module of Python v3.11.3#
module help Python/3.11.3-GCCcore-12.3.0

Modules can be loaded as follows:

Load Python v3.11.3#
module load Python/3.11.3-GCCcore-12.3.0

You can check which modules are currently loaded with:

module list

Unloading all currently loaded modules can be done like this:

module purge

See also

Helpdesk If you need software that is not yet installed in our cluster, we can install it.

1.2. How can I find specific Python or R packages?#

We extensively use module extensions to handle Python and R packages. Software modules can contain multiple extensions, which are additional software packages necessary or useful by the main software of the module.

Since 2023, the base Python and R modules only contain those extensions needed to build and install extra packages on top of them. Other Python or R packages missing in those modules may be provided in bundle modules or in their own specific modules. For instance, Python has the following bundles available:

  • Python-bundle-PyPI: collection of modules from PyPI

  • SciPy-bundle: collection of modules for scientific Python, including numpy, pandas and scipy

R has its own bundles of modules as well:

  • R-bundle-CRAN: collection of modules from CRAN

  • R-bundle-Bioconductor: collection of modules from Bioconductor

You can check the complete list of extensions in any module with the command module help

Example to list all extensions in the module SciPy-bundle version 2023.07#
module help SciPy-bundle/2023.07-gfbf-2023a

The command module spider can be used to search in Hydra for specific Python or R packages, either as modules or extensions in bundles.

Example to find R package ggplot2#
$ module spider ggplot2
--------------------------------------------------------
  ggplot2:
--------------------------------------------------------
    Versions:
      ggplot2/3.3.6 (E)
      ggplot2/3.4.4 (E)

Names marked by a trailing (E) are extensions provided by another module.

--------------------------------------------------------
  For detailed information about a specific "Seurat" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

    $ module spider ggplot2/3.4.4
--------------------------------------------------------

In the previous example for the R package ggplot2, the command module spider tells the available versions installed in the system. They are all labelled with (E) because they are extensions of a module. You can find the specific module that has a specific version of ggplot2 by executing module spider with the full name of the extension

Example to find the module with extension ggplot2 v3.4.4#
$ module spider ggplot2/3.4.4
--------------------------------------------------------
  ggplot2: ggplot2/3.4.4 (E)
--------------------------------------------------------
    This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.

       R-bundle-CRAN/2023.12-foss-2023a

Names marked by a trailing (E) are extensions provided by another module.

Hence, loading the module R-bundle-CRAN/2023.12-foss-2023a will make ggplot2 v3.4.4 loadable in our R environment.

Helpdesk If you need any R packages that are currently unavailable in Hydra, we can install them.

1.3. How can I find old software?#

Searching for software with module av or module spider will only show software added in the last 2 years, older modules are hidden by default. You can still search, load, and use old/hidden modules, but a warning message will be shown upon loading, suggesting to upgrade to a more recent version.

To search for hidden modules, type:

List all hidden and visible Python modules#
module --show-hidden spider Python

Using the above command, all installed Python modules are shown.

1.4. How can I find legacy software?#

Legacy modules contain all software built for the former operating system. To load those, first load the legacy-software module:

Load legacy-software module#
module load legacy-software

Note

Please be aware that this software stack is unavailable on the zen4 nodes. If you require legacy software, ensure you use the --partition option when submitting your job, directing it to the appropriate partition(s).

Warning

While many legacy-software installations will work, we cannot guarantee their performance or functionality, as the software was built for a different operating system than the one currently used in the cluster. If you must continue using older software that is no longer working, we can provide specific solutions based on containers or, as a last resort, carry out re-installation, please contact VUB-HPC Support for help.

1.5. The toolchain of software packages#

The name of a software package such as TensorFlow/2.13.0-foss-2023a not only contains the name of the package TensorFlow and its version 2.13.0, but also has the term foss-2023a, which is known as the toolchain.

The toolchain foss is the set of tools used to build that package and it has its own version 2023a. Software modules are made available in specific toolchains depending on their requirements. The most common ones are:

GCCcore

Toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada. The GCCcore version corresponds to the version of the GNU Compiler Collection (GCC).

GCC

GCCcore toolchain with additional libraries provided by binutils (libstdc++, libgcj,…).

gfbf

GCC-based compiler toolchain, including FlexiBLAS (BLAS and LAPACK support) and (serial) FFTW.

gompi

GCC-based compiler toolchain, including OpenMPI for MPI support.

foss

The Free and open-source software (FOSS) toolchain is based on GCCcore and also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.

intel

Intel-based toolchain including Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).

iomkl

Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.

Some toolchains offer improved performance on specialised hardware, for instance certain software may be faster with the intel toolchain if used in compute nodes with Intel hardware; whereas other toolchains offer additional features, such as gompi which adds the MPI stack to GCC for inter-node communication.

The name of software packages may contain after their toolchain specification additional information regarding other dependencies of the package, such as needed interpreters. For instance, the package R-bundle-Bioconductor/3.15-foss-2022a-R-4.2.1 corresponds to Bioconductor version 3.15 built with foss toolchain version 2022a and uses R version 4.2.1.

Multiple modules can only be loaded together as long as their toolchains and toolchain versions are compatible. Otherwise, conflicts may occur.

1.5.1. Compatible toolchains#

Modules with different toolchains are generally not compatible, unless one is a sub-toolchain of the other. Hence, a module in foss is not compatible with another one in intel. The table below shows the relation between the base toolchain GCCcore, sub-toolchains, and full toolchains foss and intel:

base toolchain

sub-toolchain

sub-toolchain

full toolchain

GCCcore

GCC

gompi

foss

GCCcore

GCC

gfbf

foss

GCCcore

iccifort

iimpi

intel

1.5.2. Compatible toolchain versions#

Different versions of the same toolchain are always incompatible. For example, you cannot load a module in foss-2022a or GCCcore-11.3.0 with another one in foss-2023a or GCCcore-12.3.0. Please check the foss component table showing the GCC or GCCcore version corresponding to a given foss toolchain.

Helpdesk If you cannot find a compatible set of modules that provides all software required for your work, we can install them.

1.6. European Environment for Scientific Software Installations#

The European Environment for Scientific Software Installations or EESSI for short is a collaboration between different European partners in HPC community.

The goal of this project is to build a common stack of scientific software installations for HPC systems and beyond, including laptops, personal workstations, and cloud infrastructure.

For more information, see the official EESSI documentation, which also includes a list of systems on which EESSI is available

It can be used on Hydra by calling:

export MODULEPATH=/cvmfs/software.eessi.io/init/modules/
module load EESSI/2023.06

After this you can load all EESSI modules. We recommend to avoid mixing EESSI modules with our local modules.

On this page