1. Module System#

1.1. Where can I find installed software?#

Most end-user software on Hydra is available via modules that can be dynamically loaded by the user. That’s why it might seems that the software in Hydra is very limited, but it is not the case, we provide more than 2500 modules. To check the full list of installed software, type:

module av

Use the spider option to list all available versions of a given software package:

List all modules of Python#

module spider Python

Use the help option to list all extensions included in a given module:

List all extensions in the module of Python v3.11.3#

module help Python/3.11.3-GCCcore-12.3.0

Modules can be loaded as follows:

Load Python v3.11.3#

module load Python/3.11.3-GCCcore-12.3.0

You can check which modules are currently loaded with:

module list

Unloading all currently loaded modules can be done like this:

module purge

1.2. How can I find specific Python or R packages?#

We extensively use module extensions to handle Python and R packages. Software modules can contain multiple extensions, which are additional software packages necessary or useful by the main software of the module.

Since 2023, the base Python and R modules only contain those extensions needed to build and install extra packages on top of them. Other Python or R packages missing in those modules may be provided in bundle modules or in their own specific modules. For instance, Python has the following bundles available:

Python-bundle-PyPI: collection of modules from PyPI
SciPy-bundle: collection of modules for scientific Python, including numpy, pandas and scipy

R has its own bundles of modules as well:

R-bundle-CRAN: collection of modules from CRAN
R-bundle-Bioconductor: collection of modules from Bioconductor

You can check the complete list of extensions in any module with the command module help

Example to list all extensions in the module SciPy-bundle version 2023.07#

module help SciPy-bundle/2023.07-gfbf-2023a

The command module spider can be used to search in Hydra for specific Python or R packages, either as modules or extensions in bundles.

Example to find R package ggplot2#

$ module spider ggplot2
--------------------------------------------------------
  ggplot2:
--------------------------------------------------------
    Versions:
      ggplot2/3.3.6 (E)
      ggplot2/3.4.4 (E)

Names marked by a trailing (E) are extensions provided by another module.

--------------------------------------------------------
  For detailed information about a specific "Seurat" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

    $ module spider ggplot2/3.4.4
--------------------------------------------------------

In the previous example for the R package ggplot2, the command module spider tells the available versions installed in the system. They are all labelled with (E) because they are extensions of a module. You can find the specific module that has a specific version of ggplot2 by executing module spider with the full name of the extension

Example to find the module with extension ggplot2 v3.4.4#

$ module spider ggplot2/3.4.4
--------------------------------------------------------
  ggplot2: ggplot2/3.4.4 (E)
--------------------------------------------------------
    This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.

       R-bundle-CRAN/2023.12-foss-2023a

Names marked by a trailing (E) are extensions provided by another module.

Hence, loading the module R-bundle-CRAN/2023.12-foss-2023a will make ggplot2 v3.4.4 loadable in our R environment.

Helpdesk If you need any R packages that are currently unavailable in Hydra, we can install them.

1.3. How can I find old software?#

Searching for software with module av or module spider will only show software added in the last 2 years, older modules are hidden by default. You can still search, load, and use old/hidden modules, but a warning message will be shown upon loading, suggesting to upgrade to a more recent version.

To search for hidden modules, type:

List all hidden and visible Python modules#

module --show-hidden spider Python

Using the above command, all installed Python modules are shown.

1.4. How can I find legacy software?#

Legacy modules contain all software built for the former operating system. To load those, first load the legacy-software module:

Load legacy-software module#

module load legacy-software

Warning

While many legacy-software installations will work, we cannot guarantee their performance or functionality, as the software was built for a different operating system than the one currently used in the cluster. If you must continue using older software that is no longer working, we can provide specific solutions based on containers or, as a last resort, carry out re-installation, please contact VUB-HPC Support for help.

1.5. The toolchain of software packages#

The name of a software package such as TensorFlow/2.13.0-foss-2023a not only contains the name of the package TensorFlow and its version 2.13.0, but also has the term foss-2023a, which is known as the toolchain.

The toolchain foss is the set of tools used to build that package and it has its own version 2023a. Software modules are made available in specific toolchains depending on their requirements. The most common ones are:

GCCcore: Toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada.
GCC: GCCcore toolchain with additional libraries provided by binutils (libstdc++, libgcj,…).
gfbf: GNU Compiler Collection (GCC) based compiler toolchain, including FlexiBLAS (BLAS and LAPACK support) and (serial) FFTW.
gompi: GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support.
foss: The Free and open-source software (FOSS) toolchain is based on GCCcore and also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.
intel: Intel-based toolchain including Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).
iomkl: Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.

Some toolchains offer improved performance on specialised hardware, for instance certain software may be faster with the intel toolchain if used in compute nodes with Intel hardware; whereas other toolchains offer additional features, such as gompi which adds the MPI stack to GCC for inter-node communication.

The name of software packages may contain after their toolchain specification additional information regarding other dependencies of the package, such as needed interpreters. For instance, the package R-bundle-Bioconductor/3.15-foss-2022a-R-4.2.1 corresponds to Bioconductor version 3.15 built with foss toolchain version 2022a and uses R version 4.2.1.

Multiple modules can only be loaded together as long as their toolchains and toolchain versions are compatible. Otherwise, conflicts may occur.

Compatible toolchains

Modules with different toolchains are generally not compatible, unless one is a sub-toolchain of the other. Hence, a module in foss is not compatible with another one in intel. The table below shows the relation between the base toolchain GCCcore, sub-toolchains, and full toolchains foss and intel:

base toolchain	sub-toolchain	sub-toolchain	full toolchain
GCCcore	GCC	gompi	foss
GCCcore	GCC	gfbf	foss
GCCcore	iccifort	iimpi	intel

Compatible toolchain versions

Different versions of the same toolchain are always incompatible. Hence, you cannot load a module in foss-2022a with another one in foss-2023a. Please check the compatibility table below showing recent base GCCcore toolchain versions:

toolchain version	compatible with
2023a	GCCcore-12.3.0
2022a	GCCcore-11.3.0

Helpdesk If you cannot find a compatible set of modules that provides all software required for your work, we can install them.