1. Module System#

1.1. Where can I find installed software?#

Most end-user software on Hydra is available via modules that can be dynamically loaded by the user. That’s why it might seems that the software in Hydra is very limited, but it is not the case, we provide more than 2500 modules. To check the full list of installed software, type:

module av

Use the spider option to list all available versions of a given software package:

List all modules of Python#
module spider Python

Use the show option to list all extensions included in a given module:

List all extensions in the module of Python v3.9.5#
module show Python/3.9.5-GCCcore-10.3.0

Modules can be loaded as follows:

Load Python v3.9.5#
module load Python/3.9.5-GCCcore-10.3.0

You can check which modules are currently loaded with:

module list

Unloading all currently loaded modules can be done like this:

module purge

Helpdesk If you need software that is not yet installed in our cluster, we can install it.

1.2. How can I find specific Python or R packages?#

We extensively use module extensions to handle Python and R packages. Software modules can contain multiple extensions, which are additional software packages necessary or useful by the main software of the module.

There are already many packages included in the main modules of Python and R in Hydra. You can check the complete list of extensions in each module with the command module show

Example to list all extensions in the module of R v4.1.0#
module show R/4.1.0-foss-2021a

Python or R packages missing in the main modules may be provided in other modules or even their own module. Use the command module spider to search in the repository of Hydra for specific Python or R packages.

Example to find R package Seurat#
$ module spider Seurat
--------------------------------------------------------
Seurat:
--------------------------------------------------------
Versions:
Seurat/3.1.5-foss-2019b-R-3.6.2
Seurat/3.1.5 (E)
Seurat/4.0.0 (E)
Seurat/4.0.3 (E)

Names marked by a trailing (E) are extensions provided by another module.

--------------------------------------------------------
For detailed information about a specific "Seurat" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:

$ module spider Seurat/4.0.3
--------------------------------------------------------

In the previous example for the R package Seurat, module spider tells that there is a specific module for this software called Seurat/3.1.5-foss-2019b-R-3.6.2, which can be loaded right away. All other results labelled with (E) are extensions in other modules. In those cases, you can find the specific module with Seurat by executing module spider with the full name of the extension

Example to find the module with extension Seurat/4.0.3#
$ module spider Seurat/4.0.3
--------------------------------------------------------
  Seurat: Seurat/4.0.3 (E)
--------------------------------------------------------
    This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.


       R-bundle-Bioconductor/3.13-foss-2021a-R-4.1.0


Names marked by a trailing (E) are extensions provided by another module.

Hence, loading the module R-bundle-Bioconductor/3.13-foss-2021a-R-4.1.0 will make Seurat v4.0.3 loadable in our R environment.

Helpdesk If you need any R packages that are currently unavailable in Hydra, we can install them.

1.3. How can I find old software?#

Searching for software with module av or module spider will only show software added in the last 2 years, older modules are hidden by default. You can still search, load, and use old/hidden modules, but a warning message will be shown upon loading, suggesting to upgrade to a more recent version.

To search for hidden modules, type:

List all hidden and visisble Python modules#
module --show-hidden spider Python

1.4. The toolchain of software packages#

The name of a software package such as SciPy-bundle/2021.05-foss-2021a not only contains the name of the package SciPy-bundle and its version 2021.05, but also has the term foss-2021a, which is known as the toolchain. The toolchain foss is the set of tools used to build that package and it has its own version 2021a. Any specific software can be made available with different toolchains. The most common ones are

GCCcore

Toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada.

GCC

GCCcore toolchain with additional libraries provided by binutils (libstdc++, libgcj,…).

foss

The Free and open-source software (FOSS) toolchain is based on GCCcore and also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.

intel

Intel-based toolchain including Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).

iomkl

Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.

fosscuda

The foss toolchain with support for CUDA.

Some toolchains offer improved performance on specialised hardware, for instance certain software may be faster with the intel toolchain if used in compute nodes with Intel hardware; whereas other toolchains offer additional features, such as fosscuda which is needed to execute code on GPUs.

The name of software packages may contain after the toolchain specification additional information regarding other dependencies of the package, such as needed interpreters. For instance, the package R-bundle-Bioconductor/3.13-foss-2021a-R-4.1.0 corresponds to Bioconductor version 3.13 built with foss toolchain version 2021a and requires R version 4.1.0.

Multiple modules can only be loaded together as long as both their toolchains and toolchain versions are compatible. Otherwise, conflicts may occur.

  1. Compatible toolchains

    Modules with different toolchains are generally not compatible, unless one is a sub-toolchain of the other. Hence, a module in foss is not compatible with another one in intel. The table below shows the relation between ‘base’ toolchain GCCcore, sub-toolchains, and ‘full’ toolchains foss and intel:

    base toolchain

    sub-toolchain

    sub-toolchain

    full toolchain

    GCCcore

    GCC

    gompi

    foss

    GCCcore

    iccifort

    iimpi

    intel

  2. Compatible toolchain versions

    Different versions of the same toolchain are always incompatible. Hence, you cannot load a module in foss-2020a with another one in foss-2020b. Please check the compatibility table below showing recent ‘base’ and ‘full’ toolchain versions:

    base toolchain

    compatible with

    GCCcore-10.3.0

    foss-2021a and intel-2021a

    GCCcore-10.2.0

    foss-2020b and intel-2020b

    GCCcore-9.3.0

    foss-2020a and intel-2020a

    GCCcore-8.3.0

    foss-2019b and intel-2019b

    GCCcore-8.2.0

    foss-2019a and intel-2019a

Helpdesk If you cannot find a compatible set of modules that provides all software required for your work, we can install them.