1. Module System#
1.1. Where can I find installed software?#
Most end-user software on Hydra is available via modules that can be dynamically loaded by the user. That’s why it might seems that the software in Hydra is very limited, but it is not the case, we provide more than 2500 modules. To check the full list of installed software, type:
module av
Use the spider
option to list all available versions of a given software
package:
module spider Python
Use the help
option to list all extensions included in a given module:
module help Python/3.11.3-GCCcore-12.3.0
Modules can be loaded as follows:
module load Python/3.11.3-GCCcore-12.3.0
You can check which modules are currently loaded with:
module list
Unloading all currently loaded modules can be done like this:
module purge
See also
Helpdesk If you need software that is not yet installed in our cluster, we can install it.
1.2. How can I find specific Python or R packages?#
We extensively use module extensions to handle Python and R packages. Software modules can contain multiple extensions, which are additional software packages necessary or useful by the main software of the module.
Since 2023, the base Python and R modules only contain those extensions needed to build and install extra packages on top of them. Other Python or R packages missing in those modules may be provided in bundle modules or in their own specific modules. For instance, Python has the following bundles available:
Python-bundle-PyPI
: collection of modules from PyPISciPy-bundle
: collection of modules for scientific Python, includingnumpy
,pandas
andscipy
R has its own bundles of modules as well:
R-bundle-CRAN
: collection of modules from CRANR-bundle-Bioconductor
: collection of modules from Bioconductor
You can check the complete list of extensions in any module with the
command module help
module help SciPy-bundle/2023.07-gfbf-2023a
The command module spider
can be used to search in Hydra for
specific Python or R packages, either as modules or extensions in bundles.
$ module spider ggplot2
--------------------------------------------------------
ggplot2:
--------------------------------------------------------
Versions:
ggplot2/3.3.6 (E)
ggplot2/3.4.4 (E)
Names marked by a trailing (E) are extensions provided by another module.
--------------------------------------------------------
For detailed information about a specific "Seurat" package (including how to load the modules) use the module's full name.
Note that names that have a trailing (E) are extensions provided by other modules.
For example:
$ module spider ggplot2/3.4.4
--------------------------------------------------------
In the previous example for the R package ggplot2
, the command module
spider
tells the available versions installed in the system. They are all
labelled with (E) because they are extensions of a module. You can
find the specific module that has a specific version of ggplot2
by executing
module spider
with the full name of the extension
$ module spider ggplot2/3.4.4
--------------------------------------------------------
ggplot2: ggplot2/3.4.4 (E)
--------------------------------------------------------
This extension is provided by the following modules. To access the extension you must load one of the following modules. Note that any module names in parentheses show the module location in the software hierarchy.
R-bundle-CRAN/2023.12-foss-2023a
Names marked by a trailing (E) are extensions provided by another module.
Hence, loading the module R-bundle-CRAN/2023.12-foss-2023a
will
make ggplot2 v3.4.4 loadable in our R environment.
Helpdesk If you need any R packages that are currently unavailable in Hydra, we can install them.
1.3. How can I find old software?#
Searching for software with module av
or module spider
will only show
software added in the last 2 years, older modules are hidden by default. You
can still search, load, and use old/hidden modules, but a warning message will
be shown upon loading, suggesting to upgrade to a more recent version.
To search for hidden modules, type:
module --show-hidden spider Python
Using the above command, all installed Python modules are shown.
1.4. How can I find legacy software?#
Legacy modules contain all software built for the former operating system. To
load those, first load the legacy-software
module:
module load legacy-software
Warning
While many legacy-software installations will work, we cannot guarantee their performance or functionality, as the software was built for a different operating system than the one currently used in the cluster. If you must continue using older software that is no longer working, we can provide specific solutions based on containers or, as a last resort, carry out re-installation, please contact VUB-HPC Support for help.
1.5. The toolchain of software packages#
The name of a software package such as TensorFlow/2.13.0-foss-2023a
not
only contains the name of the package TensorFlow
and its version
2.13.0
, but also has the term foss-2023a
, which is known as the
toolchain.
The toolchain foss
is the set of tools used to build that package and it
has its own version 2023a
. Software modules are made available in specific
toolchains depending on their requirements. The most common ones are:
- GCCcore
Toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada.
- GCC
GCCcore
toolchain with additional libraries provided bybinutils
(libstdc++, libgcj,…).- gfbf
GNU Compiler Collection (GCC) based compiler toolchain, including FlexiBLAS (BLAS and LAPACK support) and (serial) FFTW.
- gompi
GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support.
- foss
The Free and open-source software (FOSS) toolchain is based on
GCCcore
and also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.- intel
Intel-based toolchain including Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).
- iomkl
Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.
Some toolchains offer improved performance on specialised hardware, for instance
certain software may be faster with the intel
toolchain if used in compute
nodes with Intel hardware; whereas other toolchains offer additional features,
such as gompi
which adds the MPI stack to GCC
for inter-node
communication.
The name of software packages may contain after their toolchain specification
additional information regarding other dependencies of the package, such as
needed interpreters. For instance, the package
R-bundle-Bioconductor/3.15-foss-2022a-R-4.2.1
corresponds to Bioconductor
version 3.15 built with foss
toolchain version 2022a and uses R
version 4.2.1.
Multiple modules can only be loaded together as long as their toolchains and toolchain versions are compatible. Otherwise, conflicts may occur.
- Compatible toolchains
Modules with different toolchains are generally not compatible, unless one is a sub-toolchain of the other. Hence, a module in
foss
is not compatible with another one inintel
. The table below shows the relation between the base toolchainGCCcore
, sub-toolchains, and full toolchainsfoss
andintel
:base toolchain
sub-toolchain
sub-toolchain
full toolchain
GCCcore
GCC
gompi
foss
GCCcore
GCC
gfbf
foss
GCCcore
iccifort
iimpi
intel
- Compatible toolchain versions
Different versions of the same toolchain are always incompatible. Hence, you cannot load a module in
foss-2022a
with another one infoss-2023a
. Please check the compatibility table below showing recent baseGCCcore
toolchain versions:toolchain version
compatible with
2023a
GCCcore-12.3.0
2022a
GCCcore-11.3.0
Helpdesk If you cannot find a compatible set of modules that provides all software required for your work, we can install them.