1. Module System¶
1.1. Where can I find installed software?¶
Most end-user software on Hydra is available via modules that can be dynamically loaded by the user. That’s why it might seems that the software in Hydra is very limited, but it is not the case, we provide more than 2500 modules. To check the full list of installed software, type:
spider option to list all available versions of a given software package:
module spider Python
show option to list all extensions included in a given module:
module show Python/3.8.6-GCCcore-10.2.0
Modules can be loaded as follows:
module load Python/3.8.6-GCCcore-10.2.0
You can check which modules are currently loaded with:
Unloading all currently loaded modules can be done like this:
Helpdesk If you need software that is not yet installed in our cluster, we can install it.
1.2. How can I find old software?¶
Searching for software with
module av or
module spider will only show software added in the last 2 years, older modules are ‘hidden’ by default. You can still search, load, and use old/hidden modules, but a warning message will be shown upon loading, suggesting to upgrade to a more recent version.
To search for hidden modules, type:
module --show-hidden spider Python
1.3. The toolchain of software packages¶
The name of a software package such as
Python/3.6.6-foss-2018b not only contains the name of the package
Python and its version
3.6.6, but also has the term
foss-2018b, which is known as the toolchain. The toolchain is the set of tools used to build that package and it has its own version
2018b. Any specific software can be made available with different toolchains. The most common ones are
Toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada.
GCCcoretoolchain with additional libraries provided by
The Free and open-source software (FOSS) toolchain is based on
GCCcoreand also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.
Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).
Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.
fosstoolchain with support for CUDA.
Some toolchains offer improved performance on specialised hardware, for instance certain software may be faster with the
intel toolchain if used in compute nodes with Intel hardware; whereas other toolchains offer additional features, such as
fosscuda which is needed to execute code on GPUs.
The name of software packages may contain after the toolchain specification additional information regarding other dependencies of the package, such as needed interpreters. For instance, the package
R-bundle-Bioconductor/3.7-foss-2018b-R-3.5.1 corresponds to Bioconductor version 3.7 built with the
foss toolchain 2018b and requires
R version 3.5.1.
It is important to only load modules built with a common toolchain (including its version), otherwise conflicts may occur. Hence,
R-bundle-Bioconductor/3.7-foss-2018b-R-3.5.1 can be loaded along with other packages build with the
foss-2018b toolchain, but not any other. The only exception to this rule are packages built with the
GCCcore toolchain, which is compatible with both the
intel toolchains. Note that
GCCcore is not the same as
GCC, which is incompatible with the
intel toolchain. The compatibility between toolchains is version dependent, some recent ones being:
GCCcore-10.2.0 is compatible with foss-2020b and intel-2020b
GCCcore-9.3.0 is compatible with foss-2020a and intel-2020a
GCCcore-8.3.0 is compatible with foss-2019b and intel-2019b
GCCcore-8.2.0 is compatible with foss-2019a and intel-2019a
Helpdesk If you cannot find a compatible set of modules that provides all software required for your work, we can install them.
1.4. How can I build/install additional software/packages?¶
You should first check if the software is already available in Hydra, either with its own module or as part of another module. If the software package you need is not available, we strongly recommend to request its installation to VUB-HPC Support. Installations carried out by VUB-HPC have several advantages:
The HPC team will optimize the compilation for each CPU architecture present in Hydra, guaranteeing that your software/package runs efficiently on all nodes (and usually much faster than installations made by the users).
Free software will be available to all users of Hydra and licensed software can be made available to specific groups of users.
The package will be built in a reproducible way with EasyBuild: important for scientific reproducibility.
Different versions of the software can be installed alongside each other.
If you still want to install additional software/packages yourself, you can find guidance for specific development environments in the questions below.
1.4.1. Compiling and testing your software on the HPC¶
We strongly recommend to use a suitable
buildenv module to compile software in Hydra. This ensures that you work in a control and reproducible environment.
module load buildenv/default-foss-2019b
loads the compiler and any math and/or MPI libraries that may be included in the respective toolchain
defines compiler flags for optimal performance: CFLAGS, FFLAGS, CXXFLAGS, LIBBLAS, LIBLAPACK, LIBFFT, …
defines flags and paths to make sure the build system finds the right libraries: LIBRARY_PATH, LD_LIBRARY_PATH, LDFLAGS, …
It is possible to load additional development tools compatible with the buildenv module. For instance, the following modules can be loaded alongside
Users compiling their own software should be aware that software compiled on the login nodes may fail in older compute nodes if full hardware optimization is used. The CPU microarchitecture of the login nodes (Skylake) has some instruction sets not available in Hydra’s older compute nodes (Ivy Bridge and Broadwell). Therefore, there are two options to compile your own software
- Best performance
Compile on the login node (with
-march=native). The resulting binaries can only run on Skylake nodes, but they offer the best performance on those nodes. Jobs can be restricted to run on Skylake nodes with the qsub option
- Best compatibility
Compile on any Ivy Bridge node. Login to an Ivy Bridge node with
qsub -I -l feature=ivybridgeand compile your code on it. The resulting binaries can run on any node on Hydra with good performance. Alternatively, users knowing how to setup the compilation can compile on the login node with
Helpdesk The environment in the HPC might differ significantly from your development system. We can help you in case of problems or questions to transfer your development to the HPC.
VSC Docs: Software development for more information.
1.4.2. Installing additional Python packages¶
Helpdesk There is a large number of Python packages already available in Hydra, see for example the packages included in
Scipy-bundle modules. If the package you need is not available, we can install it for you.
If you would like to test some new Python package before requesting its installation, you can do so by using your personal
site-packagesdirectory in your home:
Load the appropriate Python module, i.e. the specific Python version to use with the package:
module load Python/3.8.6-GCCcore-10.2.0
Install the Python package with
pipin your user account. The following command will also download and install all required dependencies. The new package and all missing dependencies will be installed by default in
pip install --user <new_python_package>
Developers, who require using/testing software in Python that is in active development, can also use
pipto install their own packages in a personal
Load the appropriate Python module and install the Python package from the local directory containing the source code. In this case
pipwill also install any missing dependencies:
module load Python/3.8.6-GCCcore-10.2.0 pip install --user /path/to/your/source/code
Optional Updating the installation can be done at any time with the command:
pip install --user --no-deps --ignore-installed /path/to/your/source/code
1.4.3. Installing additional R packages¶
There are already many packages included in the library of R in Hydra. The complete list can be looked up for each R version in Hydra with the command
module show R/4.0.0-foss-2020a
R packages missing in the main R module may be provided with their own module. Use the
module command to search in the repository of Hydra. Please see Where can I find installed software?
Helpdesk If you need any R packages that are currently unavailable in Hydra, we can install them.
Developers can compile and install R packages in the local R library of their home directory. The R function
install.packages() will specifically ask to use your personal library. Keep in mind that if your software requires code compilation beyond R, you might need a build environment as described in Compiling and testing your software on the HPC.
The packages of your personal R library can potentially break the rest of R packages provided in Hydra. This is due to conflicts with the global R library or due to a version change of R after the installation of local R packages. If you experience errors running R scripts that are related to a failed load of a package, it is helpful to check your script in a clean R environment without your personal R library:
Remove all modules and load the desired version of R:
module purge module load R/4.0.0-foss-2020a
Enter into a clean R environment (not loading previous workspace):
.libPathsinside the shell of R or at the beginning of your R script to disable your personal R library:
$ R > .libPaths('') > your_code...
1.5. How can I run graphical applications?¶
1.5.1. X11 forwarding¶
X11 forwarding is the simplest solution, but also the slowest. This option is suitable for applications with minimal graphical requirements.
Linux and MacOS
Users can enable X11 forwarding upon login by adding the option
ssh -Y firstname.lastname@example.org
Users can take advantage of the built-in X11 support in MobaXterm, see VSC Docs: Setup MobaXterm. By default X11 forwarding is enabled; you can check this setting under the Advanced SSH settings tab in the SSH Session settings window.
After login, you can test whether X11 is enabled by running the command
xclock in the login node. If it works, a clock will be displayed in a new window. Please keep in mind that the login node is restricted to graphical applications that perform non-intensive compute tasks, such as visualization tools.
Compute intensive graphical apps should be executed in a compute node. This can be achieved by starting an interactive job and adding the option
-X to the
qsub -I -X
To avoid having to keep your SSH connection active for the duration of the interactive job, you can submit your interactive job in a tmux session.
1.5.2. Remote desktop sharing¶
Alert Setting up remote desktop sharing is a complex multi-step process, recommended only for advanced users.
For applications that are graphics-heavy we provide an alternative solution to X11 forwarding that is more efficient, called Virtual Network Computing (VNC). The steps are outlined below.
Create a VNC password
The VNC password should be created only once and it will be used in all future connections to your VNC sessions in Hydra. Type the following command to set the password:
When asked about a view-only password, reply no. The entered password is obfuscated and stored by default in the file
Start a VNC server
The following command starts a VNC server inside a job in a compute node. Adapt the requested resources and the screen resolution as needed:
(echo vncserver -geometry 1600x900 -fg) | qsub -l walltime=12:0:0 -N VNC -e VNC.err -
Verify that the VNC server has successfully started
Once the VNC job has started, look for a line that starts with ‘New’ in your job error file (
grep '^New' VNC.err
If your VNC session is running, then a line similar to the following should be shown:
$ grep '^New' VNC.err New 'node379.hydra.os:1 (vsc10xxx)' desktop is node379.hydra.os:1
Take note of the
compute nodeand the
display numberof your VNC session, which will be needed in the next steps. In the example above with
node379.hydra.os:1, we have:
compute node: node379
display number: 1
Start a VNC client in your local computer
Linux and macOS
Linux and macOS users can use TigerVNC from the command line, which allows connecting via the gateway (the login node) in a single command. Linux users can install it via their package manager; macOS users can install it via Homebrew.
Adapt the following command: replace node379:1 with the compute node and display number of your VNC session:
vncviewer node379:1 -via email@example.com -maximize
A new window asking your password should pop up. Enter your VNC server password. The VNC session should now appear in a new window.
Windows users can use the VNC client that is included in recent MobaXterm versions.
Sessionin the top toolbar to create a new session. A new Session settings window should pop up:
VNCin the top toolbar.
Basic VNC settings, fill out:
Advanced settingsand check
Use new embedded VNC engine.
Network settingsand click the rectangular bar
SSH gateway (jump host). A new window MobaXterm jump hosts configuration should pop up:
Gateway host: login.hpc.vub.be
Username: vsc10xxx (replace with your VSC-ID)
Use SSH keyand click the little file icon to browse to your private SSH key.
Back in the window Session settings, click
OK. A new window asking your password should pop up.
Enter your VNC server password. The VNC session should now appear in a new tab inside the MobaXterm interface: