Software

Hydra

1. Where can I find installed software?

Most end-user software on Hydra is available via modules. To obtain a full list of installed software, type:

module av

To get a list of available versions of a given software package (for example, Python), type:

module spider Python

To get a list of extensions included in a given software package (for example, Python 3.6), type:

module show Python/3.6.6-foss-2018b

Modules can be loaded as follows (in this example Python 3.6 is loaded):

module load Python/3.6.6-foss-2018b

You can check which modules are currently loaded with:

module list

Unloading all currently loaded modules can be done like this:

module purge

If you need software that is not yet installed, please contact us at hpc@vub.ac.be.

More information on module usage can be found in the HPC tutorial and in the HPC training slides.

2. The toolchain of software packages

The name of a software package such as Python/3.6.6-foss-2018b not only contains the name of the package Python and its version 3.6.6, but also has the term foss-2018b, which is known as the toolchain. The toolchain is the set of tools used to build that package and it has its own version 2018b. Any specific software can be made available with different toolchains. The most common ones are

  • GCCcore: The toolchain based on the GNU Compiler Collection, which includes front ends for C, C++, Objective-C, Fortran, Java, and Ada.
  • GCC: The GCCcore toolchain including libraries provided by binutils (libstdc++, libgcj,…).
  • foss: The Free and open-source software (FOSS) toolchain is based on GCCcore and also includes support for OpenMPI, OpenBLAS, FFTW and ScaLAPACK.
  • intel: The Intel compilers, Intel MPI and Intel Math Kernel Library (MKL).
  • iomkl: The Intel C/C++ and Fortran compilers, Intel MKL & OpenMPI.
  • fosscuda: The foss toolchain with support for CUDA.

Some toolchains offer improved performance on specialised hardware, for instance certain software may be faster with the intel toolchain if used in compute nodes with Intel hardware; whereas other toolchains offer additional features, such as fosscuda which is needed to execute code on GPUs.

The name of software packages may contain after the toolchain specification additional information regarding other dependencies of the package, such as needed interpreters. For instance, the package R-bundle-Bioconductor/3.7-foss-2018b-R-3.5.1 corresponds to Bioconductor version 3.7 built with the foss toolchain 2018b and requires R version 3.5.1.

It is important to only load modules built with a common toolchain (including its version), otherwise conflicts may occur. Hence, R-bundle-Bioconductor/3.7-foss-2018b-R-3.5.1 can be loaded along with other packages build with the foss-2018b toolchain, but not any other. The only exception to this rule are packages built with the GCCcore toolchain, which is compatible with both the foss and intel toolchains. Note that GCCcore is not the same as GCC, which is incompatible with the intel toolchain. The compatibility between toolchains is version dependent, the most recent ones being:

  • GCCcore-8.2.0 is compatible with foss-2019a and intel-2019a
  • GCCcore-7.3.0 is compatible with foss-2018b and intel-2018b
  • GCCcore-6.4.0 is compatible with foss-2018a, foss-2017b, intel-2018a and intel-2017b
  • GCCcore-6.3.0 is compatible with foss-2017a and intel-2017a

If you cannot find a compatible set of modules that provides the software required for your work, please contact us at hpc@vub.ac.be.

3. How can I install additional software/packages?

You should first check if the software or package you need is already available on Hydra, either as part of a module, or installed as a separate module.

If the software or package you need is not available, the HPC team recommends to kindly request its installation at hpc@vub.ac.be. This has several advantages:

  1. The HPC team will optimize the compilation for each architecture to make sure that your software/package runs efficiently (and usually much faster than when your install it yourself).
  2. The package will be available for everybody.
  3. The package will be built reproducibly with EasyBuild: important for scientific reproducibility.
  4. Different versions of the software can be installed alongside each other.

If you still want to install additional software/packages yourself, there are several resources available online:

4. How can I run graphical applications?

(TODO)

More information in the VSC Documentation.

5. How can I run MATLAB on Hydra?

MATLAB is available as a module, however it is not recommended to run intensive MATLAB calculations on Hydra: it’s performance is not optimal and parallel execution is not fully supported.

First check which MATLAB versions are available:

module av matlab

Next load a suitable version, for example (take the most recent version):

module load MATLAB/2019a

It is possible to run MATLAB in console mode for quick tests. For example, with a MATLAB script called ‘testmatlab.m’, type:

matlab -nodisplay -r testmatlab

Execution of MATLAB scripts requires compute nodes. In this case, the HPC team highly recommends to first compile your script using the MATLAB compiler mcc:

mcc -m testmatlab.m

This will generate a testmatlab binary file, as well as a ‘run_testmatlab.sh’ shell script (and a few other files). You can ignore the ‘run_testmatlab.sh’ file.

Now you can submit your matlab calculation as a batch job. Your job script should look like this:

#!/bin/bash -l
#PBS -l walltime=01:00:00
#PBS -l nodes=1:ppn=1
#PBS -l mem=1gb

module load MCR/R2019a

cd $PBS_O_WORKDIR
./testmatlab 2>&1 >testmatlab.out

The advantage of running a compiled matlab binary is that it does not require a license. We have only a limited number of MATLAB licenses that can be used at the same time, so in this way you can run your simulation even if the all licenses are in use.

More information on using the MATLAB compiler can be found here:

https://nl.mathworks.com/help/mps/ml_code/mcc.html

6. How can I use GaussView?

GaussView is a graphical interface used with the computational chemistry program Gaussian. If your ssh session is configured with X11 forwarding, you can use GaussView directly on Hydra after loading the module:

ml GaussView/6.0.16

However, using a graphical interface on Hydra is slow, thus for regular use the HPC team recommends to install GaussView locally. Binary packages of GaussView are available for Linux, Mac, and Windows users and are provided upon request.

Installation of GaussView on Mac:

  1. Untar G16 and GaussView to /Applications (Two new dirs, g16 and gv will be created)

  2. Create a file ~/Library/LaunchAgents/env.GAUSS_EXEDIR.plist and paste the following content into it:

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
    <key>Label</key>
    <string>env.GAUSS_EXEDIR</string>
    <key>ProgramArguments</key>
    <array>
    <string>launchctl</string>
    <string>setenv</string>
    <string>GAUSS_EXEDIR</string>
    <string>/Applications/g16/bsd:/Applications/g16</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    </dict>
    </plist>
    
  3. Issue the following command (only once) (or, alternatively, restart the machine):

    launchctl load ~/Library/LaunchAgents/env.GAUSS_EXEDIR.plist
    

7. How can I use R?

Depending on your needs there are different methods to use R in Hydra

  • Interactive sessions for light workloads can be performed in the login node

    1. Login to Hydra

    2. Load the following R module

      module load R/3.5.1-foss-2018b
      
    3. Start R

      R
      

    Note

    If you need a different version of R, make sure to load one with foss in the name. Those versions are based on the GNU open source toolchain and can be used in the login node.

  • Interactive sessions for heavy workloads must be performed in the compute nodes

    1. Login to Hydra

    2. Start an interactive session in a compute node with the following command

      qsub -I
      
    3. Load your R module of choice (preferably a recent version)

      module load R/3.5.1-intel-2018b
      
    4. Start R

      R
      
  • Scripts written in R can be executed with the command Rscript. A minimal job script for R only requires loading the R module and executing your scripts with Rscript

    #!/bin/bash -l
    #PBS -l walltime=01:00:00
    #PBS -l nodes=1:ppn=1
    #PBS -l mem=1gb
    
    module load R/3.5.1-intel-2018b
    
    cd $PBS_O_WORKDIR
    Rscript <path-to-script.R>
    

The quality of the graphics generated by R can be improved by changing the graphical backend to Cairo. Add the following lines in the file ~/.Rprofile to make these changes permanent for your user (create the file ~/.Rprofile if it does not exist)

# Use cairo backend for graphics device
setHook(packageEvent("grDevices", "onLoad"),
    function(...) grDevices::X11.options(type='cairo'))

# Use cairo backend for bitmaps
options(bitmapType='cairo')

8. Packages included in the R library in Hydra

There are already many packages included in the library of R in Hydra. The complete list can be looked up from the shell in Hydra with the following commands

  1. Login to Hydra

  2. List the contents of any R module, for instance R/3.5.1-foss-2018b

    module show R/3.5.1-foss-2018b
    

R packages missing in the library may be provided with their own module. In that case use the module command to search in the repository of Hydra. Please see 1. Where can I find installed software?

Unavailable R packages in Hydra can be requested for installation at our support service. Please send an email to hpc@vub.ac.be and the HPC team will proceed with the installation. See 3. How can I install additional software/packages? for more details.

Developers can compile and install R packages in the local R library of their home directory. However, it is important to note that the microarchitecture of Hydra’s nodes changes from one another and needs to be taken into account to test self-compiled R packages in the nodes.

Note

The packages of a local R library can potentially cause errors due to conflicts with the global R library or due to a version change of R after the installation of local R packages. If you experience errors running R scripts that are related to a failed load of a package, it is helpful to check your script in a clean R environment without a local R library.

  1. Remove all modules and load R

    module purge
    module load R/3.5.1-foss-2018b
    
  2. Enter into a clean R environment (not loading previous workspace)

    R --no-restore
    
  3. Inside the shell of R or at the begining of your R script

    .libPaths('')
    <your R code ...>
    

Vega

(TODO)

VSC Tier-1

(TODO)