2. Slurm clusters#

2.1. Hydra: main cluster#

The Hydra cluster has multiple job queues: one for each partition. Jobs are automatically assigned to those partitions in the cluster that have the computational resources needed to fulfil the requirements of the job. For instance, jobs requesting a GPU will automatically queue to compute nodes with a GPU, and jobs requesting more than 1 node will automatically queue on a partition with the fastest network interconnect.

Important

Jobs in Hydra can run for a maximum of 120 hours (5 days)

Jobs submitted from the login nodes will be sent to job queue of Hydra by default.

2.2. Anansi: test/debug cluster#

Jobs for interactive use, testing, and debugging can be run on the Anansi cluster. As Hydra’s smaller sister cluster, Anansi is designed for short tasks and provides minimal queue times.

The main characteristics of Anansi are:

  • CPU cores can be shared by up to 4 jobs

  • GPU devices can be shared by up to 4 jobs

  • Maximum number of GPU devices per job is 1 (100% GPU fraction)

  • Maximum job walltime is 12 hours

In contrast to Hydra, where computational resources are allocated exclusively to single jobs, the resources of Anansi can be shared between multiple jobs. This applies to CPU cores which are shared transparently, and to GPUs which can be requested in fractions of a full device. Other resources like CPU memory are not shared.

This approach is specially useful for light-weight (interactive) tasks such as testing/debugging that can involve frequent idle periods. Hence, even though the size of Anansi is smaller than Hydra, users should be able to quickly find an available slot to run their short jobs.

Moreover, the maximum time a job can run in Anansi is limited to 12 hours, which further increases the availability of its resources.

Jobs can be submitted, managed and monitored in Anansi from the login nodes of Hydra. The target cluster of any Slurm command can be changed to Anansi by adding the option -M anansi to the command arguments or as a #SBATCH option in the header of your job script.

Command to check the job queue in Anansi#
$ mysqueue -M anansi
CLUSTER: anansi
     JOBID PARTITION   NAME          USER     STATE  TIME TIME_LIMIT NODES CPUS MIN_MEMORY NODELIST(REASON)
  50005540 ada_gpu     test.job  vsc10000   RUNNING  0:05      10:00     1    4         6G node510
On this page