VUB-HPC Posts

Security lockdown of VUB-HPC clusters

2026-07-06T00:00:00+00:00

Updated on 06/07/2026

14:00 CEST: Operations on Tier-1 cluster sofia are complete and the system is now available to users. There is no indication that any breach of security has occurred in any of VUB-HPC systems.

Updated on 06/07/2026

09:00 CEST: We already patched the issue on the Tier-2 clusters (Hydra and Anansi) ahead of schedule. Both Tier-2 clusters are again accessible. All jobs that could not finished before Monday morning have been re-queued.

Updated on 05/07/2026

17:00 CEST: We are in the process of deploying the required system updates to fix the security issues disclosed on Friday.

Hydra and Anansi are expected to be fully online by Monday July 6th at 17:00 CEST

Sofia is expected to be fully online by Monday July 6th at 12:00 CEST

Access to the HPC clusters of VUB (Hydra, Anansi) and the VSC Tier-1 cluster sofia is closed for security reasons until further notice.

A new critical vulnerability was just announced that affects our systems. We decided to quickly close all access to our HPC clusters preemptively to avoid any damage on the system and user’s data in it. At the time of closure, no intrusion in our HPC clusters has occurred.

This measure includes the following services:

SSH connections through the terminal interface
OnDemand sessions through portal.hpc.vub.be
Remote sessions with tunneling such as VS Code applications

All active sessions in our clusters have been terminated as well. Compute nodes and the job scheduler will continue to process jobs in the queue. The only jobs canceled are those that involve remote sessions with tunneling through OnDemand.

Users that urgently need to access their data, can do so through the Globus platform. Please check our documentation on VSCdocGlobus if you are not familiar with it.

We are currently working on solving this new critical vulnerability on all our systems as soon as possible. However, we cannot yet provide a timeline for the complete resolution of this issue as we still need to complete our investigation on it. We will keep this post up to date with any progress on this issue.

For any comments or questions, please contact us at VUB-HPC Support.

System update rollout on VUB-HPC clusters

2026-05-22T00:00:00+00:00

We are updating the operating system and Slurm scheduler on Hydra, Anansi and sofia with the latest security and bug fixes. The rollout will start on Friday May 22nd at 14:00 CEST and will be gradually carried out on all VUB-HPC clusters in the following days. This operation has a minimal impact on our users: compute capacity will be slightly lower during the update, but there will be no downtime of the clusters and all job queues will remain operational.

On Saturday 23rd at 03:00 CEST we will also update the login nodes of Hydra and Anansi. During this upgrade, access to the clusters will be unavailable to users for approximately 30 minutes. All user sessions on the login nodes will be disconnected, but running jobs will not be impacted.

This update of the operating system brings the compute nodes to the most recent version of Rocky Linux, including all current security updates and new versions of low-level drivers for the storage, network and GPUs. In addition to the OS upgrade, Slurm will also receive a minor upgrade with several fixes and some improvements to the job scheduler. There will be no changes to your job scripts or how you use the cluster.

These operations are part of the regular updates to keep VUB-HPC infrastructure modern and up to current standards. Please contact us at VUB-HPC Support if you have any comments or questions about these changes.

19 May 2026: Online information session for Tier-1 project applications

2026-05-12T00:00:00+00:00

We are organizing an online information session for VUB users interested in submitting a Tier-1 project application. It takes place online on 19 May at 14:00. Everyone is welcome to join the session via Microsoft Teams using the following link: https://teams.microsoft.com/meet/342738294056735?p=zJOKmdywWZdBwyIZwM

The next Tier-1 application cutoff date, 1 June, is approaching quickly. We encourage all big HPC users to consider submitting a project application. Please contact vsc-support@hpc.vub.be if you would like access to our new sofia Tier-1 cluster to run benchmarks in preparation for the 1 June application cutoff. If the 1 June deadline is too soon, you can already start preparing for the next cutoff date on 5 October.

More information at https://vscentrum.be/compute

Helpdesk In case of questions, please contact the VUB HPC team.

Scheduled maintenance of VSC account page

2026-05-05T00:00:00+00:00

On Monday May 11, scheduled maintenance will be carried out on the VSC account infrastructure starting at 07:00 CEST. The expected duration of the maintenance is approximately 1–2 hours.

During this period, the VSC account page may be unavailable. As a result, other services will also be impacted. This includes the HPC portal and the VSC firewall interface, which may be temporarily inaccessible during the maintenance window.

Running jobs and SSH connections to the clusters are not affected by this maintenance.

We recommend avoiding account-related actions during this time window.

If you have any further questions, please contact us at VUB-HPC Support.

VSC Link briefly down on 6 May

2026-04-30T00:00:00+00:00

Due to a scheduled migration of the VSC link, connectivity between VUB-HPC and other VSC hubs will be temporarily suspended on 6 May between 9:00 and 10:00. This should not take longer than a few minutes. Jobs running on your home institution clusters are unaffected. Jobs running on remote VSC clusters will also continue to run, unless they require access to $VSC_DATA or $VSC_HOME.

If you have any further questions, please contact us at VUB-HPC Support.

Hydra and Sofia security shutdown

2026-05-04T00:00:00+00:00

Updated on 04/05/2026

10:30 CEST: Access to VSC web services and updates to hpc.vub.be are fully restored. These services have been unavailable for the past two days as the authentication infrastructure behind them required downtime to fully recover from this security vulnerability.

Updated on 30/04/2026

20:30 CEST: Access through the OnDemand web portal portal.hpc.vub.be restored.

Updated on 30/04/2026

19:00 CEST: Access to Hydra is open again through the terminal interface of the login nodes (SSH)

All critical systems have been patched with the necessary security updates, and we have no indication that any breach of security has occurred.

During this downtime, we had to re-queue all jobs running in the cluster. It might be that your jobs have already finished, or that they are running but in previous state, or maybe still in queueu. Some job scripts might have also failed to handle the re-queue properly, as this is not a common situation to go through for a job. Please check the output of your jobs carefully.

We are currently working to re-open the OnDemand service at portal.hpc.vub.be as well. This will happen in the next 24h.

If you experience any new issues or unexpected behaviour with the updated system, please contact us at VUB-HPC Support.

Access to the HPC clusters of VUB (Hydra, Anansi) and the VSC Tier-1 cluster Sofia is closed for security reasons until further notice.

This measure affects all means of accessing our HPC infrastructure:

SSH connections through the terminal interface
OnDemand sessions through portal.hpc.vub.be
Remote sessions with tunneling such as VS Code applications

Users that urgently need to access their data, can do so through the Globus platform. Please check our documentation on VSCdocGlobus if you are not familiar with it.

For any comments or questions, please contact us at VUB-HPC Support.

VUB-HPC briefly offline on 4 May

2026-04-28T00:00:00+00:00

Due to a scheduled network maintenance, the VUB-HPC clusters will be temporarily inaccessible on 4 May between 9:00 and 10:00. This should not take longer than a few minutes. Users will be unable to access the clusters during the migration. Running jobs will continue to run, provided they do not require external internet access.

If you have any further questions, please contact us at VUB-HPC Support.

VSC Accountpage problems

2026-04-17T00:00:00+00:00

Updated on 17/04/2026

10:30 The accountpage issue has been fixed. All users should be able to access our services normally again.

Since yesterday afternoon (April 16th 2026) the VSC accountpage will return an error about ‘Invalid account’ or ‘Error 500’ for some VUB users. We are aware and looking for a solution. You can still login through ssh.

We appreciate your patience during this time and apologize for the inconvenience. Please contact VUB-HPC Support if you need any comments or need further help with this situation.

LiteLLM python package compromised

2026-03-27T00:00:00+00:00

On March 24, 2026, the popular LiteLLM python package was compromised by a supply chain attack. If you are using LiteLLM in any form, please take immediate action and check if any of the compromised versions (v1.82.7 or v1.82.8) are installed. See the LiteLLM blog post on the incident for all the details on how to check if you are affected and what to do if that’s the case: https://docs.litellm.ai/blog/security-update-march-2026.

Beware that you are affected if a compromised LiteLLM version was installed on any system that you have access to, whether that’s your laptop, a local server, a pipeline that builds or deploys software, or a HPC cluster. The attack does not apply to any of the centrally installed software modules or containers on the VUB-HPC clusters, as LiteLLM has not been centrally installed yet.

If you have any further questions, please contact us at VUB-HPC Support.

Improved way to launch interactive jobs

2026-03-24T00:00:00+00:00

We’ve changed the recommended way to launch an interactive job on a compute node from the terminal interface. Previously, this was done using srun --pty bash -l. We now recommend using salloc, which makes it possible to launch parallel MPI jobs with srun or mpirun from within the interactive session. All related documentation has been updated accordingly, see the section on Interactive jobs for details.

If you have any further questions, please contact us at VUB-HPC Support.