Irregular low performance of Hydra’s new storage

Updated on 21/04/2022

16:00 We have pinned this issue down to a specific configuration of the login nodes. Today, April 21st, at 21:00 (CEST) we will apply an urgent fix to the login nodes that will increase the filesystem perfomance of VSC_HOME and VSC_DATA back to normal levels. The login nodes will be rebooted during the process and they will be unavailable for 10 min.

The new storage of Hydra that serves VSC_HOME and VSC_DATA is showing very low performance under certain circumstances. This issue is mostly affecting file operations with many small files, whereas data transfers of big files show good transfer rates. Moreover, not all nodes are equally affected. For instance, the login nodes show worse performance than compute nodes.

We are currently investigating the cause of this issue. In the meantime, users having trouble to carry out their workflows in Hydra can use VSC_SCRATCH. The scratch storage is as fast as before the move of the cluster.

We will update this post with any new information as the investigation goes on.

If you have any questions or a specific workflow that is much slower than before, please contact VUB-HPC Support.