Virtual Organization#

A Virtual Organization (VO) is a special type of group. The members of a VO get access to extra storage in the HPC cluster with shared directories between them to easily collaborate with their colleagues. Any user can only be a member of a single VO in each VSC institution.

VO directories#

Members of the VO have access to additional directories in the scratch and data storage of the cluster.

VO Data

Location

$VSC_DATA_VO, $VSC_DATA_VO_USER

Purpose

Storage of datasets or resulting data that must be stored in the cluster to carry out further computational jobs, and that can be shared with co-workers.

Availability: Very high

Accessible from login nodes, compute nodes and from all clusters in VSC.

Capacity: High and expandable

By default, 112.5 GB (soft limit), 125.0 GB (hard limit, 7 days grace time). Can be expanded upon request.

Perfomance: Low

Jobs must always copy any data needed from $VSC_DATA_VO to the scratch before the run and save any results from scratch into $VSC_DATA_VO after the run.

Reliability: Very High

Data is stored in a redundant file system with data replication off-site.

Back-ups: short and mid-term

Regular data snapshots allow to recover data at multiple points in time. Daily snapshots are kept for the past 7 days and weekly snapshots for the past 3 months.

VO Scratch

Location

$VSC_SCRATCH_VO, $VSC_SCRATCH_VO_USER

Purpose

Storage of temporary or transient data that can be shared with co-workers.

Availability: High

Accessible from login nodes and compute nodes in the local cluster. Not accessible from other VSC clusters.

Capacity: High and expandable

225 GB (soft limit), 250 GB (hard limit, 7 days grace time). Can be expanded upon request.

Perfomance: High

Preferred location for all data files read or written during the execution of a job. Suitable for all workload types.

Reliability: Medium

Data is stored in a redundant file system, but without off-site replication.

Back-ups: short-term

Regular data snapshots allow to recover data lost recently. Daily snapshots are kept for the past 7 days.

  • $VSC_SCRATCH_VO and $VSC_DATA_VO: top directory shared by all members of the VO

  • $VSC_SCRATCH_VO_USER and $VSC_DATA_VO_USER: each member of the VO has its own personal folder inside the VO that can only be accessed by its owner. Both folders can be used as alternatives to their personal $VSC_SCRATCH and $VSC_DATA.

Joining an existing VO#

Warning

Keep in mind that users can only be a member of a single VO in the same VSC institution. Thus, if you are member of a VO and join another VO, you will lose access to the data in the first VO.

Members of the research team can make a request to join the VO of their research group:

  1. Get the ID of the VO of your research group you belong to. VO’s ID in VUB are formed by the letters bvo followed by 5 digits.

  2. Fill in the section Join VO of your VSCaccNew/Join VO

    • Select the corresponding VO ID from the drop-down box below Group

    • Fill out the Message box with a message identify yourself for the moderator of the VO

    • Upon submission, the moderator of the VO (somebody from the research group) will receive and review your request

Creating a new VO#

Warning

VO requests from (PhD) students or postdocs will be rejected. Only group leaders (ZAP members) are allowed to create a VO.

Group leaders (ZAP members) can make a motivated request to the HPC team to create a new VO for their research group:

  1. Make sure you have an active VSC account

    If you don’t yet have a VSC account, follow the instructions in VSCdocGetting Access to request it. The process is fairly easy if you don’t need access to the HPC cluster; in that case it’s not required to generate an SSH key pair.

  2. Go to the section Request new VO in your VSCaccNew/Join VO

    • Fill out the form below ‘Why do you want to request a VO’

    • Fill out both the internal and public VO names. These cannot contain spaces, and should be 8-10 characters long. For example, genome25 is a valid VO name.

    • Fill out the rest of the form and press submit. This will send a message to the HPC administrators, who will then review your request

  3. If the request is approved, you will now be a member and moderator of your newly created VO

Requesting more storage space#

VO moderators can request additional quota for the VO and its members:

  1. Go to the section Request additional quota in your VSCaccEdit VO

    • Fill out the amount of additional storage you want for the data storage of your VO (named VSC_DATA in this section) and/or the scratch storage of your VO (named VSC_SCRATCH_RHEA in this section)

    • Add a comment explaining why you need additional storage space and submit the form

  2. Your request will be reviewed by the HPC administrators

Setting per-member VO quota#

VO moderators can tweak the share of the VO quota that each member can maximally use. By default, this is set to 50% for each user, but a moderator can change this: it is possible to give a particular user more than half of the VO quota (for example 80%), or significantly less (for example 10%).

  1. Go to the section Manage per-member quota share in your VSCaccEdit VO

    • Fill out the share (%) of the available space you want each user to be able to use and press confirm

  2. The per-member VO quota will be updated in 30 minutes maximum

Note

The total percentage per-member can be above 100%. The share for any user indicates what he/she can maximally use, but the actual limit will then depend on the usage of the other members. The total storage space of the VO will always be respected.

On this page