Skip to content

Allocations and monitoring

Resource allocations

Resource allocations (compute time, data storage) are given to projects based on the appropriate grant, agreement, service contract or EuroHPC call. Users are at least part of one project, and may be linked to several projects.

On MeluXina, projects receive a Project ID (e.g. p212345) which is used:

  • for the SLURM account that users are members of (access control, compute time allocations, etc.)
  • for the project data directory/directories (access control, data quotas, etc.)
  • in service requests users open on the ServiceDesk

For compute time allocations, the overall allocation is divided in monthly grants, which depend on the number of days in the month.

As a practical example:

  • a 1 year project with 1200 GPU node-hours allocation is granted (Jan 1st - Dec 31st 2022)
    • the GPU node-hours are converted to node-minutes: 1200 x 60 = 72000
    • the equivalent of the allocation is 72000 / 365 = 197.26 node-minutes per day
  • in January, the project will be able to access 197.26 x 31 = 6115 GPU node-minutes, or almost 102 GPU node-hours
  • in February, this will be 197.26 x 28 = 5523 node-minutes, or just over 92 GPU node-hours
    • the project's compute time utilization is set to zero on Feb 1st, and the new monthly allocation is applied
  • the same process is applied throughout the lifetime of the project

For data allocations, the overall allocation is granted throughout the lifetime of the project.

Compute time utilisation

Users shall ensure that they utilize their compute time allocation consistently and proportionally during the project lifetime. The division of compute time allocations for the duration of the projects and the roll-over process is for now tentative and subject to change.

Remember!

Users shall ensure that they submit jobs only to partitions to which they have been granted access to, jobs submitted to partitions for which no node-hours have been granted will fail to execute.

Myquota

To monitor the usage of your resource allocations, the myquota tool is available on the MeluXina login nodes. The tool will show you:

  • Compute time allocations, for each project (SLURM account) you have access to

    • in Node-Hours for every type of computing resource (CPU, GPU, FPGA and LargeMem nodes)
    • Used out of the total available (Max) in the current month
  • Data storage allocations, for each project (group account) you have access to

    • in GiB (Gibibytes) for data size
    • in Number-of-Files, for the metadata allocation
    • Used at the current moment, out of the maximum (Max) allocated for the project

myquota

By default, myquota will show both compute and data utilization and allocations.

(login/compute)$ myquota
Output
                                                COMPUTE ALLOCATIONS FOR CURRENT MONTH

 Project                             CPU node-hours             GPU node-hours           FPGA node-hours           LargeMem node-hours
                                 Used |   Max   | Use%      Used |   Max   | Use%      Used |   Max   | Use%      Used |   Max   | Use%
meluxinaproject                     5 |      10 |  50%         0 |     100 |   0%         1 |       4 |  25%         0 |       4 |   0%

                                                         DATA ALLOCATIONS

 Datapath                                      Data GiB                           No. Files
                                       Used |   Max   | Use%             Used |     Max    | Use%
/home/users/myuser                        1 |     100 |   1%            13216 |     100000 |  13%
/project/home/meluxinaproject           146 |    5000 |   2%           437033 |    5000000 |   8%
/project/scratch/meluxinaproject         29 |     100 |  29%            41160 |     100000 |  41%

Myquota can also be used to display just compute time usage and allocations:

(login/compute)$ myquota -t compute
Output
                                                COMPUTE ALLOCATIONS FOR CURRENT MONTH

 Project                             CPU node-hours             GPU node-hours           FPGA node-hours           LargeMem node-hours
                                 Used |   Max   | Use%      Used |   Max   | Use%      Used |   Max   | Use%      Used |   Max   | Use%
meluxinaproject                     5 |      10 |  50%         0 |     100 |   0%         1 |       4 |  25%         0 |       4 |   0%

Myquota can also be used to display just data storage usage and allocations:

(login/compute)$ myquota -t data 
Output
                                                                 DATA ALLOCATIONS

 Datapath                                      Data GiB                           No. Files
                                       Used |   Max   | Use%             Used |     Max    | Use%
/home/users/myuser                        1 |     100 |   1%            13216 |     100000 |  13%
/project/home/meluxinaproject           146 |    5000 |   2%           437033 |    5000000 |   8%
/project/scratch/meluxinaproject         29 |     100 |  29%            41160 |     100000 |  41%

Warning

Note that you can only run myquota once every 10 seconds. Do not run the tool repeatedly, e.g. using the watch command or similar.

Resource monitoring with SLURM & Lustre native commands

sreport

Compute time usage can be also checked with native Slurm commands:

(login/compute)$ sreport -t hours cluster AccountUtilizationByUser  Tree start=2021-06-01 end=2021-07-01
The command above will generate a report of compute resources that have been used from June 1st, 2021 (included) to July 1st, 2021 (excluded).

Output
--------------------------------------------------------------------------------
Cluster/Account/User Utilization 2021-06-01T00:00:00 - 2021-12-31T23:59:59 (18493200 secs)
Usage reported in CPU Hours
--------------------------------------------------------------------------------
  Cluster              Account     Login     Proper Name        Used   Energy 
--------- -------------------- --------- --------------- ----------- -------- 
 meluxina   p200000                                           688392        0 
 meluxina    p200000             u100000   MeluXina user         137        0 
 meluxina    p200000             u100000   MeluXina user       67121        0 
 meluxina    p200000             u100000   MeluXina user      621134        0 
 meluxina   p20000X                                           367605        0 
 meluxina    p20000X             u100000   MeluXina user      115266        0 
 meluxina    p20000X             u100000   MeluXina user      142792        0 
 meluxina    p20000X             u100000   MeluXina user       32063        0 
 meluxina    p20000X             u100000   MeluXina user       32153        0 
 meluxina    p20000X             u100000   MeluXina user       41810        0 

lfs

Data storage usage can be also checked with native Lustre commands which requires two steps. First - retrieve the Lustre FS project id for your (home) directory:

(login/compute)$ lfs project -d /home/users/$USER
Output
10011 P /home/users/myuser

The command above prints the project id of the directory specific to Lustre (not to be confused with your project number). In the second step - based on this ID, retrieve the quotas set for the directory:

(login/compute)$ lfs quota -h -p $PROJECTID /home/users/$USER
Output
Disk quotas for prj 10011 (pid 10011):
     Filesystem    used   quota   limit   grace   files   quota   limit   grace
/home/users/myuser
                   299M      0k    100G       -   15364       0  100000       -