Scheduling policy

Krypton uses simple first-in-first-out scheduling (with backfill so that shorter jobs that fit can run as long as they do not delay the start of jobs with higher priority).

The maximum wall time for a job is 7 days. The default time limit (if you do use a "-t" flag) is 2 hours. Please use the "-t" flag to set a time limit that is appropriate for each job!

Limits between different user groups

The following groups have a fixed number of nodes available for normal jobs (limits up-to-date as of 2014-02-20):

Group (SLURM Account) Nodes
rossby 73
sm_fouo 73
fm 15

These nodes are always allocated to their corresponding group. At the moment, this is implemented using SLURM partitions, but that may change in the future.

The following groups all share a partition ("other") containing 67 nodes:

Group (SLURM Account) Max nodes per group
sm_foup 59
sm_foua 14
sm_foul 30
sm_ml 12
sm_mo 12

As you see, the sum of the limits is greater than 67. This means that your jobs may be blocked due to an insufficient amount of nodes in total, or because of your group hitting the group-based limit.

Specifying your SLURM Account

If you are a member of more than one group, you should always use an option like -A rossby, -A sm_fouo etc. to sbatch/interactive to tell SLURM what account to run under.

When you login, we set the SBATCH_ACCOUNT environment variable to your default account. If you are only part of one group, that should be enough and you do not need to use the -A option for normal job submission. You might have to use it under special circumstances, such as cron jobs.

Requesting nodes with more memory

The 12 nodes with more memory are kept in a separate partition. To use them, add _fat to your account name. For example, if you are part of the sm_foup group, use -A sm_foup_fat when you submit jobs.

The 10 fat nodes (128 GiB) will be used before the 2 huge nodes (256 GiB). If you need the huge nodes, use -C huge in addition to the -A option discussed above.

Running risk jobs

All groups on Krypton can submit risk jobs that are able to run on all available nodes (even fat and huge ones). The drawback is that risk jobs will be killed as soon as the nodes they run on are needed to be able to run a non-risk job.

To submit risk jobs, add _risk to your account name. For example, if you are part of the sm_fouo group, use -A sm_fouo_risk.

If the risk job is a batch job (not interactive), you will probably also like to add --requeue, so that the risk job is requeued automatically if it is preempted. Without this flag, the job will be canceled as soon as it is preempted.

If you want to control the set of nodes the risk jobs can be placed on (instead of all available), you can use the -C flag together with one of the names fm, rossby, fouo, other, fat, huge. You can also combine them using the | character (like -C 'rossby|fouo').

Running on Nehalem nodes

The 128 Nehalem nodes on Gimle have been moved to Krypton, where they form the "nehalem" partition. Rossby and FoUo users can run jobs on these nodes by using -A rossby_nehalem or -A sm_fouo_nehalem, respectively. There are no inter-group limits, and risk jobs are not available.