Limits
Sizes
| Item | Limit |
|---|---|
| Compute nodes per cluster | 1 to 128 (up to 1,024 GPUs) |
| GPU type | H200, with 8 GPUs and 8 InfiniBand cards per compute node |
| CPUs per compute node | 224 |
| Default memory per CPU | 9,874 MB (DefMemPerCPU) |
| Disk per compute node | 500 GB to 4000 GB. You cannot change it later. |
| Login VM sizes | 2, 4 (default), 8, 16 or 32 vCPU, with 4 GB of RAM per vCPU, if available in the region |
| Login VM disk | 100 GB to 2000 GB. You cannot change it later. |
| Scheduler VM sizes | 4 (default), 8 or 16 vCPU, with 4 GB of RAM per vCPU |
| Scheduler VM disk | 100 GB, fixed |
| Clusters per filesystem | 1 at a time |
| Regions | India-02 |
Names
| Item | Rule |
|---|---|
| Cluster name | 3 to 31 lowercase letters, digits or -. The first character is a letter. |
| Login name | Up to 32 lowercase letters, digits, - or _. The first character is a letter or _. You cannot change it later. |
| Compute node name | node-1, node-2 and so on. Jarvislabs never uses a name twice in a cluster. |
People and keys
| Item | Limit |
|---|---|
| People per cluster | 50 |
| SSH keys per person | 50 |
| Size of one key | 16 KB |
| All keys of one person | About 60 KB |
| All people, keys and Slurm settings together | 63 KB. Long keys, such as RSA 4096, reach it sooner. |
Slurm settings
| Item | Limit |
|---|---|
| Extra slurm.conf | 8 KB |
| Each script box | 16 KB |
| One setting's value | 512 characters |
Jobs
| Item | Limit |
|---|---|
| Array task numbers | Up to 1000 by default (MaxArraySize) |
| Jobs in Slurm at one time | 10,000 by default (MaxJobCount) |
| Soft limit of open files in a job | 1024. You can raise it with ulimit -n. |
| Jobs shown on the Jobs tab | The newest 1,000 |
| Time before a requeued job starts again | About 3 minutes |
| Time limit of a prolog or epilog | 300 seconds by default |
What is not available
| Item | Use this |
|---|---|
| A cancel button in the console | scancel |
| A drain or reboot button in the console | scontrol, and Restart on the Compute nodes tab |
| A job history page | sacct |
| Job email | Slurm accepts --mail-type and sends nothing. |
| Group folders | One shared folder. Refer to Storage. |
AllowGroups with a team's group | AllowAccounts |
| Quotas and teams in the console | sacctmgr on the login VM |
| A limit on disk use per person | Check with df -h ~. |
seff | sacct |
scrontab | It shows scrontab is disabled on this cluster. |
| Part of a GPU | --gres=shard:1 shows Invalid generic resource (gres) specification. |
A private /tmp for each job | Delete your files in your job script. |
module | uv or conda in your home folder |
sbcast of large files | Large files can fail. Refer to Troubleshooting. |
| Pause and resume of a cluster | Delete compute nodes to stop their charges. |
| Login to the scheduler VM | Jarvislabs manages it. |
| A notice when a compute node drains | Check sinfo -R and the Compute nodes tab. |