Problem: How do Moab and SLURM job ID mechanism work?
Solution:
Syncing job IDs
With moab.cfg
==================
SCHEDCFG[Moab] MAXJOBID=499999 MINJOBID=1000
RMCFG[slurm] SYNCJOBID=TRUE EPORT=10777
RMCFG[internal] JOBIDFORMAT=integer
slurm.conf
==================
FirstJobId=500000
MaxJobId=1000000
==================
In the above example, we set the SLURM job ID and Moab job ID based off Moab. The only exception is if you directly submit to SLURM via sbatch or salloc.
Note: Without "SYNCJOBID=TRUE", we use the SLURM job ID. With "SYNCJOBID=TRUE" Moab will use its own job IDs for batch jobs, and SLURM's job IDs for interactive jobs.
Without SYNCJOBID=TRUE, there is a problem. If your site does interactive jobs, then it's possible to have an interactive job get the next SLURM job ID at the same time Moab assigns a job ID, causing a job ID collision. It's recommended to split Moab and SLURM job IDs, and use Moab's IDs for msub and SLURM IDs for sallaoc.
Affected Versions: ALL
Tags: jobid, slurm, syncjobid