These events files are found in the $MOABHOMEDIR/stats directory, and contain detailed logging of job-related events. The filename is of the format "events.<weekday>_<month>_<day>_year. For example, "events.Wed_Mar_22_2023".
The first five fields are fixed-format, and are as follows:
- <Recorded time in HH:MM:SS>
- <Event time (epoch date)>:<Event id>
- <Object type: 8 characters, left justified (%-8s)>
- <Object id: 12 characters, left justified (%-12s)>
- <Event type: 12 characters, left justified (%-12s)>
The remaining fields are space-delimited attribute/value pairs, and are defined as follows (I'll also attach a PDF file with the same information)
:
Description of the named fields in the event logs | |
Attribute | Description |
NAME | User specified name of job, job ID |
REQUESTEDNC | Number of nodes required by job, job's total "compute" node count |
REQUESTEDTC | Number of tasks required by job, job's total "compute" task count |
UNAME | UserID under which job will run, User credential name |
GNAME | GroupID under which job will run, Group credential name |
USER | Proxy user if a proxy submission, otherwise will match UNAME |
WCLIMIT | Walltime required by job, requested walltime limit |
STATE | State of job, RM job state |
RCLASS | List of <CLASSNAME>:<COUNT> pairs indicating type and number of class instances required per task. (i.e., [batch:1] or [batch:2][tape:1]), Class name |
SUBMITTIME | Time job was submitted to resource manager, time job was submitted to RM by the user |
DISPATCHTIME | time job was dispatched by RM |
STARTTIME | Time job was started by the resource manager, time job began most recent execution |
COMPLETETIME | Time job completed execution, time job execution completed (according to Moab's clock) |
RARCH | Architecture required by job, HW arch |
ROPSYS | Operating system required by job, required OS |
RMEMCMP | Real memory comparison (i.e., node must have >= 512MB RAM), One of '>=', '>', '==', '<', or '<=' |
RMEM | Real memory (RAM, in MB) required to be configured on nodes allocated to the job |
UMEM | job's total memory usage |
RDISKCMP | Local disk comparison (i.e., node must have > 2048 MB local disk) |
RDISK | Local disk space (in MB) required to be configured on nodes allocated to the job, required disk resources |
RFEATURES | List of features required on nodes |
SYSTEMQUEUETIME | time job was initially eligible to start |
TASKSPERNODE | Exact number of tasks required per node |
REQUESTEDQOS | Quality of service requested, quality of service requested |
QOS | Quality Of Service credential name |
FLAGS | Job flags |
ACCOUNT | Account credential name |
COMMAND | job executable command |
RMXSTRING | Resource Manager Extensions |
BYPASSCOUNT | number of times lower prio job was backfilled |
PSUTILIZED | procseconds utilized by job |
PARTITION | assigned partition |
DPROCS | Number of processors dedicated to the job, job's total processor count |
DMEM | Quantity of memory (in MB) that must be dedicated to each task of the job, job's total memory count (for first req) |
DDISK | Quantity of local disk space (in MB) that must be dedicated to each task of the job, total dedicated disk resources (for first req) |
DSWAP | Quantity of virtual memory (swap, in MB) that must be dedicated to each task of the job, total dedicated swap resources (for first req) |
STARTDATE | Earliest time job should be allowed to start, user specified earliest start time |
ENDDATE | Time by which job must complete, user specified latest completion date |
GRES | all GRes from all reqs |
TASKMAP | The allocation taskmap for the job, nodelist map with task counts |
SRM | rm to which job is submitted |
HOSTLIST | List of required hosts on which job must run, required hosts |
REQRSV | Name of reservation where job must run, required rsv name/group |
TEMPLATE | List of 'job match' structures |
MESSAGE | job messages |
EXITCODE | Job exit code, execution completion code |
SID | System ID (global job system owner), session id of primary task |
VARIABLE | list of (job) variables |
NODEALLOCATIONPOLICY | Node Access Policy (I know, it is named one thing and reports another) |
GMETRIC | utilized total generic metric values |
EFFECTIVEQUEUEDURATION | duration of time job was eligible to run |
RMSUBMITSTRING | raw command file |
DRMJID | destination resource manager job ID |