How can I keep Moab from multiplying my requested disk space by the number of procs I request?


If you are trying to request nodes that have a certain amount of disk space available you can do that with the file=X flag during job submission. You can run into the problem though were that request is multiplied by the number of procs requested.

For example, the following submission could end up requesting 600GB of disk space, rather than 50.
qsub -l nodes=1:ppn=12,file=50GB

There are a couple ways you can have Moab honor the amount requested in the job submission.

One way is to change the submit policy:
RMCFG[base] SUBMITPOLICY=NODECENTRIC

The default submit policy is PROCCENTRIC, which means it takes all the requested procs and divides them up into individual tasks so they can be distributed among nodes. This is useful if it's hard to find one node with all the procs available. This means that the requested disk space is treated as part of the task and gets multiplied by the number of requested procs.

With a policy of NODECENTRIC Moab puts all the requested procs in one task so they will run on the same node. This also means that the requested disk space is counted once and gives you what was asked for.

If you can't switch to using a NODECENTRIC policy you can also set a parameter in your config that will tell Moab that you only want the amount of disk space requested, not a multiple of it. You can do this with the following parameter:
FILEREQUESTISJOBCENTRIC TRUE

When you set this Moab will divide the requested disk space by the number of tasks on the job so you only get the requested amount.

Last update:
2015-07-27 18:36
Author:
Ben Roberts
Revision:
1.0
Average rating: 5 (1 Vote)

You can comment this FAQ

Chuck Norris has counted to infinity. Twice.

Records in this category

Tags