Adaptive Computing Inc - TORQUE creates many files for array jobs with a large index.

Issue:

TORQUE creates many files for array jobs with a large index. This results in

1) Using the filesystem as a database is problematic at scale.

2) having large number of jobs vastly slows down TORQUE and Moab.

Solution:

Starting in 5.1.2,6.0.1 a new parameter "use_job_subdirs" can be used.

"use_jobs_subdirs"

http://docs.adaptivecomputing.com/torque/6-0-2/adminGuide/help.htm#topics/torque/13-appendices/serverParameters.htm#use_jobs_subdirs

use_jobs_subdirs
Format	<BOOLEAN>
Default	Not set (FALSE).
Description	Lets an administrator direct the way pbs_server will store its job-related files. When use_jobs_subdirs is unset (or set to FALSE), job and job array files will be stored directly under $PBS_HOME/server_priv/jobs and $PBS_HOME/server_priv/arrays. When use_job_subdirs is set to TRUE, job and job array files will be distributed over 10 subdirectories under their respective parent directories. This method helps to keep a smaller number of files in a given directory. This setting does not automatically move existing job and job array files into the respective subdirectories. If you choose to use this setting (TRUE), you must first set use_jobs_subdirs to TRUE, shutdown the Torque server daemon, in the contrib directory, run the "use_jobs_subdirs_setup" python script with -m option, start the Torque server daemon.

use_jobs_subdirs

Format

Default

Not set (FALSE).

Description

Lets an administrator direct the way pbs_server will store its job-related files.

When use_jobs_subdirs is unset (or set to FALSE), job and job array files will be stored directly under $PBS_HOME/server_priv/jobs and $PBS_HOME/server_priv/arrays.
When use_job_subdirs is set to TRUE, job and job array files will be distributed over 10 subdirectories under their respective parent directories. This method helps to keep a smaller number of files in a given directory.
This setting does not automatically move existing job and job array files into the respective subdirectories. If you choose to use this setting (TRUE), you must first
- set use_jobs_subdirs to TRUE,
- shutdown the Torque server daemon,
- in the contrib directory, run the "use_jobs_subdirs_setup" python script with -m option,
- start the Torque server daemon.

Jira:

TRQ-3725

Tags: lots of files, use_jobs_subdirs