Skip to main content

Job Efficiency

suff​

The suff command is a utility adapted from seff, a script designed to obtain detailed information about the efficiency of jobs submitted to Slurm workload manager. It is particularly useful for analyzing resource usage and optimizing the scheduling of future jobs.

Usage​

suff [OPTIONS] <JobID>

Output Information​

The suff command provides several details about the specified job, including:

  • Job ID : The unique identifier for the job.
  • Array Job ID : If applicable, the array job ID.
  • Cluster : The name of the cluster where the job was run.
  • User/Group : The user and group associated with the job.
  • State : The current state of the job (e.g., PENDING, RUNNING, COMPLETED).
  • Cores : The number of cores allocated for the job.
  • Nodes : The number of nodes allocated for the job.
  • Cores per node : The number of cores per node.
  • CPU Utilized : The total CPU time used by the job.
  • CPU Efficiency : The efficiency of CPU usage, calculated as the percentage of requested core-walltime that was actually used.
  • Job Wall-clock time : The total wall-clock time the job ran.
  • Memory Utilized : The amount of memory used by the job.
  • Memory Efficiency : The efficiency of memory usage, calculated as the percentage of requested memory that was actually used.
  • Walltime Limit Utilized : The actual time (wallclock time) the job took to complete.
  • Walltime Limit Efficiency : The efficiency of walltime limit usage, calculated as the percentage of requested time limit that was actually used.
  • Walltime Limit Diagnostic : A short diagnostic about your walltime limit.

Example​

[user@mc-login1 ~]$ suff 1234

******************************************************
Job ID: 1234
Cluster: mcmesu
User/Group: *-*-*-*-/users
******************************************************
Nodes: 1
Cores per node: 32
******************************************************
CPU Utilized: 20-15:57:29
CPU Efficiency: 99.50% of 20-18:27:44 core-walltime
Memory Utilized: 4.45 GB (estimated maximum)
Memory Efficiency: 1.78% of 250.00 GB (7.81 GB/core)
******************************************************
State: COMPLETED (exit code 0)
Walltime Limit Utilized: 15:34:37
Walltime Limit Efficiency: 74.18% of 21:00:00
Walltime Limit Diagnostic: CORRECT (you may lower it a little)
******************************************************

Notes​

Efficiency Statistics: Efficiency statistics may be misleading for running jobs, as they are subject to change.
Pending Jobs: Efficiency information is not available for jobs in the PENDING state.
For more detailed information, consult the official Slurm documentation or the man pages associated with seff and other Slurm commands.

sacct​

sacct is another command to examine your pending, running, and finished Slurm jobs in much more detail.

Usage​

sacct [OPTIONS]

Options​

-j, --jobs=<job[.step]> : Displays information about the specified job or list of jobs.
-h, --help : Displays a general help message.
-S, --starttime=<start_time> : Selects jobs in any state (or specified states) after the specified time.
-E, --endtime=<end_time> : Selects jobs in any state (or specified states) before the specified time.
-l, --long : Displays all information available about the specified job or list of jobs.
-s, --state=<state_list> : Selects jobs based on their state during the time period given. See the Job State Code section below for a list of state designators. Multiple state names may be specified using comma separators.

  • Most relevant job state codes :

    SHORT NAMENAMEDESCRIPTION
    CACANCELLEDJob was explicitly cancelled by the user or system administrator. The job may or may not have been initiated.
    CDCOMPLETEDJob has terminated all processes on all nodes with an exit code of zero.
    FFAILEDJob terminated with non-zero exit code or other failure condition.
    OOMOUT_OF_MEMORYJob experienced out of memory error.
    PDPENDINGJob is awaiting resource allocation.
    RRUNNINGJob currently has an allocation.
    TOTIMEOUTJob terminated upon reaching its time limit.

Many more options are listed on the manual, using the man sacct commmand.

Example​

For example, to see all your completed jobs since yesterday :

[user@mc-login1 ~]$ sacct -s CD -S now-1day -E now

JobID JobName Partition Account AllocCPUS State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
3694 yambo std user1 192 COMPLETED 0:0
3694.batch batch user1 32 COMPLETED 0:0
3694.extern extern user1 192 COMPLETED 0:0
3694.0 yambo user1 192 COMPLETED 0:0