Command line interface¶
This page provides an overview of the hanythingondemand command line interface.
Note
This only applies to hanythingondemand version 3 and newer; older versions provided a significantly different command line interface.
hod
command¶
The main hanythingondemand command is a Python script named hod
, and implements the top-level
command line interface that discriminates between the various subcommands.
Running hod
without arguments is equivalent to hod --help
, and results in basic usage information being printed:
$ hod
hanythingondemand version 3.2.0 - Run services within an HPC cluster
usage: hod <subcommand> [subcommand options]
Available subcommands (one of these must be specified!):
batch Submit a job to spawn a cluster on a PBS job controller, run a job script, and tear down the cluster when it's done
clean Remove stale cluster info.
clone Write hod configs to a directory for editing purposes.
connect Connect to a hod cluster.
create Submit a job to spawn a cluster on a PBS job controller
destroy Destroy an HOD cluster.
dists List the available distributions
genconfig Write hod configs to a directory for diagnostic purposes
help-template Print the values of the configuration templates based on the current machine.
list List submitted/running clusters
relabel Change the label of an existing job.
General hod
command line options¶
hod [subcommand] --help
¶
Print usage information and supported subcommands along with a short help message for each of them, or usage information and available options for the specified subcommand.
hod
subcommands¶
The hod
command provides a number of subcommands, which correspond to different actions.
An overview of the available subcommands is available via hod --help
(see hod command).
More details on a specific subcommand are available via hod <subcommand> --help
.
Available subcommands:
- hod batch –script=<script-name>
- hod clean
- hod clone <dist-name> <destination>
- hod connect <cluster-label>
- hod create
- hod destroy <cluster-label>
- hod dists
- hod genconfig
- hod help-template
- hod list
- hod relabel <old-label> <new-label>
hod batch --script=<script-name>
¶
Create a cluster and run the script. Upon completion of the script, the cluster will be stopped.
Next to --script
(which is mandatory with batch
), all configuration options supported for create
are
also supported for batch
, see Configuration options for hod create.
When used with batch
, these options can also be specified via $HOD_BATCH_*
.
Jobs that have completed will remain in the output of hod list
with a job id of <job-not-found>
until hod clean
is run (see hod clean), or until the cluster is destroyed using hod destroy
(see hod destroy <cluster-label>).
Note
--hod-module
, --workdir
, and either --hodconf
or --dist
must be specified.
hod clean
¶
Remove cluster info directory for clusters that are no longer available,
i.e. those marked with <job-not-found>
in the output of hod list
.
hod clone <dist-name> <destination>
¶
Clone a dist for use editing purposes. If there is a provided dist that is almost what is required for some work, users can clone it and edit the files.
hod connect <cluster-label>
¶
Connect to an existing hanythingondemand cluster, and set up the environment to use it.
This basically corresponds to logging in to the cluster head node using SSH and sourcing the cluster information script
that was created for this cluster ($HOME/.config/hod.d/<label>/env
).
hod create
¶
Create a hanythingondemand cluster, with the specified label (optional) and cluster configuration file (required).
The configuration file can be a filepath, or one of the included cluster configuration files (see hod dists).
Jobs that have completed will remain in the output of hod list
with a job id of <job-not-found>
until hod clean
is run (see hod clean), or until the cluster is destroyed using hod destroy
(see hod destroy <cluster-label>).
Note
--hod-module
, --workdir
, and either --hodconf
or --dist
must be specified.
Configuration options for hod create
¶
hod create --hod-module <module name>
¶
must be specified
Specify the hanythingondemand
module that must be loaded in the job that is submitted for the HOD cluster;
can also be specified via $HOD_CREATE_HOD_MODULE
.
hod create --workdir <path>
¶
must be specified
Specify the top-level working directory to use; can also be specified via $HOD_CREATE_WORKDIR
.
hod create --hodconf <path>
¶
either --dist
or this must be specified
Specify location of cluster configuration file; can also be specified via $HOD_CREATE_HODCONF
.
hod create --dist <dist>
¶
either --hodconf
or this must be specified
Specify one of the included cluster configuration file to be used (see also hod dists);
can also be specified via $HOD_CREATE_DIST
.
hod create --label <label>
¶
Specify label for this cluster. If not label is specified, the job ID will be used as a label;
can also be specified via $HOD_CREATE_LABEL
.
The label can be used to later connect to the cluster while it is running (see hod connect <cluster-label>).
hod create --modulepaths <paths>
¶
Add additional locations for modules that need to be loaded (see hod create –modules <module names>).
Can also be specified via $HOD_CREATE_MODULEPATHS
.
hod create --modules <module names>
¶
Add modules to the dist so each node has access to them. If code submitted to the cluster requires a particular module, it should be added with this option. For example, if an IPython notebook plans to use Python modules on the worker kernels (or through Spark) they will need to be added here.
Can also be specified via $HOD_CREATE_MODULES
.
hod create --job-*
¶
The resources being requested for the job that is submitted can be controlled via the available --job
options,
see –job options for hod create / hod batch.
hod destroy <cluster-label>
¶
Destroy the HOD cluster with the specified label.
This involves deleting the job, and removing the working directory and cluster info directory
($HOME/.config/hod.d/<label>
) corresponding to this cluster, if they are still in place.
In case the cluster is currently running, confirmation will be requested.
hod dists
¶
Print a list of available cluster configurations (‘distributions‘), along with the list of modules that correspond to each of them.
See for example Available distributions.
hod genconfig
¶
Generate hanythingondemand cluster configuration files to the working directory for diagnostic purposes.
The working directory can be specified using --workdir
or via $HOD_GENCONFIG_WORKDIR
.
hod help-template
¶
Print the values for the configuration templates based on the current machine.
hod list
¶
Print a list of existing clusters, and their state (‘queued
‘ or ‘running
‘).
Jobs that have completed running will remain in the list with <job-not-found>
until
hod clean
is run (see hod clean), or until the HOD cluster is destroyed using hod destroy
(see hod destroy <cluster-label>).
hod relabel <old-label> <new-label>
¶
Change the label for a hod cluster that is queued or running.
--job
options for hod create
/ hod batch
¶
The create
and batch
subcommands accept the following options to specify requested job resources.
These can also be specified via $HOD_BATCH_JOB_*
(for hod batch
) or $HOD_CREATE_JOB_*
(for hod create
).
--job-mail
/-m <string>
¶
Send a mail when the cluster has started (b
for ‘begin‘), stopped (e
for ‘ended‘) or got aborted (a
).
For example, using -m a
will result in receiving a mail whn the cluster has started running.
--job-mailothers
/-M <main addresses>
¶
List of other mail adresses to send mails to.
--job-name
/-N <name>
¶
Specify the name for the job that will be submitted.
--job-nodes
/-n <int>
¶
The number of (full) workernodes to request for the job being submitted (default: 1).
--job-ppn <int>
¶
The number of cores per workernode to request; by default: -1
, i.e. full workernodes (request all available cores).
--job-queue
/-q <int>
¶
Name of job queue to submit to (default: none specified).
--job-walltime
/-l <int>
¶
Number of hours of walltime to request (default: 48).