Submit

Usage

hs submit [-h] [ARGS... | -f FILE]

[-q [-H ADDR] [-p NUM] [-k KEY] | --initdb] [-b NUM] [-w SEC] [-c NUM] [-m MEM] [-W SEC] [-g NUM] [--template CMD] [-t TAG...] [--no-tls | [--tls-ca PATH] [--tls-cert PATH] [--tls-key PATH]]

Description

Submit one or more tasks.

Submit one task as positional arguments. If a single positional argument refers to a non-executable file path, tasks will be read from the file, one per line (use -f/--task-file to be explicit).

Tasks are accumulated and published in bundles to the database by default. With -q/--queue, tasks are submitted directly to a live queue, bypassing the database. The -b/--bundlesize and -w/--bundlewait options control the size of these bundles and how long to wait before flushing tasks regardless of how many have accumulated.

Pre-format tasks at submit time with template expansion using --template. Any tags specified with -t/--tag are applied to all tasks submitted.

Use the special comment syntax, # HYPERSHELL: ..., to include resource limits or tags inline. If the comment is alone on the line it will be applied to all tasks that follow.

Arguments

ARGS…

Command-line task arguments (for single task submission).

Options

-f, --task-file FILE

Input file containing one task per line.

-q, --queue

Submit directly to live queue instead of database.

When enabled, tasks are sent directly to a running server’s queue for immediate scheduling. This bypasses the database entirely, which is useful for transient workflows or when a database is not configured.

Requires -H, -p, and -k options to specify the server connection.

-H, --host ADDR

Hostname or IP address for server (default: localhost).

Only used with --queue mode to specify the server to connect to.

-p, --port NUM

Port number for server (default: 50001).

Only used with --queue mode to specify the server port.

-k, --auth KEY

Cryptographic authentication key for server.

Only used with --queue mode. The key must match the server’s authentication key.

--template CMD

Command-line template pattern (default: “{}”).

This is expanded at submit-time before sending to the database. With the default “{}” the input command-line will be run verbatim. Specifying a template pattern allows for simple input arguments (e.g., file paths) to be transformed into some common form; such as --template './some_command.py {} >outputs/{/-}.out'.

See section on templates.

-b, --bundlesize NUM

Size of task bundle (default: 1).

The default value allows for greater concurrency and responsiveness on small scales. Using larger bundles is a good idea for large distributed workflows; specifically, it is best to coordinate bundle size with the number of executors in use by each client.

See also --bundlewait.

-w, --bundlewait SEC

Seconds to wait before flushing tasks (default: 5).

If this period of time expires since the previous bundle was pushed to the database, The current bundle will be pushed regardless of how many tasks have been accumulated.

See also --bundlesize.

-c, --cores NUM

Number of CPU cores required per task (default: none).

Sets the default core requirement for all submitted tasks. Individual tasks can override this with inline comments (e.g., #HYPERSHELL: cores:8).

-m, --memory MEM

Amount of memory required per task (default: none).

Sets the default memory requirement for all submitted tasks. Specify memory size with units (e.g., ‘4GB’, ‘512MB’). Individual tasks can override this with inline comments (e.g., #HYPERSHELL: memory:8GB).

-W, --timeout SEC

Task-level walltime limit in seconds (default: none).

Sets the default timeout for all submitted tasks. Individual tasks can override this with inline comments (e.g., #HYPERSHELL: timeout:3600).

-g, --group NUM

Task group for dependency management (default: 0).

Assigns submitted tasks to a specific group. Tasks are executed in ascending group order, with all tasks in a group completing before the next group begins. This enables simple workflow management without requiring explicit DAG-based definitions.

All tasks default to group 0 for backwards compatibility. Users can organize tasks into multiple groups (e.g., 0, 1, 2, …) where lower-numbered groups execute first.

--initdb

Auto-initialize database.

If a database is configured for use with the workflow (e.g., PostgreSQL), auto-initialize tables if they don’t already exist. This is a short-hand for pre-creating tables with the hs initdb command. This happens by default with SQLite databases.

See hs initdb command.

-t, --tag TAG…

Assign one or more tags.

Tags allow for user-defined tracking of information related to individual tasks or large groups of tasks. They are defined with both a key and value (e.g., --tag file:a). The default value for tags is blank. When searching with tags, not specifying a value will return any task with that key defined regardless of value (including blank).

--no-tls

Disable TLS on the queue interface (not recommended).

Transport-layer security is enabled by default. When disabled, task bundles and results travel the queue connection unencrypted; only do this on a trusted, isolated network or when transport security is already provided by an external tunnel.

See the security section.

--tls-cert PATH

Path to the TLS certificate file (default: <auto>).

With the default <auto> a self-signed certificate is generated once on first server start and stored under the site lib directory. Provide a PATH to use your own certificate instead.

--tls-key PATH

Path to the TLS private key file (default: <auto>).

Paired with --tls-cert. With the default <auto> the auto-generated key is used; a user-provided key file should be readable only by its owner.

--tls-ca PATH

Path to the TLS CA certificate file used to verify the peer (default: <auto>).

With the default <auto> the server certificate is trusted directly, which suits a single host or shared filesystem. Provide a PATH to a certificate authority to verify peers across multiple hosts.

See the security section for peer verification modes.