Runs & Logs

A run is Caasi's unit of long-running work: a simulation, a training job, a ROS launch, an SSH batch job or a container. Runs are started detached — the CLI returns immediately and the process survives your terminal — and every run keeps its own logs, metadata and lifecycle state on disk.

The run model

Where runs live

All runs are stored under paths.runs (default ~/.caasi/runs), one directory per run:

~/.caasi/runs/
└── 20260905-142301-wave/          # run id: <timestamp>-<name-slug>
    ├── manifest.yaml              # metadata (written after the process spawns)
    ├── run.sh                     # bash wrapper that records the exit code
    ├── stdout.log                 # everything the process prints
    ├── stderr.log
    ├── exit_code                  # appears when the process finishes
    └── …                          # any artifacts the process itself writes here

The run id is YYYYmmdd-HHMMSS-<slug> — a local timestamp plus the run name, lowercased with non-alphanumeric runs replaced by -. Because ids are timestamp-prefixed, sorting them sorts by time.

How a run is started

The run directory and the run.sh wrapper are created. The wrapper executes the command and writes its exit code into exit_code when it finishes.
The process is spawned with start_new_session=True (its own process group) and stdin detached, stdout/stderr redirected into the log files. The CLI never waits for it.
manifest.yaml records everything: id, name, backend, kind, full command, working directory, creation time, PID, and the paused/stopped flags.

Manifest fields

Field	Content
`id` / `name`	Run id and human name.
`backend`	What executes it: `python`, `sim`, `lab`, `ros`, `ssh`, `container`.
`kind`	What it is: `run`, `experiment`, `train`, `benchmark`, `dataset`, `test`, `ros`, `nav`, `moveit`, `remote`, `container`.
`command`	Full argv list.
`cwd`	Absolute working directory.
`created`	ISO-8601 timestamp with timezone.
`pid`	PID of the wrapper process (its whole process group is controlled).
`paused` / `stopped`	Lifecycle flags set by pause/stop.
`extra`	Command-specific metadata (experiment path, image, remote, …). Visible only in the manifest file, not in JSON output.

Status: derived, not remembered

Status is computed on every query, in strict precedence:

Status	Meaning	Derived from
succeeded	Finished with exit code 0.	`exit_code` file = 0
failed	Finished with non-zero exit code.	`exit_code` file ≠ 0
running	Alive and executing.	No exit code + PID alive
paused	Frozen with `SIGSTOP`.	No exit code + PID alive + paused flag
stopped	Terminated by `caasi run stop`.	No exit code + PID dead + stopped flag
lost	Vanished without recording an exit code (SIGKILL, reboot).	No exit code + PID dead + no flags

Note

A stopped run may report failed instead of stopped: if the wrapper survived long enough to record the SIGTERM exit code (typically 143), the exit code wins. Both mean “terminated by you”.

Referring to a run: the QUERY argument

Every command that takes a run accepts a flexible QUERY:

latest, last or newest → the most recent run;
an exact run id → that run;
a unique id prefix or exact run name → that run (caasi logs 20260905-14, caasi logs wave).

Ambiguous prefixes and unknown queries fail with Error: No run matching '<query>'. (exit 1).

caasi run

caasi run is the one command group for long-running work. run start launches a config on a chosen backend and writes its provenance bundle; run list shows every run, and the lifecycle verbs — status, logs, stop, pause, resume, attach, restart, inspect, delete — act on a run named by the QUERY argument. Two root aliases cover the commands you reach for most: caasi start → run start and caasi logs → run logs.

shellcaasi run start experiments/wave.yaml --backend sim --name wave
Run 20260905-142301-wave started in the background.
caasi run status latest
wave (20260905-142301-wave)
  Status     running
  Backend    sim
caasi run logs latest -f
# follow until the run ends — Ctrl+C stops following, the run keeps going
caasi run stop latest
Run 20260905-142301-wave stopped.
caasi run attach latest
# replay both streams, then follow — Ctrl+C detaches, the run keeps going
caasi run restart latest
Run 20260905-142301-wave restarted as 20260905-160412-wave.

The sim control verbs moved under run

In 0.3.0 caasi sim status, sim stop, sim pause, sim resume and sim logs are gone. Drive simulation runs through the run group instead — caasi run status --backend sim, caasi run stop --backend sim, caasi run logs <query>. The verbs run list, status, stop, pause and resume all accept --backend sim|lab|python to filter or target runs by backend.

caasi run start

caasi run start CONFIG [--backend sim|lab|python] [--name NAME] [--dry-run] [--json]

The generic launcher: resolves the CONFIG file (an experiment YAML, for example), picks the backend, spawns the process detached and writes the run's provenance bundle. The root alias caasi start CONFIG is identical.

Option	Type	Default	Description
`--backend`	str	—	Launcher to use: `sim`, `lab` or `python`.
`--name`	str	—	Human name for the run (feeds the run-id slug).
`--dry-run`	flag	off	Resolve and print the command without spawning a process.
`--json`	flag	off	The started run record (see run record).

shellcaasi run start experiments/wave.yaml --backend sim --name wave
Run 20260905-142301-wave started in the background.
caasi start experiments/wave.yaml --dry-run
backend  sim
command  /opt/isaac-sim-6.0/python.sh /home/you/demo/scripts/wave.py --steps 10000
cwd      /home/you/demo/experiments
# --dry-run resolves and prints the command; nothing is spawned

Every run start writes a provenance bundle into the run directory so the run can later be accounted for and reproduced — seven files: manifest.yaml, environment.yaml, hardware.yaml, dependencies.yaml, configuration.yaml, git.json and processes.json. caasi run inspect latest --json | jq '.files' lists them; the provenance bundle explains what each records. Provenance degrades honestly: no git or no repo → git.json is {"available": false}, never a fabricated commit.

caasi run restart

caasi run restart QUERY [--backend sim|lab|python] [--json]

Restarts the matched run: if it is still active it is stopped first, then a fresh run is started from the same config — a new run id and provenance bundle, with the previous run left intact for comparison. Pass --backend to restart on a different launcher.

shellcaasi run restart latest
Run 20260905-142301-wave restarted as 20260905-160412-wave.
caasi run restart latest --backend lab --json | jq '.id'
"20260905-161230-wave"

caasi run list

caasi run list [--limit|-n N] [--json]

Option	Short	Type	Default	Description
`--limit`	`-n`	int	`20`	Number of runs to show (newest first).
`--json`		flag	off	Array of run records.

shellcaasi run list
ID                    Name          Backend  Status     Created                    PID
20260905-151207-ant   ant           lab      running    2026-09-05T15:12:07+08:00  51244
20260905-142301-wave  wave          sim      succeeded  2026-09-05T14:23:01+08:00  —
20260905-091544-nav2  nav2-bringup  ros      stopped    2026-09-05T09:15:44+08:00  —

Always exits 0; an empty history prints No runs recorded yet.

The run record (JSON shape)

All --json outputs of run commands share this shape:

{
  "id": "20260905-142301-wave",
  "name": "wave",
  "backend": "sim",
  "kind": "experiment",
  "command": ["/opt/isaac-sim-6.0/python.sh", "/home/you/demo/scripts/wave.py", "--steps", "10000"],
  "cwd": "/home/you/demo/experiments",
  "created": "2026-09-05T14:23:01+08:00",
  "pid": 48213,
  "paused": false,
  "stopped": false,
  "directory": "/home/you/.caasi/runs/20260905-142301-wave",
  "status": "succeeded"
}

caasi run status

caasi run status QUERY [--json]

shellcaasi run status latest
wave (20260905-142301-wave)
  Status     succeeded
  Backend    sim
  Kind       experiment
  Created    2026-09-05T14:23:01+08:00
  PID        48213
  Directory  /home/you/.caasi/runs/20260905-142301-wave
  Command    /opt/isaac-sim-6.0/python.sh /home/you/demo/scripts/wave.py --steps 10000

Exit 0 when found, 1 when not.

caasi run stop

caasi run stop QUERY

Terminates the whole process group: resumes it first if paused, sends SIGTERM, waits up to 5 seconds, escalates to SIGKILL, and marks the manifest stopped. Already-finished runs are a no-op success.

shellcaasi run stop ant
Run 20260905-151207-ant stopped.

Exit 1 if the run is unknown or the PID somehow survives SIGKILL.

caasi run pause / resume

caasi run pause  QUERY        # SIGSTOP to the process group — only when running
caasi run resume QUERY        # SIGCONT — only when paused

Pause freezes the process in place (it holds its GPU memory — check with caasi gpu memory); resume continues exactly where it stopped. Wrong-state calls fail with e.g. Error: Run <id> is not running (status: succeeded). (exit 1).

caasi run delete

caasi run delete QUERY [--force]

Option	Type	Default	Description
`--force`	flag	off	Stop the run first if it is still active.

Removes the whole run directory (logs, manifest, artifacts). Refuses while the run is running or paused unless --force. Terminal, stopped and lost runs are always deletable.

caasi run inspect

caasi run inspect QUERY [--json]

Lists every artifact file inside the run directory with human-readable sizes. JSON adds a files array to the run record:

shellcaasi run inspect latest --json | jq '.files'
[
  { "path": "manifest.yaml", "size": 412 },
  { "path": "run.sh", "size": 118 },
  { "path": "stdout.log", "size": 28413 },
  { "path": "stderr.log", "size": 0 },
  { "path": "metrics/result.json", "size": 1024 }
]

Tip — artifacts

Your scripts can write outputs into $CAASI_RUN_DIR (set for viewers) or simply print to stdout — everything lands in the run directory, and replay treats all files beyond the five bookkeeping files as recorded data.

caasi logs

caasi logs QUERY is a thin top-level alias of caasi run logs QUERY — one underlying operation, and the command you will use most. It reads a run's captured stdout/stderr and can aggregate and filter it by component, error level or time window.

caasi logs QUERY [--lines|-n N] [--follow|-f] [--stream|-s stdout|stderr] [--component NAME] [--errors] [--since 30s|10m|2h|1d] [--json]

Option	Short	Type	Default	Description
`--lines`	`-n`	int	`50`	Number of trailing lines to show.
`--follow`	`-f`	flag	off	Tail the log until the run ends; Ctrl+C stops following (the run keeps going).
`--stream`	`-s`	str	`stdout`	Which log file: `stdout` or `stderr`.
`--component`		str	all	Only lines tagged with this component (e.g. `nav2`).
`--errors`		flag	off	Only lines that look like errors.
`--since`		str	—	Time window: `30s`, `10m`, `2h`, `1d`. Untimestamped lines are excluded.
`--json`		flag	off	Structured log records instead of plain text.

shellcaasi logs latest -n 3
[14:41:02] step 9000/10000  fps 241.3
[14:41:07] step 10000/10000 fps 244.1
[14:41:07] done.
caasi logs latest -f -s stderr
…follows stderr live until the run ends…
caasi run logs latest --component nav2 --errors
[14:23:31] [nav2] [ERROR] planner: no valid path to goal
[14:23:44] [nav2] [ERROR] controller: aborted — recovery triggered
caasi run logs latest --since 10m -n 200
# only lines from the last 10 minutes; untimestamped lines are excluded

Follow mode stops automatically when the run reaches succeeded, failed or stopped. Errors (exit 1): unknown run, unknown --stream value, or the log file does not exist. --component, --errors and --since narrow the captured lines (--since drops untimestamped ones); --json emits structured log records instead of plain text.

caasi run attach

caasi run attach QUERY

Attaches to a run's live output: both logs are read from the beginning and then followed together, each line tagged with its stream. Ctrl+C detaches — the run keeps going, as always.

shellcaasi run attach ant
Attached to run 20260905-151207-ant — Ctrl+C detaches, the run keeps going.
[stdout] Loading extension: omni.kit.viewport.window
[stderr] [Warning] deprecated API used at frame 12
[stdout] learning iteration 12/1500  mean reward 87.2
# Ctrl+C
Detached from run 20260905-151207-ant; the run keeps going.
caasi run status ant --json | jq -r .status
running

	`caasi logs -f`	`caasi run attach`
Streams	one at a time — `--stream stdout\|stderr`	both, interleaved, each line prefixed `[stdout]` / `[stderr]`
History	the last `--lines N` (default 50), then follows from the end of the file	everything from the start of both logs, then follows
Options	`-n`, `-f`, `-s`, `--component`, `--errors`, `--since`, `--json`	none beyond `QUERY` — no `--json`
Ctrl+C	stops following, silently	detaches and says so

Like follow mode, attach returns on its own once the run reaches succeeded, failed or stopped — attaching to a finished run replays its logs and exits. Exit 1 for an unknown run or a run directory without log files (Error: No log files found for this run.). There is no top-level alias; caasi view attach is the unrelated viewer command.

Typical lifecycle

shellcaasi sim run experiments/wave.yaml
Run 20260905-142301-wave started in the background.
caasi run pause latest
Run 20260905-142301-wave paused.
caasi run resume latest
Run 20260905-142301-wave resumed.
caasi run stop latest && caasi run inspect latest && caasi run delete latest
Run 20260905-142301-wave stopped.
Run 20260905-142301-wave deleted.

Previous← Configuration NextObservability →

Caasi CLI 0.3.0 — these pages live in docs/ and need no build step. Source: github.com/7a6172/caasi.