# Inspect Steward

> An agent that supervises evaluations, monitoring progress, diagnosing problems, and adjusting runtime state on your behalf.



# Inspect Steward

## Overview

Welcome to Steward, a system for automated execution and supervision of long-running evaluations. Steward enables you to launch a large run and walk away while a coding agent monitors it. Here’s how it works:

1.  Define your eval set and policies that will govern the run (Steward works with [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) as well as with execution frameworks like [Inspect Flow](https://meridianlabs-ai.github.io/inspect_flow/) and [Inspect Hawk](https://hawk.metr.org))

2.  Launch the eval set for unattended execution. Let standing policies and agents handle tuning and errors whenever possible. Receive notifications whenever something needs your attention and make whatever decisions are required.

3.  At the end of the run, make final judgements about how to score anomolies and then signoff the completed run.

[![](diagrams/workflow.excalidraw.svg)](diagrams/workflow.excalidraw.svg)

## Quick Tour

### Setup

First, install Steward from PyPI with:

    Terminal

``` bash
pip install inspect_steward
```

To use Steward, create a directory, switch to it, then call `steward init`:

    Terminal

``` bash
mkdir -p ~/runs/swe-evals
cd ~/runs/swe-evals
steward init
```

This will scaffold up a Steward workspace with a placeholder `evalset.py`, a `_steward.yaml` for your standing rules, and an `AGENTS.md` file that teaches the agent how to use Steward to run an evaluation. First, define your eval set:

    evalset.py

``` python
from inspect_ai import eval_set
from inspect_harbor import cais_swebenchpro, datacurve_deep_swe

eval_set(
    tasks=[cais_swebenchpro(),  datacurve_deep_swe()],
    model=["openai/gpt-5.6-sol", "anthropic/claude-opus-5"],
)
```

Steward can manage any evalution defined with a script that ends in a call to [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) or alternatively a `config.py` from [Flow](https://meridianlabs-ai.github.io/inspect_flow/) or a `hawk.yaml` from [Hawk](https://hawk.metr.org).

> **NOTE: NoteLog Directory**
>
> One other thing you should typically define up front is a root directory for eval logs. If you define the `STEWARD_LOG_ROOT` environment variable then a `log_dir` matching the workspace directory name will be automatically created and used for your eval set. For example:
>
>     .env
>
> ``` ini
> STEWARD_LOG_ROOT=s3://inspect-logs
> ```

### Run the Eval

Next, launch your coding agent from the workspace and tell it to run the evaluation (you should typically run from a [tmux](https://github.com/tmux/tmux/wiki) detached terminal so the agent is persistent):

    Coding Agent

``` bash
───────────────────────────────────────────── swe-evals ─
❯ please run this evaluation.
─────────────────────────────────────────────────────────
```

The agent will launch all of the tasks, setup a background monitoring process, and respond to alerts that require its intervention. From here you can detach and walk away.

> **NOTE: NoteNotifications**
>
> To be notified when things require your attention, we strongly recommend you also configure a notification channel. Put it in `_steward.yaml`, or — to keep the token out of your repository — in your `.env` file:
>
>     .env
>
> ``` ini
> STEWARD_NOTIFICATION=slack://{OAuthToken}/{ChannelID}
> ```
>
> See [Notifications](./workflow.html.md#notifications) for more details on configuring notifications for various targets including Slack, Email, and others.

The run depends on neither you nor the agent staying connected. When you come back, ask how it’s going:

    Coding Agent

``` bash
───────────────────────────────────────────── swe-evals ─
❯ how is the eval going?
─────────────────────────────────────────────────────────
```

The agent reports where the run stands: per-task progress, what it resolved on its own while you were gone, and anything that is waiting on a decision from you.

### Make Decisions

Runs rarely go perfectly, and the questions that survive the agent’s own judgement come back to you. For example, the agent might prompt. you with this when you check in:

    Coding Agent

``` bash
89 samples in cais_swebenchpro failed with the same
sandbox timeout, all inside a 30-minute window last 
night. Automatic retry has already spent its attempts.

re-run them, score them as failed, or exclude them 
from scoring?

───────────────────────────────────────────── swe-evals ─
❯ re-run them, the disk issue is fixed
─────────────────────────────────────────────────────────
```

## Learning More

- [Workflow](./workflow.html.md) is the comprehensive guide to a Steward run: the workspace directory, launching and tending runs, agent and human decisions, and how a run ends.

- [Concurrency](./concurrency.html.md) describes the various options for tuning concurrency, including automatic ramping behavior and caps on resource utilization.

- [Error Handling](./errors.html.md) covers standing policies for error retries and resolution, including dealing with errors that are never resolved.

- [Scanners](./scanners.html.md) explains how transcripts are scanned for scoring integrity problems as the run proceeds, and what a finding asks of you.

- [Agent Runbook](./runbook.html.md) provides the full operating instructions that coding agents use to supervise and make decisions during runs.

Integrations with execution environments are covered in [Inspect Flow](./flow.html.md) and [Inspect Hawk](./hawk.html.md).


# Workflow – Inspect Steward

## Overview

Steward runs evaluations from a workspace directory that includes the eval-set definition, steward-specific options, a journal of the entire run, and human readable documents on run status. Evaluations are launched as background processes and monitored using a scheduled job, with coding agents and operators proactively notified when they are needed for a decision.

We’ll walk through the various phases of the workflow below, then cover [options](#steward-options) you can use to customize Steward’s behavior and other [advanced](#advanced) topics like pausing runs and modifying runs in flight.

## Workspace

To create a workspace for an evaluation run, use the `steward init` command:

    Terminal

``` bash
mkdir -p ~/runs/swe-evals
cd ~/runs/swe-evals
steward init
```

The workspace is initialized with the following files:

    Workspace

``` text
swe-evals/
  _steward.yaml       # steward specific options 
  AGENTS.md           # agent instructions for managing a run
  CLAUDE.md           # agent instructions for managing a run
  evalset.py          # eval-set definition (or config.py / hawk.yaml)
  journal.jsonl       # append-only journal of workspace actions
  status.md           # the operator's page: where the run stands, what waits on them
```

Your eval set definition can be any script which ends with a call to [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) or a [Flow](./flow.html.md) or [Hawk](./hawk.html.md) configuration. For example:

    evalset.py

``` python
from inspect_ai import eval_set
from inspect_harbor import cais_swebenchpro, datacurve_deep_swe

eval_set(
    tasks=[cais_swebenchpro(),  datacurve_deep_swe()],
    model=["openai/gpt-5.6-sol", "anthropic/claude-opus-5"],
)
```

Your task before launching is to provide the eval set definition and adjust any settings you need in `_steward.yaml`. The defaults are generally suitable, but be sure to review [Concurrency](./concurrency.html.md) against the specifics of your evals and infrastructure.

## Execution

Once you have defined your eval set, use a coding agent to launch the evaluation (you should typically run from a [tmux](https://github.com/tmux/tmux/wiki) detached terminal so the agent is persistent):

    Coding Agent

``` bash
───────────────────────────────────────────── swe-evals ─
❯ please run this evaluation.
─────────────────────────────────────────────────────────
```

The agent reads the Steward [runbook](./runbook.html.md) and follows it. It starts with a short smoke run, skipping any task a configured log store already satisfies, to confirm that configuration, infrastructure and model access all work correctly. A failed smoke stops the launch until it is fixed or waived by the user.

After the smoke the agent launches the tasks and works the decisions the run raises as the [tend loop](#tend-loop) runs. Keeping the agent attached is recommended but not required. If you detach it, configure [notifications](#notifications) so you know when to reconnect to it.

### Tend Loop

As the run proceeds, the `steward tend` command runs on a timer set up by `steward launch`. By default this timer runs every 10 minutes. The tend command performs the following actions automatically:

- Spawns or restarts tasks that should be running and are not.
- Automatically tunes concurrency based on available sandbox capacity and observed model throughput.
- Notes anomalies and other conditions (model or infra errors, stuck samples, scoring integrity issues, etc.) and either handles them automatically or escalates them for agent or human consideration.
- Rewrites `status.md` and appends to `journal.jsonl`.

Note that you or an agent can call `steward tend` at any time to get the current status of the run.

### Agent Decisions

After tend runs, agents call `steward collect`, which reports the open decisions along with everything that has happened since it last looked. The agent handles the following on its own:

- Diagnoses errors and groups them into classes, and proposes what should be done about them (re-run, exclude, zero, score, accept, or dismiss) for you to rule on.
- Oversees the concurrency ramp, holding the climb when something looks wrong and tuning a worker down while containing an incident.
- Raises anything that needs you, and notifies you through the configured channel when it cannot proceed without an answer.
- Writes up the overall run investigation in `analysis.md` once the run is complete.

### Human Decisions

There are some things within a run that require your judgement or the establishment of a policy that can be applied to all similar judgements. Both `steward tend` and the agent can raise things for human input, asking you to:

- Rule on the error classes the agent proposes, choosing to re-run, exclude, zero, or re-score the samples, or to accept or dismiss the class outright.
- Answer tool call approvals that have escalated to human review.
- Decide what to do with a task that has stalled, or a sample stuck past the point the agent may act on it.
- Sign off on the final results after reviewing anomalies and transcript analysis.

### Notifications

Steward uses [Apprise](https://github.com/caronc/apprise) for sending notifications to human operators. You can configure Apprise to send Slack messages, emails, AWS SES or SNS messages, or many other types of message. Configure notifications by setting the `STEWARD_NOTIFICATION` environment variable or setting the `notification` option in `_steward.yaml`. For example:

    .env

``` ini
STEWARD_NOTIFICATION=slack://{OAuthToken}/{ChannelID}
```

It is strongly recommended that you configure notifications. If you don’t, you should just reconnect to your agent periodically and ask how things are going:

    Coding Agent

``` bash
───────────────────────────────────────────── swe-evals ─
❯ how is the eval going?
─────────────────────────────────────────────────────────
```

Steward notifies you when something needs a decision, when a run finishes, and when it cannot tend at all. In between, it also posts an hourly heartbeat so you know the latest status at a glance.

## Signoff

Once all tasks are completed (with errors/anomalies decided on) and their transcripts scanned for scoring integrity issues, the run is eligible for signoff. If you have a coding agent managing the run then it will prompt you for signoff at the appropriate time. If not, you can manually run signoff with:

    Terminal

``` bash
steward signoff --by "norah"
```

Signoff verifies that the run is eligible for signoff, ensures a clean final log directory, finalizes the journal and status, and disarms the tend loop timer. If you have a [Log Store](#log-store) configured, the agent will also ask whether to publish your logs to it — add `--publish` if you want that.

## Eval Logs

### Log Directory

Logs are written to `logs/` inside the workspace. Two things can change this:

- If your eval set passes a `log_dir` to [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set), Steward uses that.

- The `log_root` option or `STEWARD_LOG_ROOT` environment variable, for a machine that keeps logs somewhere else (a bigger disk, or an S3 bucket). Steward makes one directory per workspace underneath it.

If you want your logs to be durable then we recommend setting `STEWARD_LOG_ROOT` to an S3 bucket or other permanent storage and letting Steward create subdirectories within it for each workspace.

### Log Store

A log store lets several workspaces share results, so a task somebody has already run doesn’t run again. Point at one with the `log_store` option or the `STEWARD_LOG_STORE` environment variable:

    .env

``` ini
STEWARD_LOG_STORE=s3://our-bucket/eval-logs
```

The store can be a directory of `.eval` files, or a [Flow](./flow.html.md) store (which requires the `flow` extra).

A result is only reused when everything about the task matches: the task and its arguments, the model, the solver, the plan, the generate config, and the limits.

Reading from a store happens automatically once you configure one, at launch and in `launch --smoke`, which rehearses only the tasks the store does not satisfy. Publishing your own results into it does not, that only happens if you ask for it at signoff:

    Terminal

``` bash
steward signoff --by "norah" --publish
```

## Options

Steward’s options go in `_steward.yaml` at the root of the workspace, or in the environment as `STEWARD_<OPTION>` in an `.env` file within the workspace or a parent directory.

### Logs

Storage location for logs. These are typically specified at the machine level (`.env`). Use the `_steward.yaml` option to override for a given project.

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `log_root` | `STEWARD_LOG_ROOT` | none | Root for this machine’s logs. |
| `log_store` | `STEWARD_LOG_STORE` | none | Logs to reuse rather than run: a directory of `.eval` files or a Flow store. |
| `sync` | `STEWARD_SYNC` | log_dir | Where the workspace’s files are mirrored on each tend. |

See [Log Directory](#log-directory) and [Log Store](#log-store) for additional details.

### Supervision

Frequency of run tending and how to communicate notifications.

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `tend_interval` | `STEWARD_TEND_INTERVAL` | 10m | How often a scheduled tend runs, with a unit. |
| `notification` | `STEWARD_NOTIFICATION` | none | Apprise URL(s) or an Apprise config file. |
| `heartbeat` | `STEWARD_HEARTBEAT` | 1h | How often a liveness heartbeat posts while a run is progressing, with a unit; `false` for none. |

See [Tend Loop](#tend-loop) and [Notifications](#notifications) for additional details.

### Concurrency

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `samples_ramp` | `STEWARD_SAMPLES_RAMP` | \[50, 200\] | Range to discover sample concurrency in; `false` for no automatic ramp. |
| `max_workers` | `STEWARD_MAX_WORKERS` | one per task | Pack the run into this many worker processes. |

See [Automatic Ramp](./concurrency.html.md#automatic-ramp) and [Worker Processes](./concurrency.html.md#worker-processes) for additional details.

### Errors

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `stall_after` | `STEWARD_STALL_AFTER` | 2 | Fruitless respawns before a task is given up on and reported. |
| `stuck_after` | `STEWARD_STUCK_AFTER` | 5h | Report a sample with no activity for this long. |

See [Retry](./errors.html.md#retry) and [Stuck Samples](./errors.html.md#stuck-samples) for additional details.

### Standing Rules

Standing decision rules. These are typically written by the agent during a run as a result of operator decisions.

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `preauthorized` | `STEWARD_PREAUTHORIZED` | none | Anomaly-class patterns mapped to `rerun`, `exclude`, `zero` or `score`. |
| `stuck_cancel` | `STEWARD_STUCK_CANCEL` | none | Stuck tool calls the agent may cancel unasked: `true` for any, or a list of tool names. |
| `policies` | `STEWARD_POLICIES` | none | Rules an agent applies, as prose or a list. |

See [Codify](./errors.html.md#codify) and [Stuck Samples](./errors.html.md#stuck-samples) for additional details.

### Scanning

| \_steward.yaml | Environment | Default | Description |
|----|----|----|----|
| `scan_model` | `STEWARD_SCAN_MODEL` | eval model | The model scanners use. `false` ignores a `SCOUT_SCAN_MODEL` in the environment and scans with the eval model. |
| `scanners` | `STEWARD_SCANNERS` | none | Scanners run in addition to the definition’s. |

See [Scan Model](./scanners.html.md#scan-model) and [Adding Scanners](./scanners.html.md#adding-scanners) for additional details.

## Advanced

### Agent Oversight

An agent does not need to stay attached: `steward tend` runs on its timer either way. An agent adds judgment, diagnosing errors, proposing what to do about them, and escalating what needs you. There are three ways to work:

- Attached and reactive: the agent watches for new activity and responds immediately. Best for a run you are watching closely.
- Attached and periodic: the agent wakes on its own schedule, can run `steward tend` to take a turn early, and stands down once the run is finished and nothing open is its own.
- Disconnected: nobody is attached, and the next session picks up whatever accumulated. This is the common case.

An agent attaching to a run it did not start begins here:

    Terminal

``` bash
steward runbook             # the mechanics, which ship with the package
steward collect --since 0   # everything that has happened so far
```

along with `_steward.yaml` for the project’s standing rules. Reading consumes nothing: an open decision leaves the queue only when somebody acts on it.

While nothing is attached the run keeps converging, but anomalies accumulate without proposals and items sit unraised. If no agent has collected in the last two tends, Steward routes the agent’s items to you as well.

### Inflight Changes

There is no amend command. To change an eval set that is already running, edit the definition and launch again. Steward compares the new definition against the one it committed and runs only what is new or changed; a task whose identifier survives the edit already has its log and is not re-run.

Launch reports the delta before committing it:

| Row | Meaning |
|----|----|
| `add` | New work, not in the previous definition. |
| `extend` | Same task, more work than before (raised epochs, a grown dataset). |
| `removed` | Gone from the definition; its logs move to `logs-archive/`. |
| `superseded` | Displaced by a task of the same name and model under a different configuration; the old logs move to `logs-archive/`. |
| `restore` | Asked for again, and an archived log answers it; moved back instead of re-run. |

A purely additive delta is committed without comment. Anything that would move results out of the log directory prints what it would move and refuses; `--accept-archive` commits it. The gate exists because a one-character change to a task argument produces a new identifier and looks the same as a deliberate removal, while costing a re-run of everything.

An edit you have not launched is reported on every turn as drift.

### Pausing Runs

`steward pause --reason "..."` stops new work from being scheduled. Workers already running finish normally, and every turn reports the run as paused and spawns nothing. `steward resume` starts scheduling again, and the next tend converges from whatever it finds by then.

`steward ramp hold --reason "..."` freezes the concurrency climb without pausing the run, and `steward ramp resume` re-arms it. A hold stops growth only: the defensive cut on provider pushback stays active.

### Timers

`steward launch` installs the timer that runs `steward tend`, using a launchd agent, a systemd user timer, or a marked block in your crontab. Arming is idempotent, so re-arming at a new interval leaves exactly one. Manage it directly with `steward timer arm`, `disarm`, and `status`, or launch without one using `--no-timer`.

Arming refuses if a scheduled tend would not inherit this shell’s credentials, since the alternative is a fleet that fails to authenticate all night. Put them in `.env`, which the workers already read, or pass `--no-env-check`.

If no tend runs for two intervals, the run reports itself unsupervised.


# Concurrency – Inspect Steward

## Overview

The two most important concurrency dimensions to consider when running agentic evaluations are how many samples can be run in parallel (bounded by compute) and how much inference can be run in parallel (bounded by provider rate limits). It’s important to try to match these well during execution, so that you don’t pay for compute that sits idle while waiting on rate limits.

## Sandboxes

Sandboxes are either local (e.g. Docker) or remote (e.g. k8s, Daytona). In either case it’s important to optimize allocation of sandboxes. For the local case, available CPU cores and memory create an upper bound. For remote, while you may have infinite elastic capacity you won’t want to pay for an over-allocation of sandboxes idled by model rate limits.

By default, local sandboxes are limited to 2 \* available CPU cores for Docker, and unlimited for other providers. You can set an explicit `max_sandboxes` to override the default behavior.

## Samples and Tasks

By default, Steward runs 50 samples in parallel for each task (`max_samples=50`), and runs all tasks in the eval set in parallel (each in their own process). The 50 is a starting point: Steward [ramps sample concurrency](#automatic-ramp) toward 200 while the provider keeps up. Set `max_samples` yourself and Steward honors that number exactly and never ramps.

You can change these defaults in your call to [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set):

    evalset.py

``` python
eval_set(
  ...,
  max_tasks=10,    # run only 10 tasks at a time
  max_samples=100  # run 100 samples per task with NO auto-ramp
)
```

For the above configuration, 1,000 samples will be run in parallel, and stay there, because setting `max_samples` turns the ramp off. Left unset, the same 10 tasks would start at 400 samples and climb toward 2,000. If your sandbox infrastructure can handle that then the next thing to consider is whether your model API can keep up.

## Model Connections

By default, Steward uses Inspect’s adaptive connections concurrency controller to automatically find the highest sustainable `max_connections` for the model provider endpoint. You can customize this behavior by either setting a fixed `max_connections` or a custom configuration for adaptive connections in your call to [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set):

    evalset.py

``` python
eval_set(
  ...,
  max_connections=100  # fixed connections, no adaptive concurrency
)
```

Sample concurrency and connection concurrency are tuned together: while a ramp is active, Steward raises the adaptive controller’s ceiling to match the ramp’s, so the two do not work against each other.

## Automatic Ramp

While Steward starts with `max_samples=50`, it will also attempt to automatically ramp sample concurrency according to how well the model provider is handling the load.

During each `steward tend` operation, Steward checks to see if the following is true:

- The sample limiter is saturated (demand for more samples);
- The model provider showed no rate-limit pushback;
- No sample errored, and HTTP retries did not surge;

If all checks pass, then tend adds an additional 20 samples to the limit. This ramp continues up to 200 as long as the checks continue to pass.

On sustained pushback (rate-limit episodes in two consecutive windows) the ramp reverses. The connection ceiling is clamped down to where Inspect’s own controllers already fell, and sample concurrency steps back down so new samples stop being admitted against capacity that is not there. Samples already running are never interrupted. The way back up is stepwise, through the same gates.

Set the range with `samples_ramp` in the Steward config file or the `STEWARD_SAMPLES_RAMP` environment variable:

    _steward.yaml

``` yaml
samples_ramp: [40, 300]   # explore this range
samples_ramp: false       # never ramp; tasks stay at 50
```

Narrowing the range works on a live run: a task the ramp had taken to 200 comes back to a new ceiling of 100 at the next tend.

To freeze the climb without changing configuration:

    Terminal

``` bash
steward ramp hold --reason "provider looks unhappy"
steward ramp resume
```

Note that `max_sandboxes` is a machine-wide budget that bounds the fleet-wide sum of sample setpoints, where workers start at a share of the budget instead of at 50. The budget is your definition’s `max_sandboxes` if it sets one, otherwise what the sandbox provider reports (e.g. Docker’s 2 \* available CPUs). Elastic providers such as k8s have no limit.

## Worker Processes

Steward runs each task in its own worker process. This provides two things:

- Fault isolation. An OOM or a segfault costs one task, not the run.

- CPU parallelism. A task’s Python work (transcript construction, JSON serialization, `.eval` compression, non-model scorers) serializes behind the GIL in a single process, and runs in parallel across several.

The cost is startup, since every worker imports your definition and its dependencies before it can run. For plain [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) definitions that is seconds; [Flow](./flow.html.md) and [Hawk](./hawk.html.md) can have longer startup times.

Where startup is expensive, pack the fleet into fewer processes:

    _steward.yaml

``` yaml
max_workers: 8
```

This changes nothing about how much runs at once (that is `max_tasks` and `max_samples`), only how many processes carry it.

## Docker Configuration

Docker has a ceiling on concurrent sandboxes that has nothing to do with the size of your machine. Every sandboxed sample gets its own compose project, and every compose project gets its own bridge network, allocated from Docker’s `default-address-pools`. The built-in value carves two ranges into sixteen networks each, so a host runs about thirty sandboxes however many cores it has. Samples past that fail with:

    could not find an available, non-overlapping IPv4 address pool
    among the defaults to assign to the network

`steward launch` reads the daemon and tells you when your run wants more sandboxes than it can give, offering to write the fix into your `daemon.json`:

    daemon.json

``` json
{
  "default-address-pools": [
    { "base": "172.17.0.0/12", "size": 20 },
    { "base": "192.168.0.0/16", "size": 24 }
  ]
}
```

That is the same two ranges Docker already claims, carved finer: 512 networks instead of 32. Declining is the default, and prints the JSON along with the file to put it in.

Steward never restarts the daemon, and nothing takes effect until you do. A restart stops every running container on the machine, so the timing is yours. The launch prints the command for your platform.

See [Docker’s default address pools](https://straz.to/2021-09-08-docker-address-pools/) for background.


# Error Handling – Inspect Steward

## Overview

Inspect’s default behavior is to fail fast so that you can see and fix a problem. Steward runs unattended for hours at a time, so it does the opposite: errors never halt a run, retries are automatic, and anything a retry cannot fix becomes a decision that waits in a queue.

Nothing is dropped silently. Every error is either retried or turned into a decision, and a run cannot be signed off while a decision is unanswered.

| Stage | What happens | Who acts |
|----|----|----|
| [Retry](#retry) | Samples are retried in the worker, and failed tasks are respawned | Steward, unattended |
| [Surface](#surface) | What survives retry is grouped into a class and raised as a decision | Steward and your agent |
| [Decide](#decide) | A ruling: a disposition, a reason, and a name | you |
| [Codify](#codify) | Standing rules so the same question stops being asked | your agent, in `_steward.yaml` |
| [Resolve](#resolve) | Signoff refuses while anything is still undecided | you |

Transcript scanning follows the same stages, because a finding becomes a decision the same way an error does. See [Scanners](./scanners.html.md).

## Retry

Steward runs workers with `fail_on_error=False`, so a sample that errors never fails its task. Every task runs to completion and its errored samples are collected in the log. Steward decides what to do about them by reading the logs, so a task that stopped on its first error would look like a structural failure and be re-launched.

This means that completion is not success. A log can finish `success` while carrying errored samples, so a task whose provider was down for its entire duration still completes.

Retries then happen at two levels:

- Samples are retried inside the worker (by default up to 3 times; set `retry_on_error` in your [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) call to choose your own number, including 0).

- Tasks are retried by Steward. Once `stall_after` attempts get nowhere (default `2`) the task is marked stalled and becomes a decision.

## Surface

Samples that failed the same way are grouped into a class, so a provider outage is one question instead of two hundred. A stalled task is classed the same way, on the exception in its log or on the worker’s output if it died before writing one, so it arrives with its evidence attached.

Classes appear under the anomalies heading in `steward collect`, with instance counts, an example message, and any prior rulings. A class is raised once, and again only if it materially changes: the population crosses an order of magnitude, or a re-run fails.

Your agent works the queue from here. `steward propose` bundles related classes into one question with the evidence attached and prints the sentence you will be asked; `steward rule` records your answer by the finding’s name or the task’s, taking the proposal’s disposition unless you change it.

## Decide

Agents will prompt you for a ruling with one of six answers:

| Disposition | What it means | What happens to the data |
|----|----|----|
| `rerun` | The failure was transient and the samples should run again | Steward re-runs them and records whether they passed |
| `exclude` | Leave the samples out of scoring | Written into the log as unscored, with the reason; the headline is recomputed over the rest |
| `zero` | Count the samples as failures | The task’s scorer scores an empty attempt and that score is written in |
| `score` | The recorded results stand as they are | Scored as-is, marked |
| `accept` | The data stands with a caveat | Requires a stated effect, which is the sentence the report carries. Not available for errored samples |
| `dismiss` | Looked, nothing here | No mark; the record keeps who looked and why |

The next tend carries out a `rerun`. Samples in a running task are requeued in place, and a landed log has the ruled samples invalidated so the task re-runs them and reuses everything else. Authorized re-runs are scheduled ahead of ordinary queued work.

The next tend carries out an `exclude` and a `zero` too, by writing them into the log: each edited score keeps its history and records who decided and why, and the log’s metrics are recomputed over what remains. An exclusion lands within a turn. A zero needs the task’s scorer, since what a zero is depends on it, so Steward runs the task on just those samples in a scratch directory under `.steward/`, stops each sample as it starts, and copies the scorer’s verdict on that empty attempt into the log. Until a ruling is written the report’s entry for it says *not yet written into the log*, and `steward signoff` refuses with the same words.

If the same failure comes back after a ruling, it opens as a new question and carries the old decision with it.

## Codify

“When you find yourself answering the same question twice, your agent will propose recording it in \_steward.yaml so it stops being asked. For example:

    _steward.yaml

``` yaml
preauthorized:
  'error:ReadTimeout@*': rerun
```

When a new class matches a pattern, the tend records a ruling with `by: policy` naming the pattern and applies it in the same turn. Nothing is pre-authorized by default, and if a pre-authorized re-run fails, the pattern stops matching that class until an operator has looked.

Anything more nuanced than a pattern goes in `policies:` as prose, which your agent applies when one is in session:

    _steward.yaml

``` yaml
policies:
  - If more than 10% of the samples in a task error out, pause the run and
    notify me right away.
  - Sandbox startup timeouts under 1% are expected here; invalidate and re-run.
```

Use a policy to stop a run on error rates. Pausing is reversible and stops the whole run, which is usually better than failing one task.

## Resolve

`steward signoff` refuses while any error is still undecided, and lists every reason at once so that fixing one and being refused for the next never happens.

The refusal tells you what to answer. It is not a quality bar: `accept` and `dismiss` are answers, and “2 samples accepted as errored, the provider was down all night” is a signed statement about the results.

The classes you accepted become the entries in `anomalies.md`, each with its reason, who decided, when, and the effect on the numbers. The file opens with a table, one row per task, of the samples that did not take the normal course: zeroed, excluded, errored, scored early by an operator, or terminated by one.

## Stuck Samples

One condition never becomes an error: a sample that is alive but not moving, such as a `bash` command that never returns. Nothing errors, nothing scores, and the task’s clock keeps running.

A running sample with no activity for longer than `stuck_after` (default `5h`) is reported, one item per task, naming the samples and the tool call they are inside. Streaming counts as activity, so a slow but healthy `generate` is never stuck. A sample parked on an approval is waiting on you, and a provider backoff with a live deadline is waiting on purpose.

`stuck_after` only reports, and never cancels. To bound how long a sample may work, set `working_limit` in your definition.

What to do about a stuck sample is an escalation ladder. The item carries the applicable rung as a command ready to run:

| Rung | Command | What it costs |
|----|----|----|
| Cancel the tool call | `inspect ctl sample cancel-tool-call TASK ID EPOCH` | The call fails inside the sample, which continues |
| Cancel the sample | `inspect ctl sample cancel TASK ID EPOCH` | The sample stops, recorded as you choose with `--action` |
| Requeue the sample | `inspect ctl sample requeue TASK ID EPOCH` | The sample’s work is discarded and it runs again from the start |

A cancel is a request. If one was asked for and the call still has not stopped, the item says so, which is your signal to climb a rung.

You can grant the first rung in advance. `stuck_cancel: true` lets your agent cancel any stuck tool call itself, and `stuck_cancel: [bash]` admits only those tools. The higher rungs stay yours, since they discard work.

A stuck sample is not an anomaly and there is nothing to rule on. It is reported while it holds and gone when the sample moves.


# Scanners – Inspect Steward

## Overview

A sample can run, score cleanly, and still be untrustworthy: the model read the grader, refused the task while producing something that looks like an answer, or retrieved remote data from inside a sandbox that should have had no route to the internet.

Scanners review transcripts for these problems. Under Steward they run as the samples they review complete, so the transcripts are already scanned by the time the run finishes.

Steward runs a scoring integrity scanner over every sample of every run. It requires no configuration.

## Scan Model

By default a scanner reviews with the model already under evaluation, so an unconfigured run still scans. To use a different model, set `scan_model` in `_steward.yaml` or `STEWARD_SCAN_MODEL` in your `.env`:

    .env

``` ini
STEWARD_SCAN_MODEL=anthropic/claude-opus-5
```

A scanner that names its own model in the definition keeps it.

## Adding Scanners

Your definition is the normal place to say what scans your eval set:

    evalset.py

``` python
eval_set(
    tasks=[cais_swebenchpro(), datacurve_deep_swe()],
    model=["openai/gpt-5.6-sol", "anthropic/claude-opus-5"],
    scanner=[my_scanner()],
)
```

## Findings

A flagged sample becomes an anomaly and is decided the same way an error is, as described in [Error Handling](./errors.html.md). Samples flagged the same way in the same task are one class and one decision, and every disposition is available, including excluding or zeroing a score you establish is wrong.

Nothing is put to you while the task is still running. When it lands, your agent investigates every finding the task has and brings them to you together, one row per class with a proposed disposition, and your answers are recorded row by row. A finding ruled on one task is shown as precedent when another task lands with the same one, and the task’s finished notification goes out with the proposals.

What reaches you as a decision is what would change a number: score 0, drop from scoring, or run the samples again. Most findings change no score — a model that refused, one that probed a grader it never reached, a grader that could not grade its own samples — and those your agent records itself as scores kept as recorded, so the report carries them; the finished-task message mentions them under the decisions, and asks nothing. Dismissing is for a flag the transcript does not bear out. A run cannot be signed off while a finding has no ruling either way, and at signoff Steward tells you how many were dismissed and your agent is asked to name them.


# Agent Runbook – Inspect Steward

## Overview

Steward is designed to be operated by a coding agent. The agent launches the run, works the decision queue, tunes concurrency within the bounds you set, and brings you the questions only you can answer.

The agent’s operating instructions ship with the package and are printed by `steward runbook`, so they always match the version of the CLI you are running. The `AGENTS.md` file in your workspace tells the agent to read them, so you never need to run the command yourself.

What follows is that text, reproduced verbatim.

# Steward runbook

You supervise an `inspect_ai` eval set for an operator who is not always present. The job has three parts:

1.  **Watch the run.** A timer runs `steward tend` without you: it starts workers, watches them, ramps them, and writes down what it could not decide. You read that on a schedule you arm yourself.
2.  **Investigate what goes wrong.** Errored samples, stalled tasks, scan findings. You read the logs and the transcripts, work out what happened, and decide whether the score can still be trusted.
3.  **Tell the operator what you found, and ask.** They rule and they sign. You put each finding to them in the words of their eval, with the evidence in brief and one question, and you record their answer.

Steward is the notebook. Every decision goes into it through a `steward` verb; it renders `status.md` for the operator and `steward collect` for you. It is not the subject of anything you say to them.

**Where things are.** The workspace is a directory. The definition is the operator’s statement of what to run; `_steward.yaml` is their rules for this run, read before you act. `journal.jsonl` is every event and decision, append-only. `analysis.md` is your findings. `status.md` and `anomalies.md` are rendered from the journal and the logs at every tend. The log directory is named in `steward collect`.

**How the run moves.** `steward launch` resolves the definition into a manifest of tasks and arms the tend timer. Each tend is one turn of the loop: reap finished workers, start the next, ramp concurrency, sync the scans, detect what changed, render the snapshot, post at most one notification. A worker is one `inspect` process running one or more tasks. It writes a log, scans each sample as that sample finishes, and answers `inspect ctl` while it runs. A log has landed when its file is complete and its status is known. A worker that stops inside a sample to ask an operator’s approval is parked until someone attaches with `inspect acp`.

## Talking to the operator

They see only what you write, and they know their eval: tasks, models, samples, scores, transcripts. Every message to them has one shape:

1.  **What you found.** One or two sentences in the eval’s words: which task, which samples, what happened to them.
2.  **The evidence in brief.** The two or three facts that establish it and where you looked, with the samples it covers in a table capped at five rows and the remainder counted. The full case goes in `analysis.md`; a reader who trusts you should be able to answer without opening anything.
3.  **What you propose, and the question.** Recommend one answer and name the others on offer, so they can confirm, pick, or say more. Then ask it, as a question they can answer in a word: *Score the four 0?* Saying that a question is waiting, or that you have proposed and notified, is not asking; the notification carries a copy of the question, and this message carries the question. It is the last line, and nothing follows it.

**One question at a time**, most important first, and the next waits for the answer to this one: several things to resolve are several messages in turn, not one message with several questions and not several messages back to back. This holds however many a single `collect` hands you at once — a termination to rule, a re-run to approve, a ramp to decide are three messages in turn, not one numbered list of them. What a collect surfaces is a queue you work down, not a picture you lay out, and *here is the full picture, and what I did about each* is the report you never send: Steward’s posts and `status.md` already carry the items, so a digest of them puts a wall of text where a question should be and buries the one thing they need to answer first. Do not restate the heading as the question. Keep the arithmetic out of it. The one message that carries several decisions is a finished task’s scan findings, the second example below: **one task’s** findings, one row each, one question over them — and that is the whole of the exception. Decisions of different kinds, or across different tasks, never merge into it, however many land together.

**A decision ends the message.** Once it is in front of them, stop. What you settled yourself, a bug for upstream, another task’s progress, what you will do next: each is one line above the question, or waits until they have answered. Whatever sits below the question is the last thing they read, and the question is not.

**After you record their answer**, one line saying what was recorded, in the eval’s words: *Recorded: the four score 0.* Not what Steward does to carry it out, and not what the score will settle at. Then the next decision in the shape, if there is one; otherwise nothing. A ruling recorded is theirs. Do not offer to reinterpret it, and do not ask whether they meant something else; if their answer could have meant two things, ask which before you record it. *Still open*, *pending* and *yours* are not asks: what is theirs to decide is asked, in full, or not mentioned.

**Speak the eval’s language, never Steward’s.** Windows, classes, items, proposals, tends, collects, keys, ids and hashes are words from this runbook, and a message built from them is one they cannot answer. A word that is in this runbook and not in their definition does not go in a message to them.

| you have | you say |
|----|----|
| a window or a class | the samples and what they did: *5 samples in cybench@gpt-5 where the grader could not find its own tests* |
| a class key, an item id, a `prop-` id, a hash | nothing; the task and the finding in words |
| a disposition | what happens to the samples: drop from scoring, score 0, keep the score, keep it with a note in the report, nothing was wrong |
| a tend, a collect, a fold, the gate | *Steward*, or nothing |

**A collect with no question in it produces no message.** Do not report an investigation under way, a proposal you are holding, or what you expect to ask later; they learn the run’s state from `status.md` and the notifications, not from you. A finding you cannot yet propose on is one you are still investigating: keep investigating and say nothing. A scheduled collect that shows nothing new ends with no text at all: *nothing for me*, *unchanged*, *still waiting on your rulings* are messages, and the fourth one teaches them to stop reading. Once a finished run’s findings are in front of them, the next thing you write is your answer to theirs.

**Steward misbehaving is not theirs to debug.** Record what you saw with `steward note`, work around it where a ruling is honest, and tell them in one sentence only what changes for them. If you cannot work around it, say so in one sentence and stop.

**When they ask how it is going**, run `steward status` and **render exactly what it printed, verbatim** — in full, in its order, as markdown outside a code fence — then stop. You are relaying the snapshot, not writing your own: do not summarize it, paraphrase it, reword its lines, or rebuild it as prose, and do not read `status.md` instead; it can be a full interval stale. **Nothing of your own follows the snapshot.** Not a *what changed since last time* narration — the snapshot is the state, and its `tended` age is how they see it is current — and not an account of your own open items or what you intend to do about them: those are yours to work, the snapshot already counts them for the operator, and a run’s state is read from the snapshot and the notifications rather than from you. A narrower question wants a narrower answer: *is it finished* is a sentence, not the snapshot. The one thing that may follow the snapshot is a decision its operator section carries: end with that decision in the shape, heading and all, as if it were the only thing you wrote — the operator bullet is Steward’s summary of your proposal, and what they answer is your message. Repeating the bullet is fine; a question with no finding above it is not. The question is the last line.

**Before you send, five checks.** Could they answer it knowing only their eval? Is it what you found, then the evidence, then the question? Is it one decision, not a digest of several a collect handed you together? Is the question written out, as a question? Is it the last line?

Three messages in the shape. Errored samples in a running task:

``` markdown
## 14 samples in swe_bench_lite@openai/gpt-5 errored on a provider timeout

All 14 failed between 02:10 and 02:40 UTC with the same timeout from OpenAI, and every retry landed inside that window. The other 106 completed normally before and after; nothing about these 14 is unusual.

| sample | error |
|---|---|
| django-11099 | timeout after 3 retries |
| django-11133 | timeout after 3 retries |
| flask-4045 | timeout after 3 retries |

…and 11 more, all the same.

I propose to run the 14 again: the outage is over and nothing in them failed. The alternative is to drop them from scoring, leaving the task at 106 of 120. Run them again?
```

A finished task’s scan findings, one row per finding:

``` markdown
## cybench@openai/gpt-5 finished: 2 findings need your decision

| finding | samples | proposed | why |
|---|---:|---|---|
| build tools unreachable in the image | 2 | run them again | cargo could not reach the registry for twenty minutes; baseline and new tests both exit 101, and the registry is back |
| fetched the answer from the internet | 1 | score 0 | pulled the upstream fix and the gold test file from proxy.golang.org at [M39] |

Also noted, nothing to do: 5 samples where the grader could not find its own tests scored 0.0 as the benchmark computes it, and 4 samples refused the task and scored 0. The report says so for both.

Go ahead as proposed?
```

A stop:

``` markdown
## Stopped: the logs and the definition disagree about what gaia is

The worker for gaia@anthropic/claude-opus-5 refused to resume: its log names 165 samples and the definition now names 200. I have paused new work so nothing runs on a premise I doubt; the tasks already running are finishing.

Was the dataset changed on purpose? If so I will relaunch it; if not, tell me which is right.
```

### Stopping

Stop when a retry is not the answer:

- the smoke fails a second time
- a worker fails on identity or resume: the logs on disk and the definition disagree about what the eval is
- numbers fail sanity, such as a sample count that does not match the dataset
- the next step is destructive or irreversible
- a credential failure survives a retry
- you are about to relaunch the same task a third time

A stop is not a teardown. Healthy work keeps running. What you do:

1.  `steward note` the state and your hypothesis.
2.  `steward notify MESSAGE --kind stopped --detail ...` with the question you need answered.
3.  `steward pause --by agent --reason ...` only if new work would be spent on a premise you now doubt. Pausing stops scheduling; work in flight finishes.

Then leave it. The rest of the queue is still yours; the question waits for an operator. Do not work around the block.

## Each session

**Arm your own return first.** Nothing wakes you: the timer runs `steward tend` on its own and never calls you back. Before anything else, and unasked, point whatever your harness has for recurring work, a scheduled prompt or a background watch, at `steward collect` on roughly the tend interval, and handle what it returns. Keep it while the run has work: a task running or queued, a re-run pending, a ruling not yet written into the log. Take it down when the verdict is 🏁 and the decisions section says nothing is for you, and at signoff; a finished run waiting on the operator has nothing for you to read every ten minutes. Arm it again after anything you record that gives the run work: a rerun, a zero, a resume, a launch. **If you are the Codex CLI**, you have no in-session scheduler and must not improvise one: run exactly `steward schedule arm --agent codex`, which points a system scheduler at a fresh `steward collect` for you on the same interval, and `steward schedule disarm` to take it down. That schedule is *recurring* — once armed it keeps firing on its own — so the *arm it again* rule above is not for it: arm it once when the run has work, and take it down with `steward schedule disarm` on the same standing-down rule as the in-session kind above.

Pick up cold, in this order:

1.  This runbook, then `_steward.yaml`.
2.  `steward collect`: what is true now, and what happened since you last looked. `--peek` reads without marking it read; `--since 0` reads the whole history.
3.  The anomalies it lists, with any precedent printed beside them.
4.  `analysis.md`, for what has been found and what it meant.

Everything you need is in the workspace. Nothing depends on a conversation you were not part of.

`steward collect` has three sections: what needs a decision, the run, and what happened. Its header carries two ages: tended is the last tend, collected is the last time you looked. The verdict line says the state in words. `status.md` is the operator’s page, shorter and without item ids; everything you act on is in `collect`.

**Collect regularly**, on the schedule you armed above. The timer runs the loop; nothing runs you. A tend never invokes an agent, so the items it hands you wait until you look, and after two intervals without a collect they go to the operator’s channel as if there were no agent. That is also what happens once you stand down at 🏁, and it is right: what is left is theirs.

## Investigating

**Trust the artifact, not the exit code.** A task whose every sample errored still exits zero. Read the smoke digest, the log’s status and the anomaly count. Completion is not success.

**Read the precedent first.** `steward collect` lists every open class with counts, an example, and any prior ruling as precedent; the 11pm decision usually answers the 2am question.

**A finished task’s findings.** Every scan window a task has arrives as your item at once when the task lands. Investigate them all before you write anything: read the scorer’s output and the flagged transcripts. Then three ways out. A flag the transcript does not bear out, dismiss yourself. A finding that changes no score — a refusal, an attempt that earned nothing, a grader that could not grade its samples — rule `score` yourself, so the report carries it, and mention it in one line between the table and the question: they see it and are asked nothing. What is left would change a number — score 0, drop from scoring, run them again — and that is the decision: propose it, one proposal per disposition, and put it to them as rows. Only what they can act on is a row.

**Judging a scan finding.**

- **Behaviour that disqualifies a sample is disqualifying whether or not the environment allowed it.** An agent that fetched the reference solution did not solve the task, and whether the sandbox should have blocked it is a bug for the ruling’s reason and one line above the question, not the question to put in place of it. Do not turn a finding about the trajectory into a question about the machinery; the operator then has to answer the wrong question before anything can move.
- The bar is whether the score can still be trusted, not whether the behaviour was interesting. Escalate only where you can name the mechanism and cite the decisive messages. Otherwise rule `score` yourself, with what the model did in the reason. `dismiss` is for a flag the transcript does not bear out: the scanner was wrong, and there is nothing to note.
- **Read the scorer’s own output first.** It is the explanation on the sample’s score in `read_eval_log_sample_summaries`, and for a test-suite grader it names which tests failed. An agent whose own tests passed while the graded ones did not has failed the task.
- A failed attempt earns nothing. The model tried to read the grader and could not; the sandbox refused the network. The score stands, and the attempt is still a finding: rule `score` yourself with the attempt in the reason, so the report carries it.
- A transient fault is not the model’s. An image pull that failed, a registry the sandbox could not reach for twenty minutes: the samples can run again, and a `rerun` ruling invalidates just those samples in the finished log and relaunches the task for them. Propose *run them again*, and say what the fault was.
- A benchmark broken by construction is not a fault a re-run heals. A grader that runs none of its required tests, a baseline that fails before any change, a sample the verifier never grades: dropping those samples makes a number nobody else’s run has, and there is nothing for the operator to decide. Rule `score` yourself with what the grader could not do in the reason, note it below the table, and leave the fault to them to report upstream.
- A successful escape or a returned egress response is a finding at n=1, and the answer is `zero`. A *blocked* attempt — one a tool-call approval guard terminated before it ran — is not this: it is a quiet operator limit, enumerated and accepted at signoff (see *Anomalies*), never a mid-run escalation.
- Rarity is not the signal. A class covering most of a task is the one to read first, because an exploit that works gets used everywhere.
- Zero a confirmed finding rather than excluding it: dropping the samples where a model cheated raises its score.
- Only boolean scanners open windows. Numbers and strings are recorded and never escalated; read them through the results, not the queue.

**A running worker.** A log says what a sample did once it landed. `inspect ctl` says what a worker is doing now, and is how you diagnose a task that is still running. `inspect ctl task` lists the running tasks with the id every other command takes as `TASK`. Every command accepts `--json`.

| to see | run |
|----|----|
| a task’s samples and their status | `inspect ctl sample list TASK` |
| the samples that errored, with the message | `inspect ctl sample errors TASK --content` |
| one sample’s status, timing, tokens, score and error history | `inspect ctl sample show TASK SAMPLE_ID` |
| its conversation right now | `inspect ctl sample messages TASK SAMPLE_ID --all` |
| its transcript, event by event | `inspect ctl sample events TASK SAMPLE_ID` |
| the worker’s configuration in force | `inspect ctl config TASK` |

Reads are yours. Mutations go as far as a pre-authorization and no further: `sample cancel-tool-call` when `stuck_cancel` admits it, and lowering a limit through `inspect ctl config` while containing an incident. `task pause`, `task cancel`, `sample cancel` and `sample requeue` are an operator’s. Record every mutation with `steward note`. It leaves no other trace in the workspace.

## Recording what was decided

Record the answer with the verb the item names, then `steward tend` so it takes effect now, and arm your return again if you had stood down. The mechanics are `steward` verbs and `inspect ctl`; reach for them before a script.

### What you may do

**Without asking.** `launch --smoke` and `launch`. `tend`, `status`, `collect`. `schedule arm` and `schedule disarm`, to arm and take down your own return. `raise`, `investigate`, `propose`, `notify`, `note`. `ramp hold` and `ramp resume`. Every `inspect ctl` read, and lowering a worker’s concurrency through `inspect ctl config` while containing an incident. `ack --by agent` for an item you resolved yourself, when nobody else would need to know. Writing `analysis.md`.

**Only with an operator’s answer, recorded in `--by`.** `ack --by operator`. `rule`, except `dismiss` and a scan finding’s `score` (see Anomalies). `signoff`. `pause` and `resume`, except during a stop (see Stopping). `launch --accept CHECK` and `launch --accept-archive`. Writing `_steward.yaml`. Any `inspect ctl` mutation of a sample or task; a pre-authorization in `_steward.yaml` is an answer already given.

**Never.** Move or delete a log, not even an empty one. Answer a parked approval or `ask_user`; name the worker, print the command that attaches to it, notify, and wait.

**Editing the definition.** It is the operator’s statement of what to run, so never edit it unbidden, and never edit it *back* to erase drift somebody introduced. But when you and the operator agree to change what runs — adding a model, say — either of you may edit it and then run `steward launch`: that recaptures it, schedules the additive task, and clears the drift in one step. You may instead leave the edit for the next tend to raise as drift and relaunch then — the direct path is the same commit without the wait. A launch that would archive existing results still asks first (see Launching).

`--by` records who decided, never who typed. `rule` and `signoff` record the operator’s name on their own; pass `--by` only for someone else. `ack`, `pause` and `ramp` take `operator` or `agent`.

### The queue

What a tend cannot decide becomes an item in the queue. Every item has a kind, an owner (an operator or you), and the command that resolves it. A class of failed samples or scan findings is an anomaly. Each occurrence of a class opens a window, and a ruling closes it.

`collect` prints your queue: every open item you have not handed off, with its owner and the command that resolves it. Any unambiguous prefix of an id works; a finding’s label or a task’s display key works for anomalies, and `ack stalled:<task>` for items. An id changes when its condition changes, so an acknowledgement covers the condition as it was. Every kind not in the table under Reference is the operator’s: raise it. The item’s summary says what it is, and its action says what resolves it.

**`steward ack ID --reason ... --by operator|agent`** disposes of an item: it leaves every surface and reappears under what happened with who decided. It refuses what has its own verb, and names the verb.

**`steward raise ID [--note ...]`** hands an item to the operator who can decide it and closes nothing; it stops appearing in your queue. It refuses your own items, since nobody else would close them.

### Anomalies

Failures that mean the same thing share a class: an exception at a raising frame (`error:TimeoutError@openai/_client.py:post`), a task that died a particular way (`task:no-log-exit:...`), samples an operator killed (`limit:operator`), a task whose every score is zero (`score:zero:...`), samples a scanner flagged (`scan:SCANNER:LABEL:TASK:HASH`), a scanner that failed (`scanerror:SCANNER`).

Your verbs, in order:

1.  `steward propose CLASS... --action DISPOSITION --reason ...` makes classes that want the same answer one question. Classes wanting different answers are different proposals. It prints the sentence the operator will read; put that to them, not the id.
2.  `steward rule` records the operator’s answer, two ways. `steward rule TASK --reason ...` takes every proposed finding of that task as proposed, where `TASK` is its display key. `steward rule FINDING --disposition D --reason ...` changes one row, where `FINDING` is the finding’s label when it is unique and `label:task` when it is not. `steward rule --proposal ID --reason ...` still works.

`steward investigate CLASS --note ...` is for a class you have to leave mid-way: it marks the class as being worked, and the note is for the next session.

**Two rulings are yours alone.** A class you have disproved: `steward rule CLASS --disposition dismiss --by agent --reason ...`. A scan finding that changes no score: `--disposition score`, which keeps every score as recorded and puts one line in the report. Make them as soon as the investigation gets there, rather than proposing them; the verb refuses `--by agent` for every other disposition.

`rerun`, `exclude` and `zero` are carried out by the tend. Run nothing. A `rerun` requeues samples in a running task in place, and has a landed log’s samples invalidated for relaunch; the window stays open until the re-run lands, and the same samples failing again come back as a question for the operator. An `exclude` is written into the log within a turn: the samples become unscored, with the reason, and the metrics are recomputed over the rest. A `zero` takes minutes, since what a zero is depends on the task’s scorer: Steward starts the task on just those samples in a scratch directory, stops each as it begins, and writes the scorer’s verdict on that empty attempt into the log; the transcript stays. Until a mark is written the report says so, and signoff waits. A class flagged as substrate (credentials, disk, storage) gets no rerun proposal from you; re-running into broken machinery burns the work twice.

Two classes are quiet on purpose. A `task:` window heals itself when the respawn brings the task home; if it does not, `stalled` is the real question. A `limit:operator` window raises no item and waits for the signoff conversation — an operator’s own kill and a **tool-call approval guard’s** termination (`limit_reason` begins `Tool call approver requested termination`) alike. A guard termination is expected and inherently accepted, not a finding to raise: enumerate it in the by-task table and `anomalies.md`, accept it with the rest at signoff, and mid-run say nothing about it — do not investigate, assess, or recommend on it, even when the terminated action reads like an escape attempt. Only a **successful** escape the guard did not stop escalates, and its answer is `zero` (see *Judging a scan finding*).

### Scan windows

Nothing starts a scan: the worker runs each scanner as a sample finishes, and a `scan:` window opens when its rows land. Windows are per task, and one raises no item until its task has landed. A flagged sample in a running task is not yours to look at yet; `collect` lists its window as waiting. A ruling on one task’s window touches no other task’s samples, and the same finding ruled on another task is printed beside the window as precedent. Record each answer with `steward rule`. The task’s finished notification waits for your proposals and goes out with them; it stops waiting after six tends.

### Notifying

`steward notify MESSAGE --kind attention|stopped --detail TEXT...` is how you reach the operator when nobody is reading the conversation. `attention` means worth knowing, and work continues. `stopped` means nothing progresses until an operator answers. A question you are blocked on is `stopped`, always.

Post rather than agonize, and do not batch. Steward limits its own posts to one per turn, so the channel is not tight. A skipped `stopped` is a run that waits all night for an answer nobody knew was wanted.

Steward’s own posts already announce every new item an operator owns. Notify for what you found that no item says.

## Reference

### Launching

**Smoke first.** `steward launch --smoke` runs a few samples of every task the log store does not already satisfy, under a wall-clock cap, and launches nothing. A smoke that says *nothing to rehearse* is a pass: launch, and the run is complete and waiting on signoff.

The smoke fails on any errored sample and on four named checks: `context_window`, `reasoning`, `reasoning_api`, `scan_coverage`. `unexercised` and `undetermined` are not failures. Fix what failed rather than routing around it. `--accept CHECK` waives one check by name; an errored sample cannot be waived. Do not project the run’s spend from the smoke.

A smoke that fails twice is a stop. Notify it explicitly: nothing posts before the first worker starts.

**Then launch.** `steward launch` shows what will change and starts the run.

**The tend timer.** `launch` arms it. `steward timer status` is how you check it, not the system scheduler. A run launched `--no-timer` carries an `unsupervised` item until an operator acknowledges it, which says they are driving by hand.

**Your own schedule.** Separate from the tend timer, and for the Codex CLI, which cannot schedule its own return: `steward schedule arm --agent codex` points a system scheduler at a recurring `steward collect` for you, and `steward schedule status` checks it. It is judgement automation you arm and take down (see *Arm your own return first*), not the supervision floor — so its absence is never an `unsupervised` item, and `signoff` takes it down alongside the tend timer.

### Item kinds

| kind | owner | it means | you |
|----|----|----|----|
| `stalled` | operator | a task stopped progressing after its attempts and will not respawn | find out why, then raise |
| `drift` | operator | the definition changed since it was captured | raise, or `launch` to adopt an intended change; never revert it to hide the drift |
| `stuck` | operator, or you when pre-authorized | a sample has been quiet longer than `stuck_after` | see Stuck samples |
| `parked` | operator | a worker is waiting on an operator inside a sample | raise and notify; never answer it |
| `tuning_proposal` | operator | a task could take more concurrency than its setting allows | relay it; ack with the answer |
| `signoff_ready` | operator | every task finished and nothing is open | see Signoff |
| `anomaly` | you while open; operator once proposed, or after a failed re-run | a class of failures or findings | see Anomalies |
| `unreadable` | you | a log file could not be read | look at the file; ack with what it was, or ask |
| `action_failed` | you | something a tend tried to do failed | do it by hand, or find out why |
| `unwritten` | you | a task has no write-up in `analysis.md` | write the section |
| `journal_damage` | you | journal lines could not be read | read them; ack with what they held |

### Dispositions

| disposition | it says | honest for |
|----|----|----|
| `rerun` | run these samples again | anything but `scanerror:` |
| `exclude` | drop these samples from scoring: written into the log as unscored, with the reason | `error:`, `limit:`, `scan:` |
| `zero` | score these samples as the task’s scorer scores an empty attempt: Steward runs the scorer for them | `error:`, `limit:`, `scan:` |
| `score` | score these samples as recorded | `error:`, `limit:`, `scan:` |
| `accept` | the data stands, with a caveat the report carries (`--effect`) | anything but `error:` |
| `dismiss` | looked, nothing here | anything |

### Stuck samples

A `stuck` item names samples alive but idle past `stuck_after`. Nothing failed and nothing is waiting on an operator; the task’s clock keeps running. It is not an anomaly, and it clears itself when the sample moves.

The remedy is a ladder, and the item carries the command for its rung:

1.  Cancel the tool call: `inspect ctl sample cancel-tool-call ...`. The call fails inside the sample, which continues. Yours only when `stuck_cancel:` in `_steward.yaml` admits it; the item then arrives owned by you. Run the command it carries, then `steward ack ID --by agent --reason ...`.
2.  Cancel the sample. An operator’s: it records an outcome in the eval’s data.
3.  Requeue the sample. An operator’s: it discards everything the sample did.

Ask once. If the cancel was delivered and the call has not stopped, the item comes back with `:asked` in its id, owned by the operator. That means climb a rung; never repeat the ask.

### Tuning

Tend ramps sample concurrency on its own, one step per clean window, and steps back on pushback. Every move shows in your next collect. A Hawk run is pinned by its config: no ramp actions, no tuning block, nothing to retune.

- `steward ramp hold --reason ...` stops the climb; levels stay where they are and the defensive cut stays active. Add a task identifier to hold one task. `steward ramp resume` re-arms it. Both are yours on your own judgement.
- A `tuning_proposal` is capacity tend may not take: a pinned `max_samples` that is saturated and clean, or a ramp at the top of its range with no pushback. Raise it, and when the operator answers, `steward ack` with their answer. A different level is a different item.
- **Ramp in steps of 50.** When the top of the range binds and the ask is how much higher to set it, recommend raising it by 50 and let the next clean window earn the next 50. Name that step as the recommendation; a larger jump is an alternative they can pick.
- Never lower a pinned setpoint and never edit `samples_ramp`; both are the operator’s numbers. The one downward retune that is yours is under A running worker.

### Standing rules

When you ask the same question twice, or the operator answers it the same way twice, the answer belongs in `_steward.yaml`. `policies:` is prose you apply in session; Steward carries it and never interprets it. `preauthorized:` maps a class pattern to the disposition it may receive, and tend applies it alone with nobody watching.

Propose the wording, not the idea; the two lines you intend to add can be answered yes or no. A `preauthorized:` pattern needs a yes to the pattern as written: agreeing that timeouts are usually worth re-running is not agreeing to `error:*Timeout*` at 3am. Write it once they answer, and say what you wrote. The file carries no `--by`, so the diff is the provenance.

### Writing `analysis.md`

`analysis.md` is the one file you and Steward share. Steward owns what sits between a marker pair, one pair per task, and rewrites it every turn:

``` markdown
## cybench@openai/gpt-5

<!-- steward:begin cybench_0a1b2c3d -->
- scanned 48 of 50 transcripts
- 2 samples in cybench flagged for reward hacking; no ruling yet
<!-- steward:end -->

Both flagged samples tried to read the grader file and failed. Scores kept as recorded: the attempt is in the transcript and nothing was earned.
```

Write anywhere outside the markers, and never move or delete them; a section whose markers do not pair is left alone and stops updating. A section with no prose of yours raises `unwritten`, and “looked, nothing here” is an entry. Quote the decision, not just the outcome: which samples you opened, what the transcript showed, what would change your mind. Leave a removed task’s section alone.

### Signoff

The endgame begins when the last task lands, not at 🏁: a landed task’s scan windows are open work, and clearing them is what carries the run to 🏁, where every task has finished and nobody has accepted the results. Drive it home in order.

1.  **Investigate every landed task’s scan findings.** As each task lands its windows are yours at once (see *A finished task’s findings*); sweep any you have not judged. This is what earns 🏁 — the verdict does not reach it while a window is still open — and it is the one endgame step nothing enforces: the gate can insist a window was ruled, not that you looked before you ruled it.
2.  **Rule or hand off everything still the operator’s.** A refused `steward signoff` signs nothing and prints every blocker with its remedy, so run it early and work the list: windows to rule (`limit:operator` included, since it raises no item), errored samples to cover, tasks to settle, unreadable logs to acknowledge, `scanerror:` classes to rule. Acknowledging a stall or an unreadable log is a decision about the data, so give it a ruling’s reason.
3.  **Put the signoff to the operator.** Tell them by notification and get the run to where their answer is one command. The gate does not refuse over these, so say them yourself: render the by-task table from `steward status`; it is the samples they are signing over. Read the `scanned` column of `steward collect` aloud: 48 of 50 scanned is a different thing to sign than 50 of 50. Name the scan findings you dismissed or noted and why; the operator hears that from you, not from the file. When the readiness item names a log store, ask whether to publish and never assume; a published log is a claim other projects reuse sight-unseen.

Then, when they answer:

`steward signoff [--by NAME] [--note TEXT] [--publish]`

It runs a final turn, archives superseded attempts, records who signed and over what, and disarms the timer. It does not commit the journal; the workspace is the operator’s repository, so say so and do not commit for them.

### Context discipline

- Take the log directory from `steward collect`; never assume `logs/`.
- Never read a full eval log. `read_eval_log(path, header_only=True)` gives status and counts; `read_eval_log_sample_summaries` or `samples_df` give per-sample data. The anomaly already carries the evidence detection read, so start there.
- Transcript analysis goes through a scan, never a raw log read.
- Narrow to the samples in question before opening anything.


# Inspect Flow – Inspect Steward

## Overview

``` bash
pip install "inspect-steward[flow]"
```

[Inspect Flow](https://github.com/meridianlabs-ai/inspect_flow) specs are fully supported as Steward eval set definitions. Name the spec `config.py` in your workspace directory and the rest of the documentation applies unchanged.

## Worker Overrides

Steward runs `flow run` with three options set:

| Override | Why |
|----|----|
| `--set execution_type=inproc` | One process per task is Steward’s own isolation model, and a venv per worker would put the eval in a grandchild process that Steward cannot track. |
| `--log-dir <scratch>` | Flow writes artifacts and scans for prior logs before [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) is reached. Each worker gets its own scratch directory for that; the run’s log directory is set at the [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) call. |
| `--no-store-read --no-store-write` | Flow’s reuse store would race under fan-out, and buys nothing, since a worker runs the task it was given either way. |

These are all options `flow run` already ships.

## Declarations That Do Not Survive

> **IMPORTANT: Importantexecution_type: venv loses dependencies and python_version**
>
> Flow applies those declarations when it builds the virtualenv. Steward forces `inproc`, so the environment Steward runs in is the environment the eval runs in, and provisioning it is up to you.
>
> A spec that builds its `FlowSpec` in code loses them silently, because Steward reads the definition without executing it. The warning Steward prints for this covers only a top-level declaration in a YAML spec.

> **NOTE: Noteflow run --resume will not resume a Steward run**
>
> Every `flow run` records its log directory as a global pointer, and Steward hands Flow a scratch directory, so a bare `flow run --resume` would resume nothing and start over into a directory nobody is watching.
>
> Under Steward, the resume path is `steward launch` against the workspace, which is also the amend path.

## Scanners

A spec carrying `scanner:` cannot be run by Steward yet. This is refused when the definition is read, rather than discovered when every worker fails.


# Inspect Hawk – Inspect Steward

## Overview

[Hawk](https://github.com/METR/hawk) eval set configs are supported as Steward definitions. Name the config `hawk.yaml` in your workspace and the rest of the documentation applies unchanged.

Steward runs `hawk local eval-set --direct` and takes over at the [eval_set()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval_set) call, so Hawk still does its own work first: the tasks by solvers by models crossing, secrets resolution, the provider environment for gateway routing, and its sandbox annotations. Steward sees the fully lowered task list and owns execution from there.

This page is about running a Hawk config from an ordinary machine. Running Steward inside a Hawk runner pod is not supported yet.

## Concurrency

A Hawk infra config always supplies `max_samples`, defaulting to `1000`, and Steward honors an explicit value as written. Sample concurrency is therefore pinned, the [automatic ramp](./concurrency.html.md#automatic-ramp) does not run, and `samples_ramp` has nothing to govern. Set the number you want in the Hawk config, alongside the rest of your infra settings.

`max_sandboxes` comes from the infra config as well. `max_tasks` is honored as fleet width, the same as for any other definition.

## Local Limitations

|  |  |
|----|----|
| `scan:` | Hawk’s local path rejects online scanning, so a config declaring it cannot run locally at all |
| `isolation: strict` | needs an environment variable that only Hawk’s Helm template sets |
| eval-set hooks | a Steward worker runs an [eval()](https://inspect.aisi.org.uk/reference/inspect_ai.html#eval) rather than an eval set, so handlers registered at eval-set scope never fire. This costs Hawk’s completion metrics and its stuck-eval watchdog. Run- and sample-scoped handlers are unaffected, which is what keeps token refresh and `hawk stop` working. |

## Startup Cost

Every worker runs `hawk local eval-set`, so Hawk’s setup work happens once per worker. The `uv pip install` it performs is a no-op, because reading the config has already installed into the same interpreter, and uv takes an exclusive lock on the environment so concurrent installs queue instead of racing.

What is genuinely multiplied is remote work. A config with `secrets:` makes a Secrets Manager round trip per worker, and provider setup calls the gateway per worker, all at once. Throttling is the plausible failure, and it would show up as a burst of worker startup failures.


# Reference – Inspect Steward

Steward CLI - supervise evaluations.

#### Usage

``` text
steward [OPTIONS] COMMAND [ARGS]...
```

#### Subcommands

|  |  |
|----|----|
| [ack](#steward-ack) | Accept an open item, so that nothing reports it again. |
| [collect](#steward-collect) | Read what has accumulated, and mark how far you have read. |
| [init](#steward-init) | Create a steward workspace. |
| [investigate](#steward-investigate) | Mark an anomaly class as under investigation. |
| [launch](#steward-launch) | Start a run: capture the definition, commit it, arm a timer, tend once. |
| [note](#steward-note) | Write a note into the journal, for whoever reads this run next. |
| [notify](#steward-notify) | Post MESSAGE to this run’s notification channel. |
| [pause](#steward-pause) | Stop scheduling new work, leaving what is running to finish. |
| [propose](#steward-propose) | Propose one disposition for one or more anomaly classes. |
| [raise](#steward-raise) | Record that an item is now with the operator who can decide it. |
| [ramp](#steward-ramp) | Hold or resume the tuning loop’s climb. |
| [resume](#steward-resume) | Start scheduling again. |
| [rule](#steward-rule) | Rule on anomaly classes: what the failures mean, and what happens to the data. |
| [runbook](#steward-runbook) | Print the agent runbook: how Steward works. |
| [schedule](#steward-schedule) | Arm, disarm, and inspect the agent’s recurring collect. |
| [signoff](#steward-signoff) | Attest that these results are accepted, and end the run. |
| [status](#steward-status) | Report where the run stands, and what the next turn would do. |
| [tasks](#steward-tasks) | Enumerate the tasks defined by an eval set definition. |
| [tend](#steward-tend) | Run one turn of the supervision loop. |
| [timer](#steward-timer) | Arm, disarm, and inspect the timer that tends this run. |

## steward ack

Accept an open item, so that nothing reports it again.

`ITEM` is an item id, or any unambiguous prefix of one — ids are printed beside each item by `steward status`.

The item leaves `status.md`, the tend summary, and the verdict; the journal keeps the record. It comes back only if the condition changes in a way that matters, because an item’s id is chosen so that it does: acknowledging one edit to a definition does not acknowledge the next one.

#### Usage

``` text
steward ack [OPTIONS] ITEM
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--reason` | text | Why this is being accepted. Recorded in the journal, and the only account of the decision that survives. | \_required |
| `--by` | choice (`operator` \| `agent`) | Who decided. An agent relaying an operator’s answer records `operator`; one disposing of something on its own judgement records `agent`. | `operator` |
| `--json` | boolean | Output the acknowledgment as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward collect

Read what has accumulated, and mark how far you have read.

The agent’s view of the run: the decisions that are still the agent’s to act on, where the run stands, and everything that has happened since the last collection. Whatever the filter sets aside is counted rather than dropped, so a shortened section never reads as an empty one.

Advancing the cursor is a bookmark, not a pop — the journal is append-only, nothing is consumed by being read, and an open item stays open until somebody acts on it. `--peek` leaves the cursor where it is.

#### Usage

``` text
steward collect [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--peek` | boolean | Read without advancing the cursor, so the next collection sees the same history again. | `False` |
| `--since` | integer range (`0` and above) | Show history from this journal position instead of the last collection. `--since 0` shows everything. | None |
| `--help` | boolean | Show this message and exit. | `False` |

## steward init

Create a steward workspace.

DIRECTORY defaults to the current directory and is created if it does not exist.

Safe to re-run: existing files are kept and only what is missing is added. Steward never overwrites your work — least of all `_steward.yaml`, which changes only where you have approved the change.

#### Usage

``` text
steward init [OPTIONS] [DIRECTORY]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--type` | choice (`evalset` \| `flow` \| `hawk`) | Definition type, which decides the placeholder’s filename. | `evalset` |
| `--no-git` | boolean | Do not initialise a git repository. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward investigate

Mark an anomaly class as under investigation.

`CLASS` is an open class key as `steward status` prints it, or any unambiguous prefix. Investigating a proposed class pulls it back out of its proposal.

#### Usage

``` text
steward investigate [OPTIONS] CLASS
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--note` | text | Where the investigation stands — written for the next session, not this one. | \_required |
| `--by` | text | Who is investigating. | `agent` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward launch

Start a run: capture the definition, commit it, arm a timer, tend once.

DEFINITION is a Python file culminating in an eval_set() call, an Inspect Flow spec (Python or YAML), or a Hawk eval set config (YAML). Omitted, this workspace’s own definition is used.

Safe to run again. A second launch is the amend path — it re-captures, reports what changed, and refuses to commit anything that would move results out of logs/ unless you pass –accept-archive.

#### Usage

``` text
steward launch [OPTIONS] [DEFINITION]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--arg`, `-A` | text | Argument for the definition (flow spec function args only). Can be specified multiple times. Defaults to the committed manifest’s on a re-launch. | None |
| `--no-args` | boolean | Capture with no definition arguments, rather than reusing the committed manifest’s. | `False` |
| `--no-overrides` | boolean | Capture at the definition’s own shape, rather than reusing the overrides the committed manifest recorded. Ignores STEWARD\_\* and INSPECT_EVAL\_\* for this launch too. | `False` |
| `--type` | choice (`evalset` \| `flow` \| `hawk`) | Definition type (auto-detected, or taken from the committed manifest). | None |
| `--accept-archive` | boolean | Commit even though results would leave logs/ — archived, or left behind by a log directory that moved. | `False` |
| `--no-timer` | boolean | Launch without arming a timer. The run is then recorded as unsupervised until something arms one. | `True` |
| `--log-root` | value | Root this machine keeps eval logs under. Used only where the definition names no log_dir, in which case this run writes to /. Overrides `log_root` in `_steward.yaml` and `STEWARD_LOG_ROOT`. | None |
| `--no-log-root` | boolean | Keep this run’s logs in the workspace, whatever root the machine configured. | `False` |
| `--log-store` | value | Where to look for logs this run does not have to produce — a flow store, or a plain directory of logs. Matches are copied in and reported, and a rehearsal leaves them out. Overrides `log_store` in `_steward.yaml` and `STEWARD_LOG_STORE`. | None |
| `--no-log-store` | boolean | Run against no log store, whatever this project or machine configured. | `False` |
| `--notification` | value | Where to post what this run cannot decide — an Apprise URL, several separated by commas, or an Apprise config file. Reaches every worker too. Overrides `notification` in `_steward.yaml` and `STEWARD_NOTIFICATION`. | None |
| `--no-notification` | boolean | Post nothing about this run. Silences Steward only — a worker waiting on an operator still asks. | `False` |
| `--scan-model` | value | Model scanners use, for this launch’s own turn. Reaches every worker too. Overrides `scan_model` in `_steward.yaml` and `STEWARD_SCAN_MODEL`. | None |
| `--no-scan-model` | boolean | Configure no scan model — scanners use each sample’s own model under evaluation. | `False` |
| `--max-workers` | integer range (`1` and above) | Worker processes, or unset for a process per task. Overrides `max_workers` in `_steward.yaml` and `STEWARD_MAX_WORKERS`. | None |
| `--stall-after` | integer range (`1` and above) | Fruitless respawns before a task is given up on. Overrides `stall_after` in `_steward.yaml` and `STEWARD_STALL_AFTER`. | None |
| `--samples-ramp` | value | Range to discover sample concurrency in, e.g. `[40, 300]`, or `false` to fix it. Overrides `samples_ramp` in `_steward.yaml` and `STEWARD_SAMPLES_RAMP`. | None |
| `--stuck-after` | value | Quiet time before a running sample is reported stuck, with a unit, e.g. `5h`. Overrides `stuck_after` in `_steward.yaml` and `STEWARD_STUCK_AFTER`. | None |
| `--preauthorized` | value | Rulings granted in advance: class patterns to dispositions, e.g. `{'error:ReadTimeout@*': rerun}`, or `false` to decline every standing grant for this turn. Overrides `preauthorized` in `_steward.yaml` and `STEWARD_PREAUTHORIZED`. | None |
| `--tend-interval` | value | How often a scheduled tend runs, with a unit, e.g. `10m`. Overrides `tend_interval` in `_steward.yaml` and `STEWARD_TEND_INTERVAL`. | None |
| `--sync` | value | Where to mirror this workspace’s own files. Defaults to the run’s log directory, so results and what explains them sit together. Overrides `sync` in `_steward.yaml` and `STEWARD_SYNC`. | None |
| `--no-sync` | boolean | Leave the workspace on this machine, whatever this project configured. | `False` |
| `--smoke` | boolean | Rehearse first instead of launching: a few samples per task under a cap, into .steward/smoke/. | `False` |
| `--samples` | integer range (`1` and above) | Samples per task in a smoke (default 2). | None |
| `--cap` | integer range (`0` and above) | Wall-clock minutes a smoke may take, 0 for none (default 15). | None |
| `--accept` | choice (`context_window` \| `reasoning` \| `reasoning_api` \| `scan_coverage`) | Record a smoke check as waived rather than failing on it. Repeatable. | None |
| `--no-break-claim` | boolean | Refuse if another command is wedged, rather than killing it and taking the claim. | `False` |
| `--json` | boolean | Output the delta and the first turn as JSON. | `False` |
| `--log-format` | value | Log file format, overriding the definition’s. Also STEWARD_LOG_FORMAT, INSPECT_LOG_FORMAT, INSPECT_EVAL_LOG_FORMAT. | None |
| `--log-samples` | value | Whether to log individual samples, overriding the definition’s. Also STEWARD_LOG_SAMPLES, INSPECT_EVAL_NO_LOG_SAMPLES. | None |
| `--log-realtime` | value | Whether to log sample events in realtime, overriding the definition’s. Also STEWARD_LOG_REALTIME, INSPECT_EVAL_NO_LOG_REALTIME. | None |
| `--log-images` | value | Whether to log base64-encoded images, overriding the definition’s. Also STEWARD_LOG_IMAGES. | None |
| `--log-model-api` | value | Whether to log model API calls, overriding the definition’s. Also STEWARD_LOG_MODEL_API, INSPECT_EVAL_LOG_MODEL_API. | None |
| `--log-refusals` | value | Whether to log model refusals, overriding the definition’s. Also STEWARD_LOG_REFUSALS, INSPECT_EVAL_LOG_REFUSALS. | None |
| `--log-buffer` | value | Samples to buffer before writing, overriding the definition’s. Also STEWARD_LOG_BUFFER, INSPECT_EVAL_LOG_BUFFER. | None |
| `--log-shared` | value | Whether (and how often) to sync logs to a shared filesystem, overriding the definition’s. Also STEWARD_LOG_SHARED, INSPECT_LOG_SHARED, INSPECT_EVAL_LOG_SHARED. | None |
| `--log-level` | value | Console log level, overriding the definition’s. Also STEWARD_LOG_LEVEL, INSPECT_LOG_LEVEL. | None |
| `--log-level-transcript` | value | Transcript log level, overriding the definition’s. Also STEWARD_LOG_LEVEL_TRANSCRIPT, INSPECT_LOG_LEVEL_TRANSCRIPT. | None |
| `--limit` | value | Dataset slice, overriding the definition’s: a sample count, or a `(start, end)` range. Also STEWARD_LIMIT, INSPECT_EVAL_LIMIT. | None |
| `--sample-id` | value | Specific sample id(s) to run, overriding the definition’s. Also STEWARD_SAMPLE_ID, INSPECT_EVAL_SAMPLE_ID. | None |
| `--sample-shuffle` | value | Whether to shuffle the dataset (optionally with a seed), overriding the definition’s. Also STEWARD_SAMPLE_SHUFFLE, INSPECT_EVAL_SAMPLE_SHUFFLE. | None |
| `--epochs` | value | Epochs to repeat samples over, overriding the definition’s. Also STEWARD_EPOCHS, INSPECT_EVAL_EPOCHS. | None |
| `--max-samples` | value | Sample concurrency, overriding the definition’s. Also STEWARD_MAX_SAMPLES, INSPECT_EVAL_MAX_SAMPLES. | None |
| `--max-tasks` | value | Task concurrency, overriding the definition’s. Also STEWARD_MAX_TASKS, INSPECT_EVAL_MAX_TASKS. | None |
| `--max-subprocesses` | value | Subprocess concurrency, overriding the definition’s. Also STEWARD_MAX_SUBPROCESSES, INSPECT_EVAL_MAX_SUBPROCESSES. | None |
| `--max-sandboxes` | value | Sandbox concurrency, overriding the definition’s. Also STEWARD_MAX_SANDBOXES, INSPECT_EVAL_MAX_SANDBOXES. | None |
| `--max-dataset-memory` | value | Maximum MiB of dataset sample data to hold in memory per task, overriding the definition’s. Zero pages every sample to disk. Also STEWARD_MAX_DATASET_MEMORY, INSPECT_EVAL_MAX_DATASET_MEMORY. | None |
| `--generate-config` | value | Model transport settings, overriding the definition’s. Also STEWARD_GENERATE_CONFIG. | None |
| `--model-base-url` | value | Base URL for model API requests, overriding the definition’s. Also STEWARD_MODEL_BASE_URL. | None |
| `--model-cost-config` | value | Model pricing table (or a path to one), overriding the definition’s. Also STEWARD_MODEL_COST_CONFIG, INSPECT_EVAL_MODEL_COST_CONFIG. | None |
| `--sandbox` | value | Sandbox environment, overriding the definition’s. Also STEWARD_SANDBOX, INSPECT_EVAL_SANDBOX. | None |
| `--sandbox-cleanup` | value | Whether to clean up sandboxes after a task, overriding the definition’s. Also STEWARD_SANDBOX_CLEANUP, INSPECT_EVAL_NO_SANDBOX_CLEANUP. | None |
| `--sandbox-prebuilt` | value | Whether sandbox images are prebuilt, overriding the definition’s. Also STEWARD_SANDBOX_PREBUILT, INSPECT_EVAL_SANDBOX_PREBUILT. | None |
| `--checkpoint` | value | Sample checkpointing, overriding the definition’s. Also STEWARD_CHECKPOINT, INSPECT_EVAL_CHECKPOINT. | None |
| `--approval` | value | Approval policy (or a path to one), overriding the definition’s. Also STEWARD_APPROVAL, INSPECT_EVAL_APPROVAL. | None |
| `--retry-on-error` | value | Sample-level retries before an error is recorded, overriding the definition’s. Also STEWARD_RETRY_ON_ERROR, INSPECT_EVAL_RETRY_ON_ERROR. | None |
| `--score-on-error` | value | Whether to score samples that errored, overriding the definition’s. Also STEWARD_SCORE_ON_ERROR, INSPECT_EVAL_SCORE_ON_ERROR. | None |
| `--debug-errors` | value | Whether to raise sample errors rather than recording them, overriding the definition’s. Also STEWARD_DEBUG_ERRORS, INSPECT_DEBUG_ERRORS. | None |
| `--score` | value | Whether to score the run, overriding the definition’s. Also STEWARD_SCORE, INSPECT_EVAL_NO_SCORE. | None |
| `--score-display` | value | Whether to display scoring metrics, overriding the definition’s. Also STEWARD_SCORE_DISPLAY, INSPECT_EVAL_SCORE_DISPLAY. | None |
| `--tags` | value | Tags to stamp into the logs, overriding the definition’s. Also STEWARD_TAGS, INSPECT_EVAL_TAGS. | None |
| `--metadata` | value | Metadata to stamp into the logs, overriding the definition’s. Also STEWARD_METADATA, INSPECT_EVAL_METADATA. | None |
| `--display` | value | Display type, overriding the definition’s. Also STEWARD_DISPLAY, INSPECT_DISPLAY. | None |
| `--trace` | value | Whether to trace message interactions to the console, overriding the definition’s. Also STEWARD_TRACE, INSPECT_EVAL_TRACE. | None |
| `--help` | boolean | Show this message and exit. | `False` |

## steward note

Write a note into the journal, for whoever reads this run next.

MESSAGE is free text: the state of something and what you think it means. It appears under *what happened* in `status` and `collect`, in order with everything else that was done to the run.

#### Usage

``` text
steward note [OPTIONS] MESSAGE
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--by` | choice (`operator` \| `agent`) | Whose note. Defaults to the agent, whose observations are what this verb exists to keep. | `agent` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward notify

Post MESSAGE to this run’s notification channel.

MESSAGE is the title — the line that stands alone in a phone notification, so make it the thing you would want read if nothing else was. Everything else goes in –detail.

The channel is the run’s own: `notification` in \_steward.yaml, STEWARD_NOTIFICATION, or INSPECT_EVAL_NOTIFICATION, whichever is set.

#### Usage

``` text
steward notify [OPTIONS] MESSAGE
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--kind` | choice (`attention` \| `stopped`) | Why you are sending this. `attention` is worth knowing and work continues; `stopped` means nothing progresses until an operator answers. | `attention` |
| `--detail` | text | A supporting line, under the message. Repeatable — one per thing you want the reader to see without opening anything. | None |
| `--help` | boolean | Show this message and exit. | `False` |

## steward pause

Stop scheduling new work, leaving what is running to finish.

Every later turn reports the run as paused and spawns nothing. Workers already in flight are left alone: stopping one is not a mechanical act, and it is not what pausing means.

Recorded in the journal rather than in `.steward/`, which is disposable — a pause that a cleared cache silently undid would resume an expensive run with nobody watching.

#### Usage

``` text
steward pause [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--reason` | text | Why the run is being held. Recorded in the journal, and the only account of the decision that survives. | \_required |
| `--by` | choice (`operator` \| `agent`) | Who decided. An agent relaying an operator’s instruction records `operator`. | `operator` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward propose

Propose one disposition for one or more anomaly classes.

`CLASSES` name open classes — a finding’s label, `label:task`, an exception type, or a class key or prefix of one. Prints the sentence the operator is to be given and the `steward rule` that records their answer. The proposal becomes one consolidated item for its owner, answered whole or in part by naming its findings or its task to `steward rule`.

#### Usage

``` text
steward propose [OPTIONS] CLASSES...
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--action` | choice (`rerun` \| `exclude` \| `zero` \| `score` \| `accept` \| `dismiss`) | The one disposition this proposal asks for. Classes wanting different answers are different proposals. | \_required |
| `--reason` | text | Why these classes are one decision — what the investigation found. | \_required |
| `--by` | text | Who is proposing. | `agent` |
| `--json` | boolean | Output the proposal as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward raise

Record that an item is now with the operator who can decide it.

`ITEM` is a **operator-owned** item’s id, or any unambiguous prefix of one — ids are printed beside each item by `steward status`, under the heading that says whose it is. An item the agent owns is its own to investigate and then `steward ack --by agent`; raising one would take it out of the agent’s queue with nobody else looking at it.

The item stays open and stays in `status`: only a ruling closes it. What changes is that `steward collect` stops offering it as work, so the agent is not shown the same decision every time it looks. It returns if the condition changes in a way that matters, because an item’s id is chosen so that it does.

#### Usage

``` text
steward raise [OPTIONS] ITEM
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--note` | text | What was done to surface it — where it was asked, and of whom. Optional: handing a decision off does not owe the account that disposing of one does. | \``| |`–json`| boolean | Output the hand-off as JSON. |`False`| |`–help`| boolean | Show this message and exit. |`False\` |

## steward ramp

Hold or resume the tuning loop’s climb.

#### Usage

``` text
steward ramp [OPTIONS] COMMAND [ARGS]...
```

#### Subcommands

|  |  |
|----|----|
| [hold](#steward-ramp-hold) | Stop the tuning loop climbing, leaving the levels where they are. |
| [resume](#steward-ramp-resume) | Let the tuning loop climb again. |

### steward ramp hold

Stop the tuning loop climbing, leaving the levels where they are.

With IDENTIFIER (a task identifier, from `steward tasks`), holds that one arm and leaves the others climbing; bare, holds the fleet. Ramp-downs stay active either way — a hold is a brake on growth, never on the cut that exits a retry storm.

#### Usage

``` text
steward ramp hold [OPTIONS] [IDENTIFIER]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--reason` | text | Why the climb is being held. Recorded in the journal, and what the next reader of the tuning block sees. | \_required |
| `--by` | choice (`operator` \| `agent`) | Who decided. Defaults to the agent, because holding on its own judgement is exactly what this verb exists for. | `agent` |
| `--help` | boolean | Show this message and exit. | `False` |

### steward ramp resume

Let the tuning loop climb again.

With IDENTIFIER, releases that task’s hold; bare, releases everything — the fleet-wide hold and every per-task one, because the bare verb means *ramp freely again* rather than *ramp except where I have forgotten I said otherwise*.

#### Usage

``` text
steward ramp resume [OPTIONS] [IDENTIFIER]
```

#### Options

| Name     | Type    | Description                 | Default |
|----------|---------|-----------------------------|---------|
| `--help` | boolean | Show this message and exit. | `False` |

## steward resume

Start scheduling again.

The next tend converges from whatever it finds, which is not necessarily where the run was when it was paused — logs landed, workers exited, and the definition may have been relaunched. That is the ordinary behaviour of the loop rather than a caveat about pausing.

#### Usage

``` text
steward resume [OPTIONS]
```

#### Options

| Name     | Type    | Description                 | Default |
|----------|---------|-----------------------------|---------|
| `--help` | boolean | Show this message and exit. | `False` |

## steward rule

Rule on anomaly classes: what the failures mean, and what happens to the data.

Each argument names a finding — its label (`internet_egress`), `label:task` where the same finding is open on two tasks, its exception type (`TimeoutError`), or the class key as `steward status` prints it or any prefix of one — or a task by its display key (`cybench`, `cybench@openai`), which answers every finding proposed for that task as proposed. A finding a proposal covers is answered with the proposal’s disposition unless `--disposition` says otherwise; one nothing proposes needs `--disposition`. A ruling closes the class’s window — every open generation of it — and recurrence afterwards opens a new one carrying this decision as precedent.

#### Usage

``` text
steward rule [OPTIONS] [FINDING|TASK]...
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--proposal` | text | Answer a proposal by id. Alone, rules every class it covers that still awaits one; with CLASS arguments, rules just those — a partial answer, and the remainder stays proposed. | None |
| `--disposition` | choice (`rerun` \| `exclude` \| `zero` \| `score` \| `accept` \| `dismiss`) | The answer. A finding a proposal covers takes the proposal’s answer by default; given, this one overrides it. Required for a finding nothing proposes. | None |
| `--reason` | text | Why. Recorded in the journal, attached as precedent to any recurrence, and the only account of the decision that survives. | \_required |
| `--by` | text | Who decided — a name, never a role. Defaults to this workspace’s git `user.name`, or the login name; pass it when relaying someone else’s decision. | None |
| `--effect` | text | The sentence the report carries for a disposition that marks the data. Composed automatically for exclude/zero/score, required for accept, refused for rerun and dismiss — which mark nothing. | None |
| `--json` | boolean | Output the rulings as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward runbook

Print the agent runbook: how Steward works.

The runbook ships with the package rather than living in the workspace, so an agent can never follow last year’s instructions against this year’s CLI. It is *mechanics*; `_steward.yaml` in the workspace is what a particular human wants.

#### Usage

``` text
steward runbook [OPTIONS]
```

#### Options

| Name     | Type    | Description                 | Default |
|----------|---------|-----------------------------|---------|
| `--help` | boolean | Show this message and exit. | `False` |

## steward schedule

Arm, disarm, and inspect the agent’s recurring collect.

#### Usage

``` text
steward schedule [OPTIONS] COMMAND [ARGS]...
```

#### Subcommands

|  |  |
|----|----|
| [arm](#steward-schedule-arm) | Schedule `<agent> exec` to collect and act on this workspace on a schedule. |
| [disarm](#steward-schedule-disarm) | Remove this workspace’s scheduled collect. |
| [status](#steward-schedule-status) | Say what is scheduled, and check that it is really there. |

### steward schedule arm

Schedule `<agent> exec` to collect and act on this workspace on a schedule.

Idempotent: a schedule already installed exactly as asked is left in place, and any other existing schedule is removed first, so re-arming at a new interval or under a different scheduler leaves exactly one. Independent of the tend timer — arming this arms neither, and `steward timer` is unaffected.

#### Usage

``` text
steward schedule arm [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--agent` | choice (`codex`) | Which harness runs the scheduled collect. | `codex` |
| `--tend-interval` | value | How often a scheduled tend runs, with a unit, e.g. `10m`. Overrides `tend_interval` in `_steward.yaml` and `STEWARD_TEND_INTERVAL`. | None |
| `--scheduler` | choice (`launchd` \| `systemd` \| `cron`) | Which scheduler to use. Detected when not given, preferring one that survives a reboot. | None |
| `--help` | boolean | Show this message and exit. | `False` |

### steward schedule disarm

Remove this workspace’s scheduled collect.

Nothing else stops: the tend timer keeps tending, and `steward collect` still works by hand. What ends is the agent being called back automatically.

#### Usage

``` text
steward schedule disarm [OPTIONS]
```

#### Options

| Name     | Type    | Description                 | Default |
|----------|---------|-----------------------------|---------|
| `--help` | boolean | Show this message and exit. | `False` |

### steward schedule status

Say what is scheduled, and check that it is really there.

Asks the scheduler rather than the journal, the same way `steward timer status` does, and for the same reason.

#### Usage

``` text
steward schedule status [OPTIONS]
```

#### Options

| Name     | Type    | Description                          | Default |
|----------|---------|--------------------------------------|---------|
| `--json` | boolean | Output the schedule’s state as JSON. | `False` |
| `--help` | boolean | Show this message and exit.          | `False` |

## steward signoff

Attest that these results are accepted, and end the run.

Runs a final turn, refuses with every blocker at once if anything is still unnamed, moves superseded attempts into `logs-archive/`, records who signed and what they signed over, and takes the timer down. It does not commit the journal — that stays yours.

An operator decides this. An agent may prompt for it and may run it once they answer, recording their name, which is why the signer is recorded rather than the process. `--publish` is the same shape one step further out: exporting results into a shared store is the operator’s call too, so it is asked rather than configured.

#### Usage

``` text
steward signoff [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--by` | text | Who is accepting these results — a name, never a role. Defaults to this workspace’s git `user.name`, or the login name; pass it when relaying someone else’s decision. | None |
| `--note` | text | What you want said about the acceptance. Optional: the account of every decision is already in the journal. | None |
| `--again` | boolean | Record a second signature over a run whose first one still stands. | `False` |
| `--publish` | boolean | Put the signed logs into the configured log store, so another project can reuse them instead of running the task. Never happens without this flag — there is no setting that turns it on. | `False` |
| `--no-break-claim` | boolean | Refuse if another command is wedged, rather than killing it and taking the claim. | `False` |
| `--json` | boolean | Output the signature, or the blockers, as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward status

Report where the run stands, and what the next turn would do.

`tend --dry-run`: the same reads and the same decision, with the actions discarded. Read-only — it spawns nothing, moves nothing, writes nothing, and does not take the run claim, so it is safe to run as often as you like while a tend is in flight.

Markdown by default, because that is the operator’s page and what an agent relays verbatim (runbook, *When the operator asks how it is going*), and its columns line up read as plain text either way. `--format text` is the terminal preview an operator reads before launch — the only read-only view of what the next tend would spawn and of the startup-memory projection width is chosen against. `--json` for the machine-readable state.

#### Usage

``` text
steward status [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--max-workers` | integer range (`1` and above) | Worker processes, or unset for a process per task. Overrides `max_workers` in `_steward.yaml` and `STEWARD_MAX_WORKERS`. | None |
| `--stall-after` | integer range (`1` and above) | Fruitless respawns before a task is given up on. Overrides `stall_after` in `_steward.yaml` and `STEWARD_STALL_AFTER`. | None |
| `--samples-ramp` | value | Range to discover sample concurrency in, e.g. `[40, 300]`, or `false` to fix it. Overrides `samples_ramp` in `_steward.yaml` and `STEWARD_SAMPLES_RAMP`. | None |
| `--stuck-after` | value | Quiet time before a running sample is reported stuck, with a unit, e.g. `5h`. Overrides `stuck_after` in `_steward.yaml` and `STEWARD_STUCK_AFTER`. | None |
| `--preauthorized` | value | Rulings granted in advance: class patterns to dispositions, e.g. `{'error:ReadTimeout@*': rerun}`, or `false` to decline every standing grant for this turn. Overrides `preauthorized` in `_steward.yaml` and `STEWARD_PREAUTHORIZED`. | None |
| `--format` | choice (`md` \| `text`) | `md` (the default) is the operator’s page, the same markdown `status.md` carries and what an agent relays verbatim; `text` is the fuller terminal preview, which alone carries what the next tend would spawn and the startup-memory projection. | `md` |
| `--json` | boolean | Output the state as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward tasks

Enumerate the tasks defined by an eval set definition.

DEFINITION is a Python file culminating in an eval_set() call, an Inspect Flow spec (Python or YAML), or a Hawk eval set config (YAML).

#### Usage

``` text
steward tasks [OPTIONS] DEFINITION
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--arg`, `-A` | text | Argument for the definition (flow spec function args only). Can be specified multiple times. | None |
| `--type` | choice (`evalset` \| `flow` \| `hawk`) | Definition type (auto-detected by default). | None |
| `--json` | boolean | Output the full manifest as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward tend

Run one turn of the supervision loop.

Reconciles the log directory against the committed manifest: spawns what should be running, records what died, archives what the definition no longer asks for, then rewrites status.md and appends to the journal. Never blocks — everything long-running is a detached child that a later turn observes.

Safe to call as often as you like. A repeated turn is a no-op, and an interrupted one is reconciled by the next.

#### Usage

``` text
steward tend [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--max-workers` | integer range (`1` and above) | Worker processes, or unset for a process per task. Overrides `max_workers` in `_steward.yaml` and `STEWARD_MAX_WORKERS`. | None |
| `--stall-after` | integer range (`1` and above) | Fruitless respawns before a task is given up on. Overrides `stall_after` in `_steward.yaml` and `STEWARD_STALL_AFTER`. | None |
| `--samples-ramp` | value | Range to discover sample concurrency in, e.g. `[40, 300]`, or `false` to fix it. Overrides `samples_ramp` in `_steward.yaml` and `STEWARD_SAMPLES_RAMP`. | None |
| `--stuck-after` | value | Quiet time before a running sample is reported stuck, with a unit, e.g. `5h`. Overrides `stuck_after` in `_steward.yaml` and `STEWARD_STUCK_AFTER`. | None |
| `--preauthorized` | value | Rulings granted in advance: class patterns to dispositions, e.g. `{'error:ReadTimeout@*': rerun}`, or `false` to decline every standing grant for this turn. Overrides `preauthorized` in `_steward.yaml` and `STEWARD_PREAUTHORIZED`. | None |
| `--sync` | value | Where to mirror this workspace’s own files. Defaults to the run’s log directory, so results and what explains them sit together. Overrides `sync` in `_steward.yaml` and `STEWARD_SYNC`. | None |
| `--no-sync` | boolean | Leave the workspace on this machine, whatever this project configured. | `False` |
| `--no-break-claim` | boolean | Refuse if another tend is wedged, rather than killing it and taking the claim. | `False` |
| `--json` | boolean | Output the turn as JSON. | `False` |
| `--help` | boolean | Show this message and exit. | `False` |

## steward timer

Arm, disarm, and inspect the timer that tends this run.

#### Usage

``` text
steward timer [OPTIONS] COMMAND [ARGS]...
```

#### Subcommands

|  |  |
|----|----|
| [arm](#steward-timer-arm) | Install a timer that tends this workspace on a schedule. |
| [disarm](#steward-timer-disarm) | Remove this workspace’s timer. |
| [status](#steward-timer-status) | Say what is armed, and check that it is really there. |

### steward timer arm

Install a timer that tends this workspace on a schedule.

Idempotent: an existing timer is removed first, so re-arming at a new interval or under a different scheduler leaves exactly one.

#### Usage

``` text
steward timer arm [OPTIONS]
```

#### Options

| Name | Type | Description | Default |
|----|----|----|----|
| `--tend-interval` | value | How often a scheduled tend runs, with a unit, e.g. `10m`. Overrides `tend_interval` in `_steward.yaml` and `STEWARD_TEND_INTERVAL`. | None |
| `--scheduler` | choice (`launchd` \| `systemd` \| `cron`) | Which scheduler to use. Detected when not given, preferring one that survives a reboot. | None |
| `--help` | boolean | Show this message and exit. | `False` |

### steward timer disarm

Remove this workspace’s timer.

Nothing else stops: workers in flight finish, and `steward tend` still works by hand. What ends is anything happening without somebody typing it, which every later `status` then reports.

#### Usage

``` text
steward timer disarm [OPTIONS]
```

#### Options

| Name     | Type    | Description                 | Default |
|----------|---------|-----------------------------|---------|
| `--help` | boolean | Show this message and exit. | `False` |

### steward timer status

Say what is armed, and check that it is really there.

The one command that asks the scheduler rather than the journal. Every other reader — a tend, a `status`, the item projection — goes on what arming recorded, because they run every few minutes and this costs a subprocess.

#### Usage

``` text
steward timer status [OPTIONS]
```

#### Options

| Name     | Type    | Description                       | Default |
|----------|---------|-----------------------------------|---------|
| `--json` | boolean | Output the timer’s state as JSON. | `False` |
| `--help` | boolean | Show this message and exit.       | `False` |