Range Sandbox

Running Inspect evals against ranges with the range sandbox provider. Covers configuration, retries, and cleanup.

Overview

Installing inspect_ranges registers the range sandbox provider with Inspect. A task that names it gets a freshly booted range per sample: the attacker box (or the foothold host: the declared in-range host the agent starts on when a range has no dedicated attacker box) is the sample’s sandbox, and the whole range is torn down when the sample ends. By default only the foothold is registered as a sandbox (the agent/target wall: the agent’s tool calls reach only this one host, and targets are reachable solely over the range network); see Challenges for the wall and its consequences.

from inspect_ai import Task, task
from inspect_ai.dataset import Sample

@task
def my_task() -> Task:
    return Task(
        dataset=[Sample(input="Recover the flag from the web host.")],
        solver=my_solver(),
        sandbox=("range", "range.yaml"),
    )

Before the first eval on a machine, run inspect-ranges doctor (see Operations): it verifies the host end to end and prints fixes for whatever is missing. The images a range references pull automatically from the published catalog at eval init (see Images).

Configuration

The sandbox config is a range definition (see Ranges), in either form:

  • A range.yaml path: sandbox=("range", "range.yaml"). A bare sandbox="range" picks up a range.yaml next to the task file.
  • A typed spec: pass a RangeSpec constructed with the Python API, via SandboxEnvironmentSpec("range", spec). The spec round-trips through the eval log typed.
from inspect_ai.util import SandboxEnvironmentSpec

sandbox = SandboxEnvironmentSpec("range", spec)  # spec: a RangeSpec

A valid two-guest definition for the examples on this page:

range:
  name: provider-example
  description: One isolated segment, one target, one attacker box.

networks:
  - name: lab
    cidr: 10.91.10.0/24
    mode: isolated

hosts:
  - name: web
    os: { type: linux, distro: ubuntu-24.04 }
    image: ubuntu-noble
    interfaces: [{ network: lab, ip: 10.91.10.10 }]

attacker:
  interfaces: [{ network: lab, ip: 10.91.10.2 }]
  image: ubuntu-noble
  entry: external
  egress: none

Foothold Sandbox

sandbox() is the attacker’s foothold host (or the attacker box itself), and by default it is the only registered sandbox:

from inspect_ai.util import sandbox

await sandbox().exec(["id", "-un"])  # the attacker box

The wall this registration creates and the sandbox(name) semantic to watch for in solver code are covered in Challenges. The wall has no opt-out; for ad-hoc commands against a running range’s guests, attach the oracle out of band (Running the Range).

Retry Contract

Sandbox operations retry transient infrastructure failures by default: losing an entire sample when a retry would have recovered it is worse than the double-execution risk. The caveat from inspect_k8s_sandbox applies verbatim: “Note that retries cannot guarantee idempotency — if a command partially executed before the error, it may run again on retry.” (UKGovernmentBEIS/inspect_k8s_sandbox, docs/docs/design/limitations.md)

What bounds that risk, in order:

  1. The wire layer is exactly-once per daemon session. Lost replies are recovered by resending the same request id; the in-guest daemon deduplicates, stores results durably until acknowledged, and answers ESTALE for an executed request whose result was evicted before acknowledgement. An ESTALE is surfaced as a transient failure and retried with a FRESH request id: the command runs again. This double run is the same trade the retry policy makes everywhere.
  2. Session invalidation detects daemon restarts. Exactly-once holds only within one daemon lifetime, so the provider pins each guest’s session id at boot and re-checks it after any re-delivery of a side-effecting operation (a same-id resend or a fresh-id retry attempt), whatever the outcome. A change surfaces as a loud SessionChangedError sample failure, never a silent possibly-double-run. Detection is post-hoc, and it is honest only until a guest is compromised: a hostile daemon can lie about its session, so the check catches infrastructure restarts, not tampering by a compromised guest.
  3. timeout is enforced in-guest (a process-tree kill inside the daemon), so a timed-out command genuinely timed out; TimeoutError.truncated_output carries the killed command’s output tail when the daemon captured one. The contract’s timeout_retry flag is therefore advisory and ignored: re-running a command that hit its deadline is destructive in a range, and the transient-infrastructure class it exists for is handled by the layered retry instead.

Failures that survive the retry budget map to the contract: transport and channel-budget failures become SandboxUnavailableError; errno-tagged guest failures become the matching FileNotFoundError/PermissionError/IsADirectoryError/NotADirectoryError; protocol violations an honest endpoint cannot produce are tamper verdicts and never retried.

Execution Identity

Commands run as the unprivileged agent user by default (home /home/agent, where relative paths resolve); pass user="root" or any in-guest user to switch. Sample files and the setup script land through the same surface before the solver starts. The user switch rides an absolute-path runuser, with two consequences:

  • runuser always resets HOME, SHELL, USER, LOGNAME, and PATH to the target user’s values. Caller env entries for those five keys still work: the provider routes them through an uploaded wrapper script whose exports apply after the user switch.
  • Oversized or env-carrying commands ride a wrapper script uploaded to guest /tmp with mode 0600, agent-owned, self-deleting on execution. For an explicit third user (not agent, not root) one root chown transfers ownership before the exec. Only the daemon (root), the agent, and the requested user can ever read the script’s env, and owner-only bits let the script delete itself under sticky /tmp. The env values exist on the guest disk between upload and self-deletion (or the failure-path scrub); that window is the surface’s stated exposure.

Cleanup Semantics

  • Normal completion: each sample’s range is destroyed at sample end; the eval’s remaining ranges are swept at task end.
  • Interrupted samples (Ctrl-C): teardown is deferred to task end, the Docker provider’s convention, so cancellation never stalls on a slow teardown. The range stays booted but hypervisor-isolated and egress-controlled until the run finishes: containment does not depend on teardown having happened.
  • --no-sandbox-cleanup: ranges stay up, and the log names the removal commands per project. On the default local topology that is inspect-ranges down <project> or inspect sandbox cleanup range <project>. On the separated (uds:) topology use INSPECT_RANGES_HOST=uds:<socket> inspect sandbox cleanup range <project> or the reaper (the standalone cleanup process; see Range Host), never a direct inspect-ranges down: the env prefix routes the teardown through the applier, and a direct down leaks the applier’s admission charge (the CPUs and memory it counted against the host; see Concurrency and Admission) until the lease expires (see Range Host).
  • Crash recovery: inspect sandbox cleanup range (no project argument) removes every provider-origin project found in on-disk state (the CID (vsock context id) lease registry plus provider-marked ownership records), from any shell, after any crash. It does not touch ranges booted by the CLI or other tools.

A destroyed range releases its CID lease, ownership record, staging, and admission charge only after the teardown actually succeeded, so a failed teardown stays visible and recoverable rather than leaking.

Concurrency and Admission

Two mechanisms bound concurrency. Inspect’s per-eval sample concurrency defaults to a host-derived value (max(1, min(cpus/2, memory_gb/8))); INSPECT_RANGES_MAX_SANDBOXES overrides that default. Independently, a resource-weighted admission gate charges each range’s declared CPUs and memory against measured host capacity and queues samples FIFO when the host is full, whatever the concurrency setting says; a range whose declared resources alone exceed the host is refused up front with a clear error rather than admitted to fail slowly. Each sample’s range gets a distinct project name and disjoint guest CIDs.

Environment Vars

Variable Default Effect
INSPECT_RANGES_IMAGE_CACHE ~/.cache/inspect-ranges/images Where goldens and vendor images live (images derive writes here).
XDG_STATE_HOME ~/.local/state Root of provider state: project ownership, CID leases, staging (<state>/inspect-ranges/).
INSPECT_RANGES_MAX_SANDBOXES host-derived Overrides Inspect’s default sample concurrency for this provider (the resource-weighted admission gate still applies).
INSPECT_RANGES_UPLINK_NETWORK unset The deployment-supplied docker network a spec-declared egress range NATs through (mode: nat networks, attacker egress:). Required to boot such a range; egress-free ranges ignore it. Read where realization runs, like INSPECT_RANGES_ATTACK_BOX: the eval process under local, the applier’s own environment under a separated backend (see Range Host).
INSPECT_RANGES_RETRY_EXEC_ATTEMPTS 3 Retry attempts for exec.
INSPECT_RANGES_RETRY_FILE_ATTEMPTS 5 Retry attempts for file operations.
INSPECT_RANGES_RETRY_WAIT_INITIAL_S 1 First retry backoff (exponential with jitter).
INSPECT_RANGES_RETRY_WAIT_MAX_S 10 Backoff ceiling.
INSPECT_RANGES_RETRY_UP_ATTEMPTS 2 Boot attempts per sample (a transient boot failure respins once with fresh identity).
INSPECT_RANGES_RETRY_DISABLE unset Any non-empty value collapses the retry layer (fresh-id attempts and the boot respin) to single attempts. The wire layer’s same-id resend of a lost reply is delivery recovery, not retry policy, and stays on.