long-horizon-terminal-bench/lhtb

Coding

Assistants

Long-Horizon Terminal-Bench: 46-task benchmark measuring how well LLM agents sustain useful work in a containerized terminal over hundreds of steps. Hidden rebuild-from-artifact verifiers; self-reported progress does not count.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/long_horizon_terminal_bench --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import long_horizon_terminal_bench

eval(long_horizon_terminal_bench(), model="openai/gpt-5")

Dataset information

Harbor registry	long-horizon-terminal-bench/lhtb
Inspect task	`long_horizon_terminal_bench`
Latest digest	sha256:5ad11c23718b34c647c756c6a565fbff234982bbb95feda4fb45e8479d8e845d
Samples	46
Paper	arxiv
Source	https://github.com/zli12321/LHTB

See Task Parameters for the parameter set shared across all Harbor tasks.