terminal-bench/terminal-bench-3

Coding
Assistants

Terminal-Bench 3.0 is the third benchmark for measuring agents’ abilities to complete tasks using a terminal.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/terminal_bench_3 --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import terminal_bench_3

eval(terminal_bench_3(), model="openai/gpt-5")

Dataset information

Harbor registry terminal-bench/terminal-bench-3
Inspect task terminal_bench_3
Latest digest sha256:88433fbcecd1a3f81f7a71bff4cc76c394d0edbefb7e028f90d4109b639fefba
Samples 75
Paper arxiv
Source https://github.com/harbor-framework/terminal-bench-3

See Task Parameters for the parameter set shared across all Harbor tasks.