Agents and Solvers

Solvers are the execution components in Inspect AI. They can run agent scaffolds (like ReAct), execute solution scripts (like the Oracle solver), perform prompt engineering, and more. Both solvers and agents can be used to solve Harbor tasks.

Default Agent Scaffold

When no agent or solver is specified, Inspect Harbor provides a default agent scaffold for your model:

• Agent Type: ReAct agent.
• Tools: bash(timeout=300), python(timeout=300), update_plan().
• Compaction: CompactionEdit() for context window management.

This default configuration is suitable for most Harbor tasks that require command execution and file manipulation.

Using Custom Agents

You can provide your own agent or solver implementation using the --solver flag.

Using a custom agent:

inspect eval inspect_harbor/terminal_bench \
  --solver path/to/custom/agent.py@custom_agent \
  --model openai/gpt-5

Using the Inspect SWE agent framework:

First install the required package:

pip install inspect-swe

CLI:

inspect eval inspect_harbor/terminal_bench_sample \
  --solver inspect_swe/claude_code \
  --model anthropic/claude-sonnet-4-5

Python API:

from inspect_ai import eval
from inspect_harbor import terminal_bench_sample
from inspect_swe import claude_code

eval(
    terminal_bench_sample(),
    solver=claude_code(),
    model="anthropic/claude-sonnet-4-5"
)

For more details:

• Agents documentation
• Solvers documentation
• Inspect SWE documentation

Oracle Solver

The oracle() solver is useful for verifying that a dataset is correctly configured and solvable. It executes the task’s reference solution (solution/solve.sh script) instead of using a model.

CLI:

inspect eval inspect_harbor/hello_world \
  --solver inspect_harbor/oracle

Python API:

from inspect_ai import eval
from inspect_harbor import hello_world, oracle

eval(hello_world(), solver=oracle())