swe-bench/swe-smith
Coding
SWE-smith: NeurIPS 2025 toolkit for synthesizing unlimited SWE-bench-style task instances from any Python repository, plus released task instances and agent trajectories.
Run this task
CLI:
inspect eval inspect_harbor/swe_bench_swe_smith --model openai/gpt-5Python:
from inspect_ai import eval
from inspect_harbor import swe_bench_swe_smith
eval(swe_bench_swe_smith(), model="openai/gpt-5")Dataset information
| Harbor registry | swe-bench/swe-smith |
| Inspect task | swe_bench_swe_smith |
| Latest digest | sha256:e1ffe3cc3517754681749eda196485f51b38b4552340a48b12f4b6953b06aa86 |
| Samples | 100 |
| Paper | arxiv |
| Source | https://github.com/SWE-bench/SWE-smith |
See Task Parameters for the parameter set shared across all Harbor tasks.