swe-rebench/swe-rebench-leaderboard
Coding
SWE-rebench leaderboard: 860 curated Python SWE tasks from Nebius AI R&D. All instances have pre-built Docker images. Continuously updated monthly splits.
Run this task
CLI:
inspect eval inspect_harbor/swe_rebench_leaderboard --model openai/gpt-5Python:
from inspect_ai import eval
from inspect_harbor import swe_rebench_leaderboard
eval(swe_rebench_leaderboard(), model="openai/gpt-5")Dataset information
| Harbor registry | swe-rebench/swe-rebench-leaderboard |
| Inspect task | swe_rebench_leaderboard |
| Latest digest | sha256:ebe7444e313a0d8db94fa541139826eaebe2b0abcd4900c6f73e750494910dca |
| Samples | 860 |
| Paper | arxiv |
| Source | https://huggingface.co/datasets/nebius/SWE-rebench-leaderboard |
See Task Parameters for the parameter set shared across all Harbor tasks.