openai/swe-lancer-diamond-all
Coding
SWE-Lancer Diamond (full): public split of OpenAI’s SWE-Lancer benchmark — real Upwork freelance software-engineering tasks worth $500,800, combining IC engineering tasks and managerial decision tasks.
Run this task
CLI:
inspect eval inspect_harbor/openai_swe_lancer_diamond_all --model openai/gpt-5Python:
from inspect_ai import eval
from inspect_harbor import openai_swe_lancer_diamond_all
eval(openai_swe_lancer_diamond_all(), model="openai/gpt-5")Dataset information
| Harbor registry | openai/swe-lancer-diamond-all |
| Inspect task | openai_swe_lancer_diamond_all |
| Latest digest | sha256:ef4cbde34973eb4aebf63c4fc3d7ec386e71c177192302ffa1971d39855ce222 |
| Samples | 463 |
| Paper | arxiv |
| Source | https://github.com/openai/SWELancer-Benchmark |
See Task Parameters for the parameter set shared across all Harbor tasks.