deveval/deveval

Coding

DevEval: manually-annotated code-generation samples from real-world Python repositories, aligned to practical software development.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/deveval --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import deveval

eval(deveval(), model="openai/gpt-5")

Dataset information

Harbor registry deveval/deveval
Inspect task deveval
Latest digest sha256:362b00a719d161f95bbd9a8186ec612d07fcd441fa617fb367f9f9142ecd3c61
Samples 63
Paper arxiv
Source https://github.com/seketeam/DevEval

See Task Parameters for the parameter set shared across all Harbor tasks.