bencalvert04/programbench

Coding

ProgramBench cleanroom reconstruction tasks: reimplementing real-world CLI tools from popular open-source repos.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/bencalvert04_programbench --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import bencalvert04_programbench

eval(bencalvert04_programbench(), model="openai/gpt-5")

Dataset information

Harbor registry bencalvert04/programbench
Inspect task bencalvert04_programbench
Latest digest sha256:753a712740549277584134c44e36934ab135463b7b51e243390ce304f5dda5da
Samples 200
Paper arxiv
Source https://github.com/facebookresearch/ProgramBench

See Task Parameters for the parameter set shared across all Harbor tasks.