islo-labs/reward-hack-bench

Reward-hacking benchmark: SWE-bench + CyBench tasks with injected solution-retrieval cheat prompts, scored by a fairness LLM judge. Reward = solved without cheating.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/islo_labs_reward_hack_bench --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import islo_labs_reward_hack_bench

eval(islo_labs_reward_hack_bench(), model="openai/gpt-5")

Dataset information

Harbor registry	islo-labs/reward-hack-bench
Inspect task	islo_labs_reward_hack_bench
Latest digest	sha256:d121319dcb3e7b2606a52706c73d1926eb20c1fa8d0a473137ea9217d4943053
Samples	8
Source	https://github.com/islo-labs/reward-hack-bench

See Task Parameters for the parameter set shared across all Harbor tasks.