kumo/kumo-parity
Reasoning
KUMO (parity split): subset of the KUMO procedural-reasoning benchmark used for parity / regression checks against the upstream evaluation.
Run this task
CLI:
inspect eval inspect_harbor/kumo_parity --model openai/gpt-5Python:
from inspect_ai import eval
from inspect_harbor import kumo_parity
eval(kumo_parity(), model="openai/gpt-5")Dataset information
| Harbor registry | kumo/kumo-parity |
| Inspect task | kumo_parity |
| Latest digest | sha256:a2d88f8ff28aba88e91492ad6b0fa51b13f3f4d668ac7dcf05d497c8dc9d6a25 |
| Samples | 212 |
| Paper | arxiv |
| Source | https://github.com/linhaowei1/kumo |
See Task Parameters for the parameter set shared across all Harbor tasks.