gpqa-diamond/gpqa-diamond

GPQA Diamond: expert-validated graduate-level multiple-choice questions in biology, physics, and chemistry, designed to be Google-proof for non-experts.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/gpqa_diamond --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import gpqa_diamond

eval(gpqa_diamond(), model="openai/gpt-5")

Dataset information

Harbor registry	gpqa-diamond/gpqa-diamond
Inspect task	gpqa_diamond
Latest digest	sha256:87d299c2cd19daac11ff3e8f3dfb61f614cad21a8f93f22e188f4b11282dfcf1
Samples	198
Paper	arxiv
Source	https://github.com/idavidrein/gpqa

See Task Parameters for the parameter set shared across all Harbor tasks.