arcprize/arc-agi-2

Reasoning
Multimodal

ARC-AGI-2: visual reasoning tasks testing general fluid intelligence — humans solve them easily but state-of-the-art models still struggle.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/arcprize_arc_agi_2 --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import arcprize_arc_agi_2

eval(arcprize_arc_agi_2(), model="openai/gpt-5")

Dataset information

Harbor registry arcprize/arc-agi-2
Inspect task arcprize_arc_agi_2
Latest digest sha256:eb5b06cd04808ec3b88a6fe065b81f28b6d5c9deff35b895f1f8931b49eb7809
Samples 167
Paper arxiv
Source https://github.com/arcprize/ARC-AGI-2

See Task Parameters for the parameter set shared across all Harbor tasks.