android-bench/android-bench

Coding

A dataset of 100 Android software engineering tasks derived from real-world repositories.

← Back to Registry

Run this task

CLI:

inspect eval inspect_harbor/android_bench --model openai/gpt-5

Python:

from inspect_ai import eval
from inspect_harbor import android_bench

eval(android_bench(), model="openai/gpt-5")

Dataset information

Harbor registry android-bench/android-bench
Inspect task android_bench
Latest digest sha256:e9c64ac5470aa2323be3f8391a831c4cb7392317d2e921c08422b46851067dd5
Samples 100
Source https://github.com/android-bench/android-bench

See Task Parameters for the parameter set shared across all Harbor tasks.