Android Bench 2.0 Benchmark for Real-World Android Development by Google

Google introduced Android Bench 2.0, a benchmark measuring model+agent combinations on real Android development tasks using production apps like Signal and Bitwarden. The 30-task suite includes app conversion, architecture migration, and new feature development. Top scores reached 28%, with GPT-6 Astra+Codex and Claude Fable 5.1+Claude Code leading, while many XML-to-Compose migrations saw zero passes.
Key insights
Benchmark Design and Task Categories
Android Bench 2.0 evaluates model+agent pairs on 30 tasks derived from Google's Android best practices, using real production apps. Categories include app conversion (Flutter/React Native to native Android), building apps from scratch, architecture migration (XML to Compose, Hilt to Koin, etc.), and new feature development (CameraX, Media3, picture-in-picture). Architecture migration is the largest category with 13 tasks.
Results: Top Models Achieve 28% Pass Rate, Many Tasks Unconquered
GPT-6 Astra+Codex and Claude Fable 5.1+Claude Code tied for first with 28% pass rate. Qwen3.8 Max+Qwen Coder and Kimi K3+Kimi Code followed. Google's own Gemini models placed lower but with cost advantages. Several tasks saw zero passes from all models, including Flutter-to-Android conversion and most XML-to-Compose migrations, indicating these remain challenging for current AI.