Artificial Analysis Releases Decision Index v2 With Private Benchmarks and Calibration Metric

Artificial Analysis released the second version of its Decision Index, raising private datasets to 45% of the index and introducing a Decision Score that rewards both correctness and calibrated confidence. Drex Max v1.1 from Nace AI and GLiDE v1.2 from Fastino Labs both score 73, with Mammoth 27B from Base Labs at 72, while Claude Haiku 5.5 (Max) scores 77 and GPT-6 Luna (Max) scores 76 at substantially higher cost and latency.
Event version 1.
Key insights
Private data share rises to 45%
Decision Index v2 builds 45% of the index on private datasets, up from 25% in v1. The new AA-Paperclip benchmark covers 1,500 documents including resumes, receipts, invoices, and AI and machine learning research papers, with models scored correct only if all 15 answers per document are right.
Decision Score rewards calibrated confidence
The new headline metric scores each question from 0 to 100, with right answers above 50 and wrong answers below 50, and confidence in the top answer moving the score within each half. AA-Symmetry tests 120 questions with reordered options, swapped labels, and negated yes/no variations to measure probability consistency.
Top decision models cluster within one point
Drex Max v1.1 from Nace AI and GLiDE v1.2 from Fastino Labs both score 73, and Mammoth 27B from Base Labs scores 72. Celeris-1.1 Decision, Decider v1.2, and Drex Max v1.1 sit on the Decision Index versus Cost Pareto frontier.
Cost per 1M tasks varies more than 60x
Celeris-1.1 Decision costs $14 per 1M tasks, Decider v1.2 costs $68, Drex Max v1.1 costs $102, and Clef costs $859. GLiDE v1.2 costs $780, nearly 8x higher than Drex Max v1.1, because it bills a document's input for every question rather than once.
Language models score higher at higher cost and latency
Claude Haiku 5.5 (Max) scores 77 and GPT-6 Luna (Max) scores 76 on Decision Index v2, about 3 points ahead of Drex Max v1.1. GPT-6 Luna (Max) costs $1,530 per 1M tasks versus $102 for Drex Max v1.1 and takes 11.34 seconds per task versus 275 milliseconds.