NVIDIA Spatial-IQ Benchmark for 3D Object Counting in Multimodal Models

NVIDIA Research introduced Spatial‑IQ, a diagnostic benchmark for 3D object counting in multimodal models. Humans achieve 82.1% accuracy on images with hidden blocks; the best off‑the‑shelf model manages only 17.7%. Spatial‑IQ decomposes the task into nine perceptual and cognitive sub‑tasks, scoring each separately. After training Qwen2.5‑VL‑32B on these sub‑tasks, its object‑counting accuracy rose from 2.9% to 62.6%.
Key insights
Spatial‑IQ exposes a critical gap in multimodal AI’s spatial reasoning
Even the strongest multimodal model scored 17.7% on the benchmark, far below human performance. The nine sub‑tasks isolate failures from simple column‑counting to inferring occluded support blocks, providing a precise instrument to diagnose and fix spatial understanding.