NVIDIA NIM optimizations yield 2.78× more users for Nemotron 3 Ultra on B200

NVIDIA used NIM 2.0 full‑stack optimizations to improve Nemotron 3 Ultra throughput from 718 to 1,997 tok/s on a 4×B200 system, enabling 2.78× concurrent users at 50 tok/s per user. Key optimizations: NVFP4 quantization, automatic kernel tuning, prefix/Mamba state caching, and multi‑token prediction speculative decoding for agentic workloads with long input, short output.