NVIDIA Optimizes Nemotron 3 Ultra with NIM, Serving 2.78x More Users

NVIDIA used full-stack NIM optimizations on a 4×B200 system to increase Nemotron 3 Ultra throughput from 718 to 1,997 tok/s, enabling 2.78x more concurrent users at 50 tok/s per user. Optimizations included NVFP4 quantization, prefix caching, and speculative decoding.