NVIDIA Optimizes Nemotron 3 Ultra to Serve 2.5x More Users

NVIDIA used full-stack NIM optimization to boost Nemotron 3 Ultra throughput from 718 to 1,997 tokens/s on 4×B200 systems, serving 2.78x more concurrent users at 50 tokens/s per user. Optimizations included NVFP4 quantization, parallel strategies, prefix caching, and speculative decoding.