NVIDIA NIM Optimizations Boost Nemotron 3 Ultra Throughput 2.5x on B200

NVIDIA used NIM microservices to optimize the Nemotron 3 Ultra 550B MoE model on 4×B200, increasing system throughput from 718 to 1,997 tokens per second. Under a 50 token/s per user SLO, this allows serving 2.78x more concurrent users. Optimizations include NVFP4 quantization, prefix caching, speculative decoding, and expert-aware execution.