XNews

NVIDIA NIM boosts Nemotron 3 Ultra throughput 2.5x on B200 systems

NVIDIA NIM boosts Nemotron 3 Ultra throughput 2.5x on B200 systems

NVIDIA optimized Nemotron 3 Ultra on 4×B200 systems using NIM microservices, achieving 2.5x throughput improvement from 718 to 1,997 tok/s. Under a 50 tok/s per user SLO, the same hardware serves 2.78x concurrent users. Optimizations include NVFP4 quantization, prefix caching, speculative decoding, and coordinated tuning.

Source posts