XNews

NVIDIA boosts Nemotron 3 Ultra throughput 2.78x with full-stack NIM optimization

NVIDIA boosts Nemotron 3 Ultra throughput 2.78x with full-stack NIM optimization

NVIDIA documented a full-stack optimization on 4×B200 systems that increased system throughput from 718 to 1,997 tok/s, serving 2.78x more users at 50 TPS each, using NVFP4 quantization, speculative decoding, and prefix caching.

Source posts