SGLang integrates NVIDIA NVFP4 KV cache quantization, boosting agent throughput by up to 78%

SGLang, Qwen, and NVIDIA collaborated to implement NVFP4 4-bit KV cache quantization in SGLang, reducing memory to 56% of FP8. In agentic workloads, it increased decode throughput by up to 78% at 1M context and improved radix cache hit rates, with minimal accuracy loss except 1.6 pp drop on SWE-bench.