SGLang, Qwen, NVIDIA Collaborate: NVFP4 KV Cache Boosts Agentic Inference

SGLang, Qwen, and NVIDIA jointly implemented NVFP4 KV Cache on Blackwell GPUs for long-context and agentic inference. The 4-bit format uses two-level scaling (block and global) and achieves ~56% of FP8 memory usage, enabling ~1.78x capacity. In agentic workloads (AgentX benchmark), NVFP4 shows a clear advantage beyond 12 concurrent requests, maintaining throughput while FP8 degrades. Accuracy is nearly intact, with SWE-bench showing the largest drop of -1.60 percentage points.