XNews

SGLang, Qwen, NVIDIA collaborate on NVFP4 KV cache for long contexts

SGLang, Qwen, NVIDIA collaborate on NVFP4 KV cache for long contexts

SGLang, Qwen, and NVIDIA introduced NVFP4 KV cache quantization, achieving 1.78x capacity over FP8 with minimal accuracy loss. In agentic workloads, NVFP4 maintains throughput at high concurrency where FP8 degrades. SWE-bench showed a 1.6 pp drop, the main accuracy concern.

Source posts