SGLang, Qwen, NVIDIA collaborate on NVFP4 KV Cache for long-context inference

NVFP4 4-bit quantization with two-level scaling reduces KV cache memory by ~1.78x vs FP8, with minimal accuracy loss. In agentic workloads, it doubles cache hit rate and improves throughput by up to 78% at 1M context. The technique is most beneficial for long-context and agentic scenarios.