NVFP4 KV cache: 1.78x capacity over FP8 for long-context AI inference

SGLang, Qwen, and NVIDIA collaborated to implement NVFP4 KV cache on Blackwell GPUs, achieving 1.78x capacity over FP8 with minimal accuracy loss (e.g., -1.6 pp on SWE-bench). In agentic workloads, it enables higher concurrency and cache hit rates, with decode throughput gains up to 78% at 1M context.