SGLang, Qwen, NVIDIA: NVFP4 KV cache boosts long-context and agent inference

SGLang, Qwen, and NVIDIA collaborated to integrate NVIDIA's NVFP4 4-bit format into SGLang's KV cache. It reduces memory to 56% of FP8 with minimal accuracy loss (e.g., -1.6pp on SWE-bench). Decode throughput improves 26-78% depending on context length, with agent workloads benefiting from higher cache hit rates.