XNews

GLM-5.3-Flash Inference Optimization by Zhipu AI

GLM-5.3-Flash Inference Optimization by Zhipu AI

Zhipu AI announced that its GLM-5.3-Flash model, optimized by an AI agent driven by GLM-5.3, achieved 3.2x throughput improvement in under two weeks and now runs on over 100,000 domestic accelerators. The agent used a dense feedback framework to diagnose and fix three engineering issues, including precision drift and concurrency bottlenecks.

Key insights

Self-Optimizing Agent Framework

Zhipu AI disclosed that the optimization of GLM-5.3-Flash was performed by an AI agent powered by GLM-5.3 itself, not by human engineers. The agent employed a 'dense feedback' system with three signal types: correctness, system behavior, and performance. This allowed it to identify and fix issues such as TF32 rounding error accumulation in KDA kernels and GIL contention in KV transfer. The approach represents a shift from engineers solving problems to designing feedback environments.

Engineering Trade-offs and Constraints

The optimization was conducted under severe constraints: limited memory and bandwidth on domestic accelerators, immature software stack, and support for 1M token context and multimodal requests. All improvements involved trade-offs: ReplaySSM (compute for memory), node-internal tensor parallelism (communication for memory), mixed INT8/FP8/BF16 cache (precision for capacity), and encode-prefill-decode separation (flexibility for scheduling). The final per-token cost matched mainstream NVIDIA GPUs.

Three Real Engineering Cases Solved

The agent discovered and fixed three real problems: precision drift in KDA context parallel paths due to TF32 rounding error accumulation; KV transfer and DeepEP dispatch not overlapping due to GIL not being released; and a decode kernel computing the same normalization four times due to block partitioning. Each case represents a different capability: numerical correctness attribution, cross-language debugging, and pattern migration across codebases.

Source posts