GLM-5.3-Flash inference optimized by its own AI agent

Z.ai (Zhipu) announced that GLM-5.3-Flash's production inference on over 100,000 domestic accelerators was optimized by an AI agent powered by GLM-5.3 itself. The agent achieved a 3.2x end-to-end throughput improvement in under two weeks, fixing issues like precision drift and kernel inefficiencies through a dense feedback system.
Key insights
3.2x Throughput Improvement in Two Weeks
Z.ai (Zhipu) reported that GLM-5.3-Flash's production inference system, running on over 100,000 domestic AI accelerators, achieved a 3.2x end-to-end throughput improvement. The optimization was completed in less than two weeks from first run to full production. The system supports 1 million token context and multimodal requests.
AI Agent Driven by GLM-5.3 Performed the Optimization
The optimization work was carried out by an AI agent powered by GLM-5.3 itself. Zhipu CEO Tang Jie noted that the model helped optimize the system running it. The agent used a dense feedback system with three signal types: correctness (is the computation correct?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal was local, cheap, and objectively verifiable.
Specific Fix: Precision Drift in KDA Kernel
The agent discovered that TF32 rounding errors accumulated in chain-like state matrix merging during context-parallel paths, causing precision drift that worsened with sequence length. The fix was merged into the open-source project Flash Linear Attention.
Specific Fix: KV Transfer and DeepEP Dispatch Overlap
The agent found that KV cache transfer and DeepEP dispatch were not overlapping as intended. By tracing the call chain across Python/C++ boundaries, it identified that the intra-node path in DeepEP did not release the Global Interpreter Lock (GIL), preventing the transfer thread from executing. After fix, overhead dropped from over 30% to under 1%.
Specific Fix: Decode Kernel Repeated Normalization
A decode kernel was computing the same normalization four times due to tiling. The agent reorganized the computation, achieving a 1.71x speedup. The optimization idea came from reading kernel code from SGLang, FLA, and DeepGEMM, distilling 'optimization skeletons'.
Engineering Constraints and Trade-offs
The deployment environment had severe constraints: limited memory and interconnect bandwidth on domestic accelerators, immature software stack with missing kernels and documentation. All optimizations involved trade-offs: ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (separation for scheduling flexibility).
Toward Recursive Self-Improvement
Tang Jie emphasized that the dense feedback environment built for the agent is also valuable training data for future models. Each completed task becomes a training example for successors. He stated that while recursive self-improvement is still far off, the smallest closed loop already exists: model optimizes system, system serves model. The role of engineers shifts from problem-solver to feedback designer.