GLM-5.3-Flash Optimization by Zhipu AI Agent

Zhipu AI announced that its GLM-5.3-Flash inference system running on over 100,000 domestic AI accelerators was optimized by a GLM-5.3-powered agent. The agent achieved 3.2x end-to-end throughput improvement in about two weeks by implementing a dense feedback environment that translates expert intuition into verifiable interfaces.
Key insights
Engineering constraints and optimization trade-offs
The deployment environment was challenging: limited domestic accelerator memory and interconnect bandwidth, 1M token context, multimodal requests, and immature software stack. All optimizations therefore involved trade-offs: ReplaySSM (compute for memory), node-inner tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (separation for scheduling freedom).
Diagnosis bottleneck: sparse rewards and credit assignment
The key difficulty was not writing code but understanding why performance degraded. A '20% throughput drop' is a sparse reward and credit assignment problem—it signals something is wrong but not which layer or hypothesis to test. End-to-end benchmarks taking hours make exploration costly. Experienced engineers rely on an implicit 'process reward' to know when to check timelines or micro-benchmarks.
Dense feedback environment: three signal types
Zhipu AI made this implicit knowledge explicit via a dense feedback environment with three signal types: correctness (is the computation right), system behavior (where is time spent), and performance (which solution wins under which conditions). Each signal must be local, cheap, and objectively verifiable. This effectively designs an environment for the agent rather than patching its capabilities.
Case study 1: KDA precision drift from TF32 rounding errors
The agent discovered that KDA kernel accuracy drifted with sequence length in the context parallel path. TF32 rounding errors accumulated through chained state matrix merging, causing precision degradation on longer contexts. The fix was merged into the open-source Flash Linear Attention project.
Case study 2: KV transfer and DeepEP dispatch non-overlap due to GIL
The KV cache transfer should have overlapped with computation, but the agent found they never overlapped. Tracing the call chain across Python/C++ boundaries, it discovered that DeepEP's node-inner path did not release the Global Interpreter Lock (GIL), preventing the transfer thread from executing. Fixing this reduced overhead from over 30% to under 1%.
Case study 3: Decode kernel duplicate normalization
The agent found a decode kernel that computed the same normalization four times due to its block-splitting method. It reorganized the computation structure, achieving a 1.71x speedup. The optimization idea came from reading kernel codes of SGLang, FLA, and DeepGEMM and distilling 'optimization skeletons'.
Engineer role shift and recursive self-improvement insight
Zhipu CEO Tang Jie emphasized that the boundary remains with humans: setting goals, building the feedback environment, and vetting risky changes. But the engineer's role is shifting from problem-solver to feedback-designer. The deeper implication is that this layered verifiable feedback environment serves dual purpose: it is both the agent's workspace and training data for next-gen models. Tang concluded: 'We are far from recursive self-improvement, but the smallest loop already exists. Models optimize systems, systems serve models.'