GLM-5.3 agent optimizes its own inference stack for 3.2x throughput

Zhipu AI revealed that its GLM-5.3-powered Infra Agent optimized the inference stack of GLM-5.3-Flash on over 100,000 domestic AI accelerators, achieving a 3.2x throughput improvement within two weeks. The agent discovered and fixed three real engineering issues, including precision drift and concurrency bottlenecks.
Key insights
Self-Optimization Agent
The GLM-5.3-driven Infra Agent autonomously optimized the production inference stack for GLM-5.3-Flash. Deployed on over 100,000 domestic accelerators, the end-to-end throughput increased 3.2x from first run to production in under two weeks. Zhipu CEO Tang Jie emphasized that the agent did most of the optimization work.
Challenging Deployment Conditions
Domestic accelerators have limited memory and bandwidth; the software stack is immature with missing kernels and poor documentation. GLM-5.3-Flash required 1M context and multimodal support. Every optimization involved trade-offs: compute for memory (ReplaySSM), communication for memory (tensor parallelism), precision for capacity (mixed INT8/FP8/BF16).
Key Bottleneck: Understanding Why Performance Degrades
The agent often got stuck not because it couldn't write code, but because it didn't know why performance regressed. A '20% throughput drop' is a sparse reward problem with credit assignment. The solution was building a layered verification interface that provides dense feedback: correctness, system behavior, and performance signals.
Three Concrete Fixes Discovered by the Agent
1) KDA kernel precision drift: TF32 rounding errors accumulated through sequential state matrix merging at long contexts. 2) KV transfer and DeepEP dispatch not overlapping: agent traced call chain across Python/C++ and found GIL not released. 3) Decode kernel recomputed normalization four times due to block partitioning. Agent restructured computation, accelerating 1.71x.
Open Source and Knowledge Transfer
The precision drift fix was merged into Flash Linear Attention, an open-source project. The agent's optimization skeletons were derived from reading kernels in SGLang, FLA, and DeepGEMM. This demonstrates cross-codebase pattern migration.
Shift in Engineer Role
Tang Jie noted that engineers' role changes from problem-solvers to feedback-designers. Humans define goals, build feedback environments, and review high-risk changes, while agents execute optimization loops. The dense feedback environment also serves as training data for the next generation of models.
Toward Recursive Self-Improvement
The paper 'Toward Recursive Self-Improvement' describes this as a minimal closed loop: the model optimizes the system, and the system serves the model. While far from full recursive self-improvement, this shows AI can iteratively improve its own infrastructure.