GLM-5.3-Flash Inference System Self-Optimized by GLM Agent

Zhipu AI revealed that a GLM-5.3-powered agent optimized its inference system across 100k+ domestic accelerators, achieving 3.2x throughput in under two weeks. The agent used dense feedback to fix precision, concurrency, and computation errors.
Key insights
AI Agent Achieves 3.2x Throughput in Under Two Weeks
Zhipu AI announced that all production inference for GLM-5.3-Flash now runs on over 100,000 domestic AI accelerators. The optimization was performed by an AI agent driven by GLM-5.3, not human engineers, achieving a 3.2x end-to-end throughput improvement from first run to full production in less than two weeks. This milestone demonstrates that AI can autonomously optimize complex infrastructure under tight constraints.
Engineering Constraints and Trade-offs
Deployment faced harsh conditions: limited memory and interconnect bandwidth on domestic accelerators, 1M token context, multimodal requests, and an immature software stack lacking kernels and documentation. Every optimization was a trade-off: ReplaySSM traded compute for memory, tensor parallelism traded communication for memory, mixed INT8/FP8/BF16 caching traded precision for capacity, and encode-prefill-decode separation traded separation for scheduling flexibility.
Diagnosis Bottleneck and Dense Feedback Solution
The core difficulty was not writing code but diagnosing why performance degraded. 'Throughput dropped 20%' is a sparse reward with credit assignment problems. End-to-end benchmarks take hours, making exploration costly. Zhipu solved this by externalizing senior engineers' implicit process rewards into a dense feedback system with three signals: correctness, system behavior, and performance. Each signal is local, cheap, and objectively verifiable, effectively designing an environment for the agent rather than adding capabilities.
Three Concrete Engineering Bugs Found and Fixed
The agent discovered and resolved three real issues: 1) KDA context parallelism precision drift due to TF32 rounding errors accumulating in chained state matrix merging; 2) KV transfer and DeepEP dispatch failing to overlap because the in-node path did not release the GIL, causing over 30% overhead reduced to under 1%; 3) A decode kernel recomputing the same normalization four times, leading to a 1.71x speedup. The agent derived optimization skeletons from reading SGLang, FLA, and DeepGEMM kernel code, showcasing pattern transfer across codebases.
Human Role Shifts to Feedback Design
Zhipu CEO Tang Jie emphasized that humans still define objectives, build the feedback environment, and review high-risk changes. However, the engineer's role is evolving from problem-solver to feedback designer. This is a fundamental shift: instead of directly fixing issues, engineers create the conditions for AI agents to identify and solve problems autonomously, redefining what human expertise means in AI-driven infrastructure.
Feedback Environment Doubles as Training Data
Tang Jie noted that the layered, verifiable feedback environment serves dual purposes: it is the agent's working environment and a source of training data for next-generation models. Every completed task becomes a training example for successors, creating a closed loop. While recursive self-improvement remains distant, the minimal loop already exists: 'the model optimizes the system, the system runs the model.' This hints at a data flywheel that could accelerate future AI development.