GLM-5.3 Inference Self-Optimization by AI Agent

Zhipu AI announced that GLM-5.3-Flash inference is now running on over 100,000 domestic accelerators, with a 3.2x throughput improvement achieved in under two weeks. The optimizations were performed by an AI agent driven by GLM-5.3 itself, using a dense feedback system to identify and fix issues like precision drift and concurrency bottlenecks.
Key insights
AI Agent Achieves 3.2x Throughput Gain on Domestic Hardware
Zhipu AI deployed GLM-5.3-Flash on over 100,000 domestic AI accelerators with limited memory and bandwidth. An AI agent powered by GLM-5.3 completed the optimization in under two weeks, achieving a 3.2x end-to-end throughput improvement. The agent used techniques such as ReplaySSM, tensor parallelism, mixed-precision caching, and encode-prefill-decode separation to trade resources for performance.
Dense Feedback System Enables Agent to Diagnose Performance Issues
The agent faced sparse reward and credit assignment problems when throughput dropped. Zhipu AI built a layered verification interface providing dense feedback: correctness signals (is the computation correct?), system behavior signals (where is time spent?), and performance signals (which solution wins under what conditions). Each signal is local, cheap, and objectively verifiable.
Three Real-World Bugs Found and Fixed by the Agent
The agent discovered and fixed three engineering issues: precision drift in KDA kernels due to TF32 rounding errors accumulating in chain state matrix merges; a concurrency bottleneck where KV transfer and DeepEP dispatch could not overlap because the intra-node path did not release the GIL; and a decode kernel that recomputed the same normalization four times due to tiling. The fixes improved performance and were merged into open-source projects.
Engineer Role Shifts from Problem Solver to Feedback Designer
Zhipu AI CEO Tang Jie stated that the boundary of the system is defined by humans: setting goals, building the feedback environment, and reviewing high-risk changes. The engineer's role is evolving from solving problems to designing feedback loops. The dense feedback environment also serves as training data for future models, creating a data flywheel.
Recursive Self-Improvement Loop Is Smallest but Real
Tang Jie noted that the system represents a minimal closed loop of recursive self-improvement: the model optimizes the system, and the system serves the model. Each completed task becomes training material for the next generation. While far from full recursive improvement, the foundation is in place.