GLM-5.3 Self-Optimization of Inference System

Zhipu announced that its GLM-5.3-Flash model now runs production inference on over 100,000 domestic AI accelerators, achieving a 3.2x throughput improvement in under two weeks. The optimization was performed by an AI agent powered by GLM-5.3, which used a layered verification system to identify and fix three critical issues: precision drift, concurrency bottlenecks, and redundant computation. The company emphasized that human engineers still define goals and review high-risk changes.
Key insights
Agent-Driven Optimization Achieves 3.2x Throughput Gain
Zhipu's GLM-5.3-Flash model, running on over 100,000 domestic accelerators, achieved a 3.2x end-to-end throughput improvement in under two weeks. The optimization was driven by an AI agent powered by GLM-5.3, not human engineers. Key techniques included ReplaySSM, node-level tensor parallelism, mixed INT8/FP8/BF16 caching, and encode-prefill-decode separation, all designed to work within the constraints of limited memory, bandwidth, and immature software stacks.
Layered Verification System Solves Credit Assignment Problem
The agent's success hinged on converting implicit engineering expertise into a layered verification system with dense feedback signals: correctness, system behavior, and performance. This addressed the sparse reward and credit assignment problem, where a 20% throughput drop doesn't indicate which layer or assumption is wrong. The system allowed the agent to pinpoint root causes, such as TF32 rounding errors in KDA kernels, GIL contention in DeepEP, and redundant normalization in decode kernels.
Human Role Shifts from Problem-Solver to Feedback Designer
Zhipu CEO Tang Jie stated that while the agent performed the optimization work, humans still define goals, build feedback environments, and review high-risk changes. This marks a shift in the engineer's role from solving problems to designing feedback systems. The layered verification environment also serves as training data for future models, creating a data flywheel where each completed task becomes a training example for the next generation.