XNews

GLM-5.3 Optimizes Its Own Inference System

GLM-5.3 Optimizes Its Own Inference System

Zhipu AI announced that its GLM-5.3 model drove the optimization of its own inference system, GLM-5.3-Flash, now running on over 100,000 domestic AI accelerators. The model, acting as an AI agent, improved end-to-end throughput by 3.2x in under two weeks. Key optimizations included ReplaySSM, tensor parallelism, mixed-precision caching, and architectural separation, all achieved under constraints of limited memory, bandwidth, and immature software stacks.

Key insights

Model-Driven Optimization Achieves Significant Gains

Zhipu AI revealed that GLM-5.3, functioning as an AI agent, optimized its own inference system, GLM-5.3-Flash, now deployed on over 100,000 domestic accelerators. The process took under two weeks from first run to full production, achieving a 3.2x improvement in end-to-end throughput. This was accomplished under challenging conditions including limited accelerator memory, restricted interconnect bandwidth, and an immature software stack, with optimizations like ReplaySSM and mixed-precision caching trading compute, communication, and precision for capacity and flexibility.

Dense Feedback Mechanism Key to Agent Problem-Solving

Zhipu AI CEO Tang Jie stated that the primary challenge was not coding but diagnosing performance issues, described as a sparse reward and credit assignment problem. To address this, they built a layered verification interface providing dense feedback signals: correctness, system behavior, and performance. These signals, which are local, cheap, and objectively verifiable, enabled the agent to identify and fix three real engineering issues: precision drift in KDA kernels, a GIL bottleneck in DeepEP, and redundant computation in a decode kernel, leading to a 1.71x speedup in one case.

Engineer Role Shifts to Designing Feedback Environments

Tang Jie emphasized that while the agent performed the optimizations, humans still define goals, build feedback environments, and review high-risk changes. This shifts the engineer's role from solving problems to designing feedback systems. He also noted a deeper implication: the layered, verifiable feedback environment serves dual purposes as both the agent's working environment and training data for future models, creating a data flywheel. Tang concluded, "We are far from recursive self-improvement, but the smallest closed loop already exists: models optimize systems, systems run models."

Source posts