XNews

Zhipu GLM-5.3-Flash Inference Optimization by AI Agent

Zhipu GLM-5.3-Flash Inference Optimization by AI Agent

Zhipu AI announced that its GLM-5.3-Flash model now runs on over 100,000 domestic AI accelerators, with inference throughput improved 3.2x in under two weeks. The optimization was performed by an AI agent driven by GLM-5.3 itself, using techniques like ReplaySSM, tensor parallelism, and mixed-precision caching.

Key insights

Agent-Driven Optimization Achieves 3.2x Throughput Gain

Zhipu AI disclosed that GLM-5.3-Flash's production inference is now deployed on over 100,000 domestic AI accelerators. The end-to-end throughput increased 3.2x from first run to full production in less than two weeks. All major optimizations were carried out by an AI agent powered by GLM-5.3, not by human engineers. The deployment conditions were challenging: limited accelerator memory and bandwidth, immature software stack, and support for 1M token context and multimodal requests.

Dense Feedback System Replaces Sparse Rewards for Agent Debugging

The key innovation was converting implicit engineering experience into a layered verification interface called 'dense feedback'. Three signal types were defined: correctness (is the computation right?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal must be local, cheap, and objectively verifiable. This design gives the agent an environment to explore rather than just adding capabilities.

Three Real-World Bugs Found and Fixed by the Agent

The agent discovered and fixed three distinct engineering issues. First, TF32 rounding errors in KDA context-parallel paths accumulated through chain state matrix merging, causing precision drift with longer sequences. Second, KV cache transfer and DeepEP dispatch could not overlap because the intra-node path did not release the GIL, blocking the transmission thread. Third, a decode kernel recomputed the same normalization four times due to block partitioning; the agent restructured it for 1.71x speedup using optimization skeletons distilled from SGLang, FLA, and DeepGEMM.

Engineer Role Shifts from Problem-Solver to Feedback Designer

Zhipu CEO Tang Jie emphasized that the bottleneck is not code writing but understanding why performance degrades. The agent's sparse reward problem (e.g., 'throughput dropped 20%' without location) mirrors reinforcement learning's credit assignment challenge. The solution is to have humans define goals, build feedback environments, and review high-risk changes. Engineers now design the feedback system rather than directly solving each optimization.

Recursive Self-Improvement Loop: Model Optimizes System, System Serves Model

Tang Jie noted that the layered verification environment serves a dual purpose: it is both the agent's working environment and training data for the next generation of models. Each completed task becomes training material for successors. He stated, 'We are far from recursive self-improvement, but the smallest closed loop already exists: the model optimizes the system, and the system runs the model.'

Source posts