XNews

Zhipu GLM-5.3-Flash Optimizes Its Own Inference System

Zhipu GLM-5.3-Flash Optimizes Its Own Inference System

Zhipu AI announced that GLM-5.3-Flash's inference system was optimized by a GLM-5.3-driven agent, achieving 3.2x throughput in two weeks on over 100,000 domestic accelerators. The agent discovered and fixed three real engineering issues using a dense feedback environment.

Key insights

GLM-5.3 Agent Optimized Its Own Inference Infrastructure

Zhipu AI disclosed that the production inference for GLM-5.3-Flash now runs on over 100,000 domestic AI accelerators. The optimization from first run to full production took less than two weeks, with end-to-end throughput improved 3.2x. The work was performed by an AI agent powered by GLM-5.3, not human engineers. CEO Tang Jie noted that the model helped optimize the system running itself.

Optimizations Were Trade-offs Under Constrained Hardware

The deployment environment had limited memory and bandwidth on domestic accelerators, immature software stacks, and required support for 1M token context and multimodal requests. Optimizations included ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (flexibility for scheduling). These trade-offs enabled single-token cost parity with mainstream NVIDIA GPUs.

Dense Feedback Environment Enabled Agent to Find Root Causes

The key insight was that the agent got stuck not because it couldn't write code, but because it didn't know why performance degraded. Zhipu built a layered verification interface providing dense feedback: correctness (is the computation right?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal was local, cheap, and objectively verifiable. This turned the agent's implicit process reward into explicit signals.

Three Real Engineering Issues Discovered and Fixed by the Agent

The agent identified and resolved three issues: (1) Precision drift in KDA context-parallel paths due to TF32 rounding errors accumulating in chain state matrix merges, fixed in Flash Linear Attention. (2) KV transfer and DeepEP dispatch not overlapping because the intra-node path did not release the GIL, causing overhead drop from 30% to below 1%. (3) A decode kernel recomputing the same normalization four times due to tiling; the agent restructured computation for 1.71x speedup using optimization skeletons distilled from SGLang, FLA, and DeepGEMM.

Engineer Role Shifts from Problem-Solver to Feedback Designer

Tang Jie emphasized that humans still define goals, build feedback environments, and review high-risk changes. But the engineer's role is evolving from solving problems to designing feedback systems. The dense feedback environment serves dual purpose: it enables the agent to work and also generates training data for future models. Each completed task becomes training material for successors, creating a data flywheel.

Source posts