XNews

Z.ai GLM-5.3-Flash Inference Infrastructure Optimized by GLM-5.3 Agent

Z.ai GLM-5.3-Flash Inference Infrastructure Optimized by GLM-5.3 Agent

Z.ai announced that its GLM-5.3-Flash inference system was optimized by a GLM-5.3-driven agent, achieving 3.2x throughput improvement in under two weeks on over 100,000 domestic AI accelerators. The agent used a dense feedback system to identify and fix three engineering issues.

Key insights

Engineering Trade-offs Under Constrained Hardware

The optimization was performed under severe hardware constraints: limited memory and interconnect bandwidth on domestic accelerators, immature software stack, and the need to support 1M token context and multimodal requests. All optimizations involved trade-offs: ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 cache (precision for capacity), and encode-prefill-decode separation (separation for scheduling flexibility). These trade-offs were necessary to achieve the final 3.2x throughput improvement.

Dense Feedback System as Key Innovation

The agent's ability to debug and optimize relied on a dense feedback system that made explicit the implicit process rewards of senior engineers. Three types of signals were provided: correctness (is the computation correct?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal was designed to be local, cheap, and objectively verifiable. This feedback environment allowed the agent to identify root causes without costly end-to-end benchmarks.

Three Real-World Engineering Cases Solved by Agent

The agent discovered and fixed three distinct issues. First, KDA context parallel path precision drift due to TF32 rounding error accumulation in chain state matrix merging. Second, KV transfer and DeepEP dispatch not overlapping because the intra-node path did not release the GIL, causing a 30% overhead reduced to under 1%. Third, a decode kernel redundantly computing the same normalization four times due to block partitioning; the agent restructured it for 1.71x speedup. These cases demonstrate numerical correctness attribution, cross-language debugging, and cross-codebase pattern transfer.

Human Role Shifts from Problem-Solver to Feedback Designer

CEO Tang Jie emphasized that the boundary of human responsibility includes defining goals, building the feedback environment, and reviewing high-risk changes. The engineer's role is evolving from solving problems to designing the feedback that enables the agent to solve problems. This shift is a key insight from the project, as the agent's success depends on the quality of the feedback infrastructure rather than its raw capability.

Recursive Self-Improvement Loop Begins

Tang Jie noted that the layered, verifiable feedback environment serves a dual purpose: it is both the agent's working environment and training data for the next generation of models. Each completed task becomes a training example for future agents. While full recursive self-improvement is still far off, the smallest closed loop now exists: the model optimizes the system, and the system runs the model.

Performance and Scale Achieved

The GLM-5.3-Flash inference system now runs on over 100,000 domestic AI accelerators. From first run to full production took less than two weeks, with end-to-end throughput improved by 3.2x. The single-token cost on domestic chips matched mainstream Nvidia GPUs. These numbers demonstrate the effectiveness of the agent-driven optimization approach under challenging conditions.

Source posts