XNews

Zhipu Uses GLM-5.3 to Optimize Its Own Inference System, Achieving 3.2x Throughput

Zhipu Uses GLM-5.3 to Optimize Its Own Inference System, Achieving 3.2x Throughput

Zhipu revealed GLM-5.3-Flash production inference now runs on over 100,000 domestic AI accelerators after a GLM-5.3-driven agent optimized its own system. Throughput rose 3.2x from first run to production in under two weeks. Zhipu credited dense feedback interfaces; humans set goals and review high-risk changes.

Key insights

GLM-5.3-driven agent optimized its own inference system

Zhipu said GLM-5.3-Flash production inference now runs entirely on more than 100,000 domestic AI accelerators. The optimization work was driven by an Infra Agent powered by GLM-5.3, not by a conventional engineering team. From first run to production took under two weeks, and end-to-end throughput rose 3.2 times. CEO Tang Jie said the recurring thought was that the model itself became the main worker, helping optimize the system that runs it. The project was disclosed in a company article on recursive self-improvement.

Hardware constraints shaped each optimization

The deployment conditions included limited memory and interconnect bandwidth on domestic accelerators, an immature software stack with missing kernels and sparse documentation, plus 1 million token context and multimodal requests. Zhipu described each optimization as a trade-off: ReplaySSM trades compute for memory, intra-node tensor parallelism trades communication for memory, mixed INT8/FP8/BF16 caching trades precision for capacity, and encode-prefill-decode separation trades architecture for scheduling flexibility. The result was per-token cost comparable to mainstream Nvidia GPUs.

Stuck agents need better feedback, not better code

Tang Jie said the bottleneck was not writing code but understanding why performance regressed. A message like 'throughput dropped 20%' does not tell which layer or assumption failed or what to test next. End-to-end benchmarks take hours, making exploration expensive. He framed this as a sparse-reward and credit-assignment problem. Senior engineers carry an implicit process reward in their heads, knowing when to check timelines, run microbenchmarks, or compare layer outputs. Zhipu's solution was to make that implicit experience explicit through dense feedback.

Dense feedback signals are local and verifiable

Dense feedback in Zhipu's system is organized into three signal types: correctness, which tells whether a computation is right; system behavior, which tells where time is spent; and performance, which tells which solution wins under which conditions. Every signal must be local, cheap to obtain, and objectively verifiable. Tang Jie described this as designing an environment for the agent, not giving the agent more abilities. The same feedback environment also becomes training material for future models, because each completed task becomes a training case for successors.

Agent diagnosed and fixed three real bugs

The agent found and fixed three real engineering problems. First, KDA's context-parallel path had precision drift because TF32 rounding errors accumulated in chained state-matrix merging; the fix was merged into Flash Linear Attention. Second, KV cache transfer and DeepEP dispatch never overlapped because the intra-node path failed to release the GIL across the Python/C++ boundary; fixing this cut overhead from more than 30% to below 1%. Third, a decode kernel recomputed the same normalization four times; the agent restructured it for a 1.71x speedup using kernel patterns distilled from SGLang, FLA, and DeepGEMM.

Humans keep control while engineers become feedback designers

Zhipu kept humans responsible for goal setting, feedback environment construction, and review of high-risk changes. The engineer role shifts from solving problems to designing feedback. Tang Jie drew a deeper implication: a layered, verifiable feedback environment built on real infrastructure tasks is exactly the material needed to train the next generation of models. He said the field is far from recursive self-improvement, but the smallest closed loop now exists: the model optimizes the system, and the system runs the model. In his view, this is the boundary of the current loop.

Source posts