XNews

Zhipu GLM-5.3 Agent Optimizes Its Own Inference System

Zhipu GLM-5.3 Agent Optimizes Its Own Inference System

Zhipu AI disclosed that its GLM-5.3-driven Infra Agent optimized the GLM-5.3-Flash inference system on over 100,000 domestic AI accelerators in under two weeks, boosting throughput 3.2x. The agent fixed real engineering issues including precision drift and concurrency bottlenecks using a dense feedback system.

Key insights

Agent-Driven Optimization on Challenging Hardware

The optimization was executed on domestic accelerators with limited memory and bandwidth, running 1M token context multi-modal requests. The Infra Agent applied four key trade-offs: using computation for memory (ReplaySSM), communication for memory (tensor parallelism), precision for capacity (INT8/FP8/BF16 hybrid cache), and separation for scheduling flexibility (encode-prefill-decode separation). The final cost per token matched mainstream Nvidia GPUs.

The Dense Feedback Design: From Sparse Reward to Actionable Signals

Zhipu identified that the main bottleneck was not code writing but diagnosing why performance degraded. They built a layered verification interface providing three signal types: correctness (computation accuracy), system behavior (time allocation), and performance (solution superiority under conditions). Each signal is local, cheap, and objectively verifiable, effectively giving the agent an explicit 'process reward' instead of a sparse reward.

Three Real-World Bugs Found and Fixed by the Agent

The agent discovered and fixed three non-trivial engineering problems. First, TF32 rounding errors in KDA context parallel paths caused chain accumulation of precision drift; fix merged into Flash Linear Attention. Second, KV transfer and DeepEP dispatch could not overlap because the intra-node path failed to release the GIL, hiding a concurrency bottleneck. Third, a decode kernel normalized the same computation four times; the agent restructured it for a 1.71x speedup using optimization patterns distilled from open-source kernels.

Implications for Recursive Self-Improvement

Zhipu's CEO stated that the layered verifiable feedback environment serves a dual purpose: it is the agent's working environment and also training data for future models, as each task completed becomes a training example for its successor. They described this as the 'smallest closed loop' toward recursive self-improvement, where the model optimizes the system and the system serves the model.

Source posts