XNews

GLM-5.3 Agent Optimizes Its Own Inference System (3.2x Throughput)

GLM-5.3 Agent Optimizes Its Own Inference System (3.2x Throughput)

Z AI's GLM-5.3-Flash inference system was optimized by an AI agent driven by GLM-5.3 itself, achieving 3.2x throughput in under two weeks on over 100,000 domestic accelerators. The agent used a dense feedback system to identify and fix three engineering issues, including precision drift and concurrency bottlenecks.

Key insights

Agent-Driven Optimization Achieves 3.2x Throughput

Z AI disclosed that GLM-5.3-Flash production inference runs on over 100,000 domestic AI accelerators. From first run to production in under two weeks, end-to-end throughput improved 3.2x. The optimizations were performed by an AI agent powered by GLM-5.3 itself. CEO Tang Jie noted that the model is helping optimize the system that runs it.

Challenging Deployment Conditions

The deployment used domestic accelerators with limited memory and bandwidth, immature software stack (missing kernels, documentation by guess), and required support for 1M token context and multimodal requests. Every optimization involved trade-offs: ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (flexibility for scheduling). Final per-token cost matched mainstream Nvidia GPUs.

Dense Feedback System as Key Enabler

The agent's main bottleneck was not writing code but understanding why performance degraded. A 20% throughput drop is a sparse reward with credit assignment problem. Z AI built a dense feedback system with three signal types: correctness (is it right?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal is local, cheap, and objectively verifiable. This transforms the engineer's role from problem-solver to feedback designer.

Three Engineering Issues Found and Fixed

The agent discovered three real problems: (1) Precision drift in KDA context parallel paths due to TF32 rounding errors accumulating in chain state matrix merges. (2) KV transfer and DeepEP dispatch not overlapping because intra-node path didn't release GIL. (3) A decode kernel recomputed the same normalization four times due to tiling. Fixes included merging into Flash Linear Attention, reducing overhead from 30% to under 1%, and 1.71x speedup respectively.

Implications for Recursive Self-Improvement

Tang Jie emphasized that the dense feedback environment serves dual purpose: it is the agent's working environment and also training data for future models. Each completed task becomes training material for successors. He stated 'We are far from recursive self-improvement, but the smallest loop already exists: the model optimizes the system, the system runs the model.'

Source posts