GLM-5.3 Agent Optimizes Its Own Inference System

Zhipu AI revealed a GLM-5.3-driven agent optimized its own production inference system on over 100,000 domestic accelerators, lifting throughput 3.2x in two weeks. It credited layered feedback interfaces covering correctness, system behavior, and performance, with humans reviewing high-risk changes.
Key insights
GLM-5.3 Agent Optimized Its Own Production Inference
Zhipu disclosed that GLM-5.3-Flash's full production inference runs on over 100,000 domestic AI accelerators, moving from first successful run to production in under two weeks with a 3.2x end-to-end throughput improvement. The bulk of the optimization work was completed by an agent powered by GLM-5.3 itself rather than an engineering team. CEO Tang Jie emphasized that the main working force was the model, with one model helping optimize the very system that runs it, and noted the domestic chips' per-token cost now matches mainstream Nvidia GPUs.
Every Optimization Was a Trade-Off Under Harsh Constraints
Deployment conditions included limited memory and interconnect bandwidth on domestic accelerators, an immature software stack with missing kernels and documentation that had to be guessed, plus requirements for 1M token context and multimodal requests. Each optimization was an explicit trade-off: ReplaySSM traded compute for memory, in-node tensor parallelism traded communication for memory, hybrid INT8/FP8/BF16 caching traded precision for capacity, and encode-prefill-decode separation traded architectural coupling for scheduling freedom.
Dense Feedback, Not Code Skill, Unlocked Agent Performance
Tang Jie said the agent stalled not because it could not write code but because it did not know why things got worse: a 20% throughput drop signals a problem without revealing the layer, the hypothesis to verify, or the next test, which is a sparse-reward plus credit-assignment problem, while a single end-to-end benchmark run takes hours. Zhipu's answer was dense feedback: layered verification interfaces with three signal classes — correctness (computed right), system behavior (where time goes), and performance (which option wins under what conditions) — each required to be local, cheap, and objectively verifiable. This designs the environment for the agent rather than supplementing the agent's capabilities.
Three Real Bugs Found and Fixed by the Agent
First, the KDA kernel's precision drifted with sequence length on the context-parallel path as TF32 rounding errors accumulated through chained state matrix merges; the fix merged into the open-source Flash Linear Attention project. Second, KV cache transfer never overlapped with DeepEP dispatch because the in-node path did not release Python's GIL across the C++ boundary; after the agent traced the call chain, overhead fell from over 30% to under 1%. Third, a decode kernel recomputed the same normalization four times due to its chunking scheme; restructuring yielded a 1.71x speedup, using optimization skeletons the agent distilled from reading SGLang, FLA, and DeepGEMM kernel code.
Engineers Shift from Problem-Solvers to Feedback Designers
The boundaries remain human-defined: goal setting, feedback environment construction, and review of every high-risk change stay with people, but the engineering role changes from solving problems to designing feedback. The three reported cases each exercised a distinct capability — numerical correctness attribution, cross-language boundary system debugging, and cross-codebase pattern transfer — indicating the selection was representative rather than low-hanging fruit. Senior engineers' implicit process rewards, knowing when to check timelines, run micro-benchmarks, and compare layer outputs, were made explicit as callable interfaces.
Feedback Environment Doubles as Next-Generation Training Data
Tang Jie argued the layered, verifiable feedback environment built on real infrastructure tasks serves two purposes: it is the agent's working environment and, because every completed task becomes training material for successors, the training ground for next-generation models, merging environment design with a data flywheel. His closing statement: we are far from recursive self-improvement, but the minimal loop already exists — the model optimizes the system, and the system serves the model.