XNews

GLM-5.3-Flash Inference Optimized by GLM-5.3 Agent

GLM-5.3-Flash Inference Optimized by GLM-5.3 Agent

Zhipu announced a GLM-5.3-powered agent optimized GLM-5.3-Flash inference on 100,000+ domestic accelerators, raising throughput 3.2x in under two weeks. Dense feedback let the agent fix precision drift, a GIL concurrency bug, and redundant compute while humans reviewed high-risk changes.

Key insights

Agent did the work: two weeks from first run to production

Zhipu disclosed that GLM-5.3-Flash production inference now runs on more than 100,000 domestic AI accelerators, and that an agent powered by GLM-5.3 performed the optimization work rather than an engineering team. From the model's first successful run to production deployment took less than two weeks, with end-to-end throughput up 3.2x. CEO Jie Tang said the main workforce was the model itself — a model helping optimize the very system that runs it — and that per-token cost on domestic chips now matches mainstream NVIDIA GPUs.

Every optimization was a trade-off under harsh constraints

The deployment environment was demanding: domestic accelerators with limited memory and interconnect bandwidth, an immature software stack with missing kernels and documentation that had to be guessed at, plus requirements for 1M-token context and multimodal requests. Each optimization was therefore an exchange: ReplaySSM traded compute for memory, intra-node tensor parallelism traded communication for memory, hybrid INT8/FP8/BF16 caching traded precision for capacity, and Encode-Prefill-Decode disaggregation traded separation for scheduling freedom. No single change was a free win.

Dense feedback attacks diagnosis, not coding ability

The posts argue the agent stalled almost never because it could not write code, but because it did not know why things got worse. A signal like 'throughput dropped 20%' is a sparse-reward plus credit-assignment problem — it reports breakage without identifying the layer, the hypothesis to verify, or the next test — while a single end-to-end benchmark run takes hours. Z.ai made senior engineers' implicit process rewards explicit as 'dense feedback': layered verification interfaces with three signal classes — correctness, system behavior, and performance — each required to be local, cheap, and objectively verifiable. This designs the environment for the agent instead of adding capability to the agent.

Three fixes showcase three distinct agent capabilities

The agent found and fixed three representative problems, one per capability. Numerical correctness attribution: TF32 rounding errors accumulated through chained state-matrix merges in the KDA context-parallel path, causing precision drift that grew with sequence length; the fix was merged into open-source Flash Linear Attention. Cross-language system debugging: KV cache transfer never overlapped with DeepEP dispatch because the intra-node path failed to release the GIL at the Python/C++ boundary; after the fix, overhead fell from over 30% to below 1%. Cross-codebase pattern transfer: a decode kernel recomputed the same normalization four times due to its chunking; restructuring using 'optimization skeletons' distilled from SGLang, FLA, and DeepGEMM kernels yielded a 1.71x speedup.

Humans keep goals, environment, and high-risk review

Both posts stress the boundary: humans define the goals, build the feedback environment, and review every high-risk change. Tang Jie framed this as the most important lesson — more important than any individual optimization. The derived role transition: engineers shift from people who solve problems to people who design feedback. The dense-feedback interfaces were built on real infrastructure tasks rather than synthetic benchmarks, and the posts note that senior engineers' edge already lies in implicit process rewards — knowing when to check timelines, run microbenchmarks, and compare layer outputs — which now get encoded as agent-facing tooling.

Feedback environment doubles as training data flywheel

The deepest implication flagged: a layered, verifiable feedback environment built on real infrastructure tasks is exactly what training next-generation models requires, because every task the agent completes becomes training material for its successors — merging environment design with the data flywheel. Tang Jie's closing formulation: 'We are far from recursive self-improvement, but the minimal loop already exists; models optimize systems, and systems serve models.' The official write-up is titled 'Toward Recursive Self-Improvement: How GLM Built Its Own Inference Infrastructure.'

Source posts