XNews

GLM-5.3-Flash Optimizes Own Inference on 100K+ Domestic Accelerators

GLM-5.3-Flash Optimizes Own Inference on 100K+ Domestic Accelerators

Z.ai announced that GLM-5.3-Flash's entire production inference now runs on over 100,000 domestic AI accelerators, achieving a 3.2x throughput improvement within two weeks. The optimization was largely carried out by a GLM-5.3-powered AI agent, which used dense feedback signals to identify and fix three real engineering issues.

Key insights

Agent-driven optimization under constrained hardware conditions

The deployment faced severe constraints including limited memory and bandwidth on domestic accelerators, 1M token context, multimodal requests, and immature software stacks. Optimizations involved trade-offs: ReplaySSM (compute for memory), node-level tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (separation for scheduling freedom).

Dense feedback environment: the key to agent success

Z.ai's solution transformed implicit engineering expertise into explicit, layered verification interfaces. Three signal types were provided: correctness (does the computation work?), system behavior (where is time spent?), and performance (which approach wins under what conditions?). Each signal met three criteria: it had to be local, cheap to obtain, and objectively verifiable. This is described as designing an environment for the agent rather than enhancing the agent's capabilities.

Three agent-discovered engineering issues and their root causes

The agent identified and fixed three real problems. First, KDA context parallelism experienced accuracy drift with sequence length due to TF32 rounding errors accumulating in chain-like state matrix merges. Second, KV transfer and DeepEP dispatch could not overlap because the node-internal path failed to release the GIL, preventing concurrency. Third, a decoder kernel redundantly computed the same normalization four times due to its tiling strategy; the agent restructured it for 1.71x speedup based on optimization skeletons distilled from reading code in SGLang, FLA, and DeepGEMM.

Engineer role shift and the path to recursive self-improvement

The boundary is defined by human-set goals, feedback environment design, and human review of high-risk changes. The engineer's role shifts from problem solver to feedback designer. CEI CEO Tang Jie noted that the layered verification environment serves a dual purpose: it is the agent's work environment and also training material for the next-generation model. He stated: 'We are still far from recursive self-improvement, but the smallest closed loop already exists. The model optimizes the system; the system serves the model.'

Source posts