XNews

GLM-5.3-Flash inference infrastructure optimized by its own AI agent

GLM-5.3-Flash inference infrastructure optimized by its own AI agent

Z.ai announced on September 17, 2026 that its GLM-5.3-Flash model, running on over 100,000 domestic AI accelerators, achieved a 3.2x end-to-end throughput gain in under two weeks. The optimization was driven by a GLM-5.3-powered infrastructure agent that applied techniques like ReplaySSM, tensor parallelism, mixed-precision caching, and encode-prefill-decode separation.

Key insights

Agent-driven discovery of three real engineering bugs

The infrastructure agent found and fixed three production issues. First, TF32 rounding errors in KDA convolution parallel paths caused precision drift with longer sequences. Second, KV cache transfer could not overlap with DeepEP dispatch because the node-internal path did not release the GIL, keeping transmission threads blocked. Third, a decode kernel recomputed the same normalization four times due to tiling structure; the agent reorganized it for a 1.71x speedup.

Dense feedback environment as the core enabler

Z.ai built a layered verification interface providing three signal types: correctness (is the math right), system behavior (where is time spent), and performance (which solution wins under what conditions). Each signal must be local, cheap to obtain, and objectively verifiable. This turns the agent's sparse reward problem into a dense feedback environment, allowing it to debug without running costly end-to-end benchmarks for every hypothesis.

A new engineer role: from problem solver to feedback designer

Z.ai CEO Tang Jie stated that the bottleneck was never code writing but understanding 'why it got worse.' The dense feedback environment externalizes the implicit process reward that senior engineers use. Humans still define goals, build the feedback environment, and review high-risk changes. The resulting approach treats each completed task as training data for the next generation of models, creating a recursive improvement loop.

Performance and cost parity with NVIDIA GPUs on domestic hardware

Running on over 100,000 domestic AI accelerators with limited memory and bandwidth, the optimized GLM-5.3-Flash achieved per-token cost parity with mainstream NVIDIA GPUs. The model supports 1 million token context and multimodal inputs. Key trade-offs included compute for memory (ReplaySSM), communication for memory (tensor parallelism), precision for capacity (INT8/FP8/BF16 caching), and separation for scheduling flexibility.

Source posts