XNews

GLM-5.3-Flash optimized by GLM-5.3-driven Infra Agent

GLM-5.3-Flash optimized by GLM-5.3-driven Infra Agent

Zhipu AI announced that GLM-5.3-Flash inference is fully running on over 100,000 domestic AI accelerators, with end-to-end throughput improved 3.2x in under two weeks. The optimization was performed by an AI agent powered by GLM-5.3 itself, using a layered verification feedback system.

Key insights

AI agent autonomously optimized its own inference system

Zhipu AI's GLM-5.3-Flash inference system was optimized not by engineers but by an AI agent driven by GLM-5.3. The agent completed the work from first run to production in under two weeks, achieving a 3.2x throughput improvement. The deployment runs on over 100,000 domestic AI accelerators with challenging constraints including limited memory and bandwidth.

Key optimizations involved trade-offs under hardware constraints

The optimization involved four main techniques: ReplaySSM (compute for memory), node-level tensor parallelism (communication for memory), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (separation for scheduling flexibility). These were necessary due to limited domestic accelerator memory, interconnect bandwidth, and immature software stack.

Dense feedback system replaced implicit engineer expertise

Zhipu AI built a layered verification interface providing dense feedback signals: correctness (is the computation correct), system behavior (where time is spent), and performance (which solution wins under what conditions). Each signal is local, cheap, and objectively verifiable. This system allowed the agent to diagnose and fix three real engineering problems.

Three real bugs discovered and fixed by the agent

The agent identified and fixed three issues: KDA kernel precision drift from TF32 rounding errors accumulating in chain state matrix merging; KV transfer not overlapping with DeepEP dispatch due to GIL not being released; and a decode kernel recomputing the same normalization four times due to tiling. The fixes included contributions to open-source projects like Flash Linear Attention.

Engineer role shifts from problem solver to feedback designer

Zhipu AI CEO Tang Jie emphasized that the boundary remains: humans define goals, build feedback environments, and review high-risk changes. The engineer's role is changing from solving problems to designing feedback systems. The feedback environment also serves as training data for next-generation models, creating a data flywheel.

Source posts