GLM-5.3-Flash Inference Self-Optimization by Infra Agent

Zhipu AI announced that GLM-5.3-Flash inference now runs on over 100,000 domestic AI accelerators, with end-to-end throughput improved 3.2x in under two weeks. The optimization was performed by an AI agent driven by GLM-5.3 itself, using a layered dense feedback system to identify and fix three real engineering issues.
Key insights
Infrastructure Constraints and Trade-offs
The deployment environment involved severe constraints: limited memory and interconnect bandwidth of domestic accelerators, 1M token context, multimodal requests, and immature software stack with missing kernels and documentation. Every optimization was a deliberate trade-off: ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 cache (precision for capacity), and encode-prefill-decode separation (separation for scheduling flexibility). The final per-token cost on domestic chips matched mainstream NVIDIA GPUs.
Dense Feedback Mechanism as the Core Innovation
The agent's bottleneck was never code generation but diagnosing why performance degraded. End-to-end benchmarks took hours, and the agent lacked process rewards. Zhipu AI explicitly encoded experienced engineers' implicit knowledge into three verification signal categories: correctness (is the computation right?), system behavior (where is time spent?), and performance (which solution wins under what conditions?). Each signal is local, cheap, and objectively verifiable.
Three Concrete Case Studies of Agent-Discovered Issues
The agent uncovered and fixed three real problems: (1) KDA kernel accuracy drifted with sequence length due to TF32 rounding errors accumulating in chained state matrix merges; (2) KV transfer could not overlap with DeepEP dispatch because the intra-node path never released the GIL across Python/C++ boundaries; (3) a decode kernel recomputed the same normalization four times due to block partitioning, requiring a 1.71x speedup after restructuring. These cases demonstrate three distinct capabilities: numerical correctness attribution, cross-language debugging, and cross-codebase pattern migration.
Shift in Engineer Role: From Solver to Feedback Designer
Zhipu AI's CEO Tang Jie emphasized that humans still define the goals, build the feedback environment, and review high-risk changes. The engineer's role is evolving from directly solving problems to designing the feedback infrastructure that enables the agent to solve problems. This layered verifiable feedback environment serves double duty: it is both the agent's working environment and training data for the next-generation model.
Recursive Self-Improvement Loop and Future Vision
Tang Jie stated: 'We are far from recursive self-improvement, but the smallest loop already exists: model optimizes the system, system runs the model.' The feedback environment built on real infrastructure tasks creates a data flywheel where each completed task becomes training material for successor agents. The architecture is designed for 'one stone, two birds' – environment design and data flywheel merged.