GLM-5.3 agent optimizes its own inference system

Zhipu AI disclosed that its GLM-5.3-Flash model's inference system was optimized by an AI agent driven by GLM-5.3 itself. The system runs on over 100,000 domestic AI accelerators. End-to-end throughput improved 3.2x within two weeks, with all major optimizations performed by the agent.
Key insights
Agent solved three real engineering problems using dense feedback
The agent identified and fixed precision drift in KDA kernels (TF32 rounding error accumulation), a concurrency bottleneck where KV cache transfer failed to overlap (due to GIL not released), and redundant computation in a decode kernel (normalization computed four times). Each fix was enabled by a layered verification environment providing correctness, system behavior, and performance signals.
Optimizations were trade-offs under domestic hardware constraints
Deployment on Chinese accelerators with limited memory and bandwidth forced trade-offs: ReplaySSM traded compute for memory, intra-node tensor parallelism traded communication for memory, mixed INT8/FP8/BF16 caching traded precision for capacity, and encode-prefill-decode separation traded separation for scheduling flexibility. Single token cost matched leading Nvidia GPUs.
Human role shifts from problem-solver to feedback-designer
Zhipu CEO Tang Jie emphasized that humans still define targets, build the feedback environment, and review high-risk changes. The dense feedback layer is a form of explicit process reward, enabling the agent to localize issues. This environment doubles as training data for subsequent models, creating a closed loop: model optimizes system, system serves model.