GLM-5.3-Flash Inference Optimized by Its Own AI Agent

Zhipu disclosed that GLM-5.3-Flash inference runs on over 100,000 domestic AI accelerators, with throughput improved 3.2x in two weeks. The optimization was performed by an AI agent powered by GLM-5.3 itself, using a layered verification feedback system. The agent identified and fixed three real engineering issues.
Key insights
Optimization results and constraints
GLM-5.3-Flash inference is deployed on over 100,000 domestic AI accelerators with limited memory and interconnect bandwidth. The model supports 1M token context and multimodal requests. Throughput was improved 3.2x end-to-end in two weeks. Key trade-offs included ReplaySSM (compute for memory), intra-node tensor parallelism (communication for memory), mixed INT8/FP8/BF16 cache (precision for capacity), and encode-prefill-decode separation (flexibility for scheduling). Single token cost matched mainstream NVIDIA GPUs.
AI agent-driven optimization process
The optimization was performed by an AI agent powered by GLM-5.3 itself. The agent faced sparse reward and credit assignment problems: 'throughput dropped 20%' tells something is wrong but not where. Zhipu built a layered verification feedback system with three signal types: correctness (is the computation correct?), system behavior (where is time spent?), and performance (which solution wins under what conditions). Each signal is local, cheap, and objectively verifiable.
Three engineering issues discovered and fixed
The agent found three real problems: (1) KDA context parallel path precision drift: TF32 rounding errors accumulated in chained state matrix merges, worsening with longer context. (2) KV transfer and DeepEP dispatch not overlapping: the agent traced the call chain across Python/C++ boundary and found the intra-node path not releasing GIL, causing transfer threads to never execute. (3) Decode kernel repeated the same normalization four times due to tiling; the agent restructured computation for 1.71x speedup, using optimization skeletons distilled from reading SGLang, FLA, and DeepGEMM kernel code.
Engineer role shift and recursive self-improvement vision
Zhipu CEO Tang Jie emphasized that the bottleneck is not writing code but knowing why things got worse. The agent's layered verification environment turns implicit engineer experience into explicit signals. Engineers shift from problem solvers to feedback designers. Tang noted that each completed task becomes training material for the next generation of models, creating a data flywheel. He stated: 'We are far from recursive self-improvement, but the smallest loop already exists: model optimizes system, system serves model.'
Boundaries and human oversight
Humans still define goals, build the feedback environment, and review high-risk changes. The agent's autonomy is bounded by these human-set guardrails. The layered verification system is described as 'one fish, two eats': it serves as both the agent's working environment and training data for future models. The approach is not yet fully autonomous but demonstrates a minimal closed loop of self-improvement.