XNews

GLM-5.3 optimizes its own inference system

GLM-5.3 optimizes its own inference system

Zhipu AI disclosed that GLM-5.3-Flash inference is running on over 100,000 domestic AI accelerators, with end-to-end throughput improved 3.2x in less than two weeks. The optimization work was performed by an AI agent driven by GLM-5.3, which autonomously identified and fixed three real engineering issues.

Key insights

Agent-Driven Optimization Achieves 3.2x Throughput

Zhipu AI's GLM-5.3-powered agent optimized the inference pipeline for GLM-5.3-Flash, running on over 100,000 domestic AI accelerators. End-to-end throughput improved 3.2x within two weeks of initial deployment. The agent identified and fixed three real engineering issues including precision drift, concurrency bottleneck, and duplicate compute.

Dense Feedback Environment as Key Innovation

The agent's success relied on a layered verification system providing three signal types: correctness (is compute correct?), system behavior (where is time spent?), and performance (which approach wins under what conditions). Each signal had to be local, cheap, and objectively verifiable. This dense feedback design enabled the agent to autonomously diagnose and fix issues.

Three Real Engineering Issues Discovered

The agent discovered three cases: 1) precision drift in KDA context parallel paths due to TF32 rounding errors accumulating over long sequences; 2) KV cache transfer failing to overlap with DeepEP dispatch because the intra-node path did not release GIL; 3) a decode kernel recomputing the same normalization four times due to chunking, simplified to 1.71x speedup. All fixes were merged upstream.

Eagineer Role Transitions to Feedback Designer

Zhipu AI CEO stated that the bottleneck is not writing code but understanding why performance degrades. Senior engineers have implicit process-reward knowledge. The dense feedback system externalizes that experience. Humans now define goals, build feedback environments, and review high-risk changes, shifting from problem-solving to designing feedback.

Minimal Recursive Self-Improvement Loop Exists

The feedback environment is double-used: it enables the current agent and provides training data for future models. Each completed task becomes training material for the next. The authors note that recursive self-improvement is still distant but the smallest closed loop exiss: model optimizes the system, system serves the model.

Optimization Under Constraint: Domestic Hardware and Software Imaturity

The deployment environment included constrained memory and interconnct bandwidth on domestic accelerators, immature software stack with missing kernels and docs. All optimizations involved trade-offs: ReplaySSM (compute for mem), node-intern tensor parallel (comunic for mem), mixed INT8/FP8/BF16 caching (precision for capacity), and encode-prefill-decode separation (separation for scheduling flexibility).

Source posts