Google Splits 8th Gen TPU into Training (8t) and Inference (8i) Chips, 3x Performance

Google's 8th generation TPU splits into two specialized chips: TPU 8t for training and TPU 8i for inference. The split addresses diverging requirements: training needs massive data throughput, while inference demands low latency. In the Agent era, multi-step reasoning tasks amplify latency sensitivity. TPU 8t targets trillion-parameter model pretraining with 9600 chips delivering 121 exaflops, 2 PB shared memory, and 3x peak performance over previous gen. TPU 8i handles post-training and real-time inference, halving latency for communication-intensive workloads.