In the age of AI agents, a new horizon in AI technology, Google has fundamentally restructured its hardware strategy. Moving away from the previous approach of handling both training and inference tasks with a single general-purpose chip, the company has independently developed and unveiled two specialized semiconductors: the TPU 8t for large-scale training and the TPU 8i for high-concurrency inference. Google officially announced these two innovative chips at the Cloud Next 2026 conference in Las Vegas, USA, on the 23rd. Google stated that the current AI market is rapidly diverging from the development stage of creating models to the operational stage of deploying them into actual services, emphasizing that the new chip lineup is designed to meet this shift in demand structure.
While the previous generation, the Ironwood TPU, was a single flagship platform prepared for the inference era, the defining feature of this new generation is its dualized structure. This is due to the explosive increase in demand for infrastructure capable of training even larger models and for inference infrastructure that can execute them quickly and efficiently in cloud environments, driven by the rapid spread of AI agent technology. Reflecting this, Google has adopted a strategy of clearly separating training and inference, demonstrating its resolve to secure performance and efficiency optimized for the characteristics of each stage.
The TPU 8t is a chip focused on workloads centered on large-scale pre-training and embedding. It applies a 3D torus network topology to maximize the scalability of large clusters. The number of connectable chips per pod has reached 9,600, surpassing the 9,216 of the previous model. Its core lies in SparseCore technology and support for 4-bit floating-point operations. SparseCore acts as a dedicated accelerator for handling irregular memory access frequently occurring during large language model retrieval processes, while combining low-bit operations to reduce memory bandwidth burden. This allows for maintaining accuracy with smaller memory usage while doubling throughput, aligning with the trend of reducing bits per parameter through quantization technology, thereby simultaneously reducing power consumption and space requirements.
The TPU 8i is designed for the inference stage where completed models are deployed into actual services, excelling in post-processing of large models and high-concurrency inference requested simultaneously by multiple users. According to Google, this chip is equipped with three times the static RAM of the previous model, allowing it to hold large key-value caches required for LLM inference, thus significantly boosting text generation speed. Additionally, it applies the Collectives Acceleration Engine inference system to accelerate synchronization and reduction operations required in autoregressive decoding and chain-of-thought inference processes. By introducing the Boardfly ICI custom network topology, it allows up to 1,152 chips to be interconnected, reducing the total communication hops by up to 50% by minimizing data packet travel distance and hop count.
Whether Google's current move can shake the market centered on NVIDIA is the focus. Both TPU 8t and 8i have doubled their performance per watt compared to the previous generation, securing power efficiency, which is a core variable determining the profitability of large AI data centers. This goes beyond the mere unveiling of new semiconductor products; it signals Google's formal separation of AI infrastructure strategy into training and inference. In the flow where AI service competition is shifting from model performance to operational costs, response speed, and throughput, this is a strategy to increase the weight of its cloud business. While the market sees the speed of actual customer adoption and software compatibility with the NVIDIA ecosystem as key variables for success, given that the demand for both training and inference is growing simultaneously with the spread of AI agents, Google's dualization strategy is likely to become an important inflection point in future AI infrastructure competition.