Google announced the expansion of its eighth-generation tensor processor (TPU) product lineup, launching two new systems, including "TPU 8tx" designed for secure and confidential inference, and "TPU 8i Max" designed for large context windows and high-density computing. Together with the TPU 8t and 8i launched in April this year, Google launched four TPU systems in the same year for the first time, further expanding its AI infrastructure layout.
Google stated that TPU was originally developed more than ten years ago to meet the needs of internal speech recognition, language translation, etc., and has now developed to the eighth generation. The application scope has also expanded from the training and inference of Google's own Gemini model, to AI agent workloads, and the cloud computing needs of Google Cloud customers. Recently, it has opened customers to deploy TPU directly in their own data centers. At present, TPU has become one of Google's rapidly growing businesses, with nearly 100 customers, including cutting-edge AI research institutions, trading companies, retailers and universities.
At the Google Cloud Next conference held in April this year, Google has taken the lead in releasing two eighth-generation TPU systems. TPU 8t is mainly responsible for large-scale AI model training, while TPU 8i targets real-time inference requirements. Google pointed out that compared with the previous generation TPU, the training performance of TPU 8t is increased by up to 2.7 times, and the inference cost-effectiveness of TPU 8i is increased by 80%.
Google has further expanded its eighth-generation TPU product lineup this time, launching two new systems, including TPU 8i Max, which is specially designed to handle large contextual windows and increase the computing density of a single cabinet, and TPU 8tx, which enhances security and confidential inference capabilities.
Google explained that in the past, TPU was mainly co-designed with Google DeepMind, but as the application needs of external companies have become increasingly diverse, Google has begun to incorporate customer opinions into product development more frequently. Especially as AI computing moves from a human-scale operation to a large number of AI agents operating collaboratively, enterprises’ demand for AI accelerators that can handle huge context windows continues to increase. Some highly regulated industries have also begun to require dedicated encryption functions for specific workloads.
In response to these emerging needs, Google has launched two new TPU systems, coupled with the recently released PyTorch related updates, to further lower the threshold for developers and Google Cloud customers to use TPU. By combining modular self-developed chips with an open and easy-to-use software architecture, Google hopes to respond more quickly to the development of the AI industry and launch hardware and system configurations that meet different computing needs.
TPU 8tx is the third system of the eighth generation. It is optimized for secure and confidential inference and is also very suitable for small-scale training. It is designed for highly regulated enterprises and government agencies, allowing them to safely execute cutting-edge or proprietary models and autonomous AI agents while processing highly sensitive data.
Google points out that general data is encrypted when it is stored or transmitted over the Internet, but during computer processing of the data, it must first be decrypted in memory. TPU 8tx uses the same silicon chip as TPU 8t, and is paired with the fifth-generation AMD EPYC CPU host. The security mechanism is built directly into the hardware layer to ensure that sensitive data and model weights are fully protected and isolated even during active use.
In addition, by using the same low-latency Boardfly network built for real-time inference, TPU 8tx allows multi-step AI agents to keep running smoothly, providing up to 2x lower latency than TPU 8t with almost zero performance loss.
Combining training-grade computing power with high-speed inference networks means companies can simultaneously execute and customize their models on a single system. TPU 8tx allows teams to fine-tune models with proprietary data, distill large models into smaller models, and train their AI agents through reinforcement learning, all without moving sensitive data between clusters.
Another new system, TPU 8i Max, is designed to solve the memory bottleneck faced by AI agents when they operate together. When dozens of AI agents work together on entire software code libraries or hours-long videos, memory bandwidth can quickly become a bottleneck. To this end, Google integrated and packaged two low-latency TPU 8i chips on a single module.
TPU 8i Max simultaneously expands computing and memory resources, providing double the computing power, double the static random access memory (SRAM), and double the high-bandwidth memory (HBM) bandwidth of TPU 8i, ensuring that the processor has critical data ready at all times to avoid stagnation in large workloads. In addition, because TPU 8i Max accommodates twice as many silicon chips in the existing cabinet space, the computing power that can be deployed per square foot and per watt of power in the data center is greatly increased.
Google stated that the four systems of the eighth-generation TPU are each designed for different computing needs, covering all stages of the AI agent application life cycle. Among them, TPU 8t is mainly responsible for large-scale model training, TPU 8i locks in low-latency inference, TPU 8tx focuses on confidential computing, and TPU 8i Max is specially designed to handle huge contextual windows and increase the computing density of a single cabinet.
Google specifically emphasized that it is launching two new TPU "systems" rather than redesigned "chips". The two are different in development methods and time required. Generally speaking, the design of new chips involves the layout of billions of micro-transistors, and the manufacturing process is complex and often takes many years. In contrast, new systems are based on existing chips, integrating computing, network and storage resources, and adjusting related packaging and configuration to respond more quickly to customer needs and emerging research applications.
Google pointed out that when it developed the first-generation TPU more than ten years ago, it had foreseen that the evolution speed of AI software would exceed the chip manufacturing cycle. Therefore, it adopted a modular design and used TPU as the basic computing module of the AI supercomputer (AI Hypercomputer), allowing engineers to flexibly adjust chip packaging, high-speed memory, optical networks and security protection mechanisms according to different needs. Through this design approach, Google was able to shorten the delivery time of a dedicated supercomputer from years to just a few months.
