Google launches training and inference TPUs in latest shot at Nvidia

After years of producing chips that can both train artificial intelligence models and run inference jobs, Google separates these tasks into different processors; his last effort in this regard Nvidia in AI hardware.

Google said on Wednesday it will be making changes for the eighth generation of its tensor processing unit (TPU). Both chips will be available later this year.

“With the rise of AI agents, we determined that the community would benefit from individually specialized chips based on their educational and service needs,” Amin Vahdat, Google’s senior vice president and chief technologist for AI and infrastructure, said in a blog post.

In March, Nvidia mentioned that the silicon that will enable models to quickly respond to users’ questions will soon be released, thanks to the technology it acquired through a $20 billion agreement with chip startup Groq. Google is a major Nvidia customer but offers TPUs as an alternative to companies using cloud services.

Many of the world’s leading technology companies are pursuing specialized semiconductor development for AI to maximize efficiency so they can produce products for specific use cases. Apple It has been incorporating neural engine AI components into its in-house iPhone chips for years. Microsoft It announced its second-generation AI chip in January. Last week, Meta He said he was working with broadcom Developing multiple versions of AI processors.

Google was at the head of this trend. The company began using processors it designed to run artificial intelligence models in 2015 and began renting them to cloud clients in 2018. Amazon In 2018, Web Services announced the Inferentia chip for processing AI requests, and in 2020 it introduced the Trainium processor for training AI models.

DA Davidson analysts estimated in September that the TPU business, along with Google’s DeepMind AI group, would be worth about $900 billion.

None of the tech giants are replacing Nvidia, and Google isn’t even comparing the performance of its new chips to those of the AI ​​chip leader. Google said the training chip delivers 2.8 times the performance of the seventh-generation Ironwood TPU announced in November for the same price, and performance is 80% better for the inference processor.

Nvidia said its upcoming Groq 3 LPU hardware will leverage large amounts of static random access memory, or SRAM, used by Cerebras, an AI chipmaker that went public earlier this month. Google’s new inference chip, called TPU 8i, also relies on SRAM. Each chip contains 384 megabytes of SRAM, three times the amount in Ironwood.

The architecture “is designed to deliver the massive throughput and low latency required to cost-effectively run millions of agents simultaneously,” Sundar Pichai, CEO of Google parent company Alphabet, wrote in a blog post.

Adoption of Google’s AI chips is on the rise. Google said Citadel Securities has developed quantitative research software that leverages Google’s TPUs, and all 17 U.S. Department of Energy national laboratories use AI collaborative scientist software built on the chips. Anthropic has committed to using multiple gigawatts worth of Google TPU.

WRISTWATCH: Broadcom agrees to expanded chip deal with Google and Anthropic