Google teams up with Meta to challenge Nvidia’s dominance in AI computing market — here’s how it plans to do it

Alphabet’s Google is working on a new initiative to develop its own AI chips so they can work well at running PyTorch, a software most people use to build and run AI models.

The tech giant aims to weaken Nvidia’s long-standing dominance in the AI ​​computing market by ensuring PyTorch runs smoothly on Google chips. Reuters.

Google is working hard to make its Tensor Processing Units a viable alternative to Nvidia’s market-leading GPUs. These chips are becoming a major source of revenue for Google Cloud, which aims to show investors that their AI investments actually yield returns.

What is this new initiative?

But just having powerful hardware isn’t enough to drive adoption. The new initiative, known internally as “TorchTPU,” is designed to remove a major hurdle that has slowed the uptake of TPU chips by making TPU chips fully compatible with PyTorch software and making them easier for developers to use.

Google is also considering open-source parts of the software to speed up adoption among customers, Reuters reported, citing sources familiar with the matter.

PyTorch, an open source project heavily supported by Meta Platforms, is used by developers worldwide to build artificial intelligence models. In Silicon Valley, very few developers write every line of code that Nvidia’s, Advanced Micro Devices’ or Google’s chips will actually run.

Instead, these developers rely on tools like PyTorch, a collection of pre-written code libraries and frameworks that automate many common tasks in AI software development.

Nvidia and Google

Nvidia’s engineers have spent years optimizing their chips to make PyTorch-based software run as quickly and efficiently as possible. By contrast, Google has traditionally relied on Jax, a different framework widely used by its internal teams, with TPU chips optimized through a compiler called XLA. As a result, much of Google’s AI software stack and performance optimization is built around Jax; This creates a growing mismatch between the way Google designs and uses its chips and the way most customers prefer to work with them.

Alphabet has long reserved the majority of its chips, or TPUs, for internal use only. That changed in 2022, when Google’s cloud computing unit successfully lobbied to audit the group selling TPU. This move significantly increased Google Cloud’s TPU allocation, and as customer interest in AI grew, Google sought to raise capital by increasing production and sales of TPUs to external customers.

But the incompatibility between the PyTorch frameworks used by most of the world’s AI developers and the Jax frameworks that Google’s chips are currently finely tuned to run means that most developers won’t be able to easily adopt Google’s chips and make them perform as well as Nvidia’s chips without doing significant, extra engineering work. Such studies require time and money in the fast-paced AI race.

If the project is successful, Google’s “TorchTPU” initiative has the potential to significantly reduce switching costs for companies looking for alternatives to Nvidia’s GPUs.

Nvidia’s dominance of the AI ​​chip market is a result not just of its hardware, but of its CUDA software ecosystem, which is deeply embedded in PyTorch and has become the default method by which companies train and run large AI models. Reuters reported.

Google reached an agreement with Meta

To accelerate the pace of growth, Google is now working closely with Meta, the creator and maintainer of PyTorch. Reuters. The two tech giants are simultaneously discussing agreements to give Meta access to more TPUs.

Initial offerings for Meta were structured as Google managed services; Within these services, customers like Meta were using Google chips designed to run Google software and models. On the other hand, Google was tasked with providing operational support.

According to the Reuters report, Meta has a strategic interest in developing software that makes it easier to operate TPUs; as this could help reduce inference costs and strengthen negotiation power by moving AI infrastructure away from Nvidia’s GPUs.