Google has introduced that it’s increasing its AI-optimised infrastructure portfolio with Cloud TPU v5e. It claims that it’s “essentially the most cost-efficient, versatile, and scalable Cloud TPU thus far.” With the brand new tensor processing unit (TPU), Google goals to deal with the insufficient computing infrastructure that’s unable to deal with growing workloads like generative AI and LLMs.
“The variety of parameters in LLMs has elevated by 10x per yr over the previous 5 years. In consequence, prospects want AI-optimised infrastructure that’s each cost-effective and scalable,” Google mentioned.
“We provide a whole resolution for AI, from computing infrastructure optimised for AI to the end-to-end software program and providers that assist the total lifecycle of mannequin coaching, tuning, and serving at international scale,” it added.
TPU v5e options, specs
In accordance with Google, Cloud TPU v5e is purpose-built to deliver the cost-efficiency and efficiency required for medium- and large-scale coaching and inference. It’s claimed to ship “as much as 2x greater coaching efficiency per greenback and as much as 2.5x inference efficiency per greenback for LLMs and gen AI fashions in comparison with Cloud TPU v4.”
Google mentioned the brand new chip is a mixture of efficiency and suppleness with price advantages.
Learn Additionally
“We stability efficiency, flexibility, and effectivity with TPU v5e pods, permitting as much as 256 chips to be interconnected with an combination bandwidth of greater than 400 Tb/s and 100 petaOps of INT8 efficiency,” Google mentioned. It additionally permits prospects to decide on the fitting configurations to serve a variety of LLM and gen AI mannequin sizes.
Google’s new supercomputer
Google has additionally introduced a brand new model of its supercomputer to run extra generative AI fashions. Referred to as A3 VMs, the machine relies on Nvidia H100 GPUs to energy large-scale AI fashions. The A3 VM options twin next-generation 4th Gen Intel Xeon scalable processors, eight Nvidia H100 GPUs per VM, and 2TB of host reminiscence.
FbTwitterLinkedin
finish of article
