NVIDIA, which holds absolute dominance in the AI training chip market, recently executed a special transaction worth approximately $20 billion, acquiring the core team and technology of Groq—one of its most promising challengers in the inference accelerator space. This deal is not a traditional full-scale company acquisition but a strategic move described by the industry as a "deconstructed control transaction." It signals NVIDIA's all-out effort to fortify its moat in the AI inference market, addressing the market shift from training to inference and the increasingly intense competition.

An Unusual Mega-Deal

According to CNBC, NVIDIA utilized one-third of its $60 billion cash reserve just before Christmas to complete this critical strategic move. The core of the transaction is a non-exclusive licensing agreement with AI inference chip startup Groq, along with the recruitment of its key engineering team, including founder and CEO Jonathan Ross and President Sunny Madra. These top-tier talents will all join NVIDIA to advance the licensed inference technology.

Notably, NVIDIA did not acquire Groq's corporate entity. The former Chief Financial Officer, Simon Edwards, will take charge of the remaining "shell company," whose primary asset is Groq's nascent cloud business (selling AI accelerator access via APIs). This sophisticated design allows NVIDIA to secure critical talent and chip technology while avoiding direct competition with its large cloud service customers. It is also seen as a flexible strategy to circumvent stringent antitrust scrutiny.

Groq: A Dark Horse in Inference Chips Born from Google

Founded in 2016 by a group of former Google engineers, many of whom contributed to Google's Tensor Processing Unit (TPU), Groq's core value lies in its Language Processing Unit (LPU), specifically designed for AI inference.

Unlike GPUs that rely on external high-bandwidth memory, Groq's LPU innovatively integrates hundreds of megabytes of SRAM as the primary weight storage. This design eliminates memory access bottlenecks, significantly reduces latency, and enables computation units to operate at full speed with higher energy efficiency. Groq claims that its LPU can achieve up to 10 times greater energy efficiency than NVIDIA and AMD GPUs when performing AI inference tasks.

The LPU adopts a single-core sequential processing architecture, making it particularly suitable for tasks that require sequential information processing, such as large language models. This gives it a significant advantage in scenarios like chatbots and real-time AI interactions.

Why $20 Billion?

A thought-provoking question arises: Why is NVIDIA willing to pay such a massive premium for a company with annual revenue targets around $500 million and a last-round valuation of $6.9 billion?

Industry analysis points to a strategic shift in the AI market's focus. NVIDIA's founder and CEO, Jensen Huang, has repeatedly emphasized that as AI development enters a new phase, the market is transitioning from "training" to "inference." Trained models need to make real-time decisions or predictions in practical applications, driving enormous demand for efficient, low-latency inference hardware.

Although NVIDIA dominates the training market, the inference market is fragmented and crowded with competitors. Besides traditional chip giants like AMD and Broadcom, there are startups like Cerebras, as well as cloud giants such as AWS, Google, and Microsoft developing their own inference chips. Groq's demonstrated potential in high-performance inference workloads makes it a critical strategic target.

This $20 billion deal not only serves as NVIDIA's expensive endorsement of the inference market's importance but also clearly indicates that real-time AI inference has become the main battlefield for the next generation of AI hardware competition.

Strategic Significance and Industry Impact

For NVIDIA, this is a strategic move that combines offense and defense. Against the backdrop of numerous core customers developing their own AI chips or seeking GPU alternatives, this move directly absorbs the core innovation capabilities of potential challengers, consolidating NVIDIA's leadership position.

According to an email from Jensen Huang to internal employees, NVIDIA plans to "integrate Groq's low-latency processors into the NVIDIA AI factory architecture" to expand its platform's services for a broader range of AI inference and real-time workloads. The future technological roadmap warrants attention: Will NVIDIA integrate Groq's LPU technology into existing GPUs, or will it develop a hybrid LPU+GPU solution?