Why OpenAI is Looking Beyond Nvidia for AI Chip Solutions

TLDR

• OpenAI has been searching for alternatives to Nvidia chips since last year, targeting solutions for approximately 10% of its inference computing requirements
• A planned $100 billion Nvidia investment in OpenAI has been delayed for months beyond its expected closure timeline
• The ChatGPT developer has signed chip supply agreements with AMD, Broadcom, and Cerebras Systems to diversify its hardware sources
• Nvidia purchased Groq’s technology for $20 billion and recruited its chip design team to enhance inference capabilities
• Sam Altman and Jensen Huang both publicly minimized tensions between the two AI industry leaders

OpenAI has been pursuing alternatives to certain Nvidia chips for over a year. The search focuses on hardware better suited for inference operations, where AI models generate responses to user requests.

The company requires faster processing for specific use cases. Software development and inter-AI communication are among the applications needing improved performance.

OpenAI plans to source alternative chips for roughly 10% of its future inference requirements. Multiple sources confirmed the company’s concerns about Nvidia’s current hardware speeds for particular tasks.

Investment Agreement Faces Delays

Nvidia revealed plans last September to invest up to $100 billion in OpenAI. The agreement was projected to finalize within weeks but remains incomplete months later.

OpenAI’s evolving product strategy has modified its computing requirements. These changes have extended the negotiation timeline with Nvidia.

The company has secured chip agreements with AMD, Broadcom, and Cerebras Systems during this period. These vendors offer hardware designed to compete with Nvidia’s products.

Performance issues surfaced in OpenAI’s Codex code generation tool. Team members linked some of Codex’s limitations to Nvidia’s GPU architecture.

Sam Altman stated on January 30 that coding customers prioritize speed. He confirmed OpenAI would address this through its Cerebras partnership.

Hardware Specifications

OpenAI has targeted companies producing chips with substantial SRAM memory. This memory type is integrated directly into the chip’s silicon.

The design provides speed benefits for chatbots handling millions of user interactions. Inference operations demand more memory than training because chips retrieve data more frequently.

Nvidia and AMD GPUs use external memory configurations. This approach increases processing delays and reduces chatbot response times.

Competitors like Anthropic’s Claude and Google’s Gemini employ different technology. They utilize Google’s tensor processing units optimized for inference computations.

OpenAI explored partnerships with Cerebras and Groq for enhanced inference chips. Nvidia’s $20 billion licensing agreement with Groq terminated OpenAI’s discussions with that company.

Tech Licensing and Acquisitions

Nvidia recruited Groq’s chip design staff alongside the licensing deal. Groq had attracted investor interest at a $14 billion valuation while negotiating with OpenAI.

Nvidia issued a statement saying customers select its chips for inference performance and cost efficiency. The company described Groq’s technology as complementary to its roadmap.

An OpenAI representative confirmed Nvidia powers most of the company’s inference infrastructure. The statement emphasized Nvidia’s performance-per-dollar leadership for inference operations.

The post Why OpenAI is Looking Beyond Nvidia for AI Chip Solutions appeared first on Blockonomi.