Etched AI makes the biggest bet in AI with its Sohu chip

Etched AI is making waves in the technology sector with its ambitious project to create an AI chip that runs only one type of model, the Sohu chip.

This innovative approach is embodied in their Sohu chip, which is specifically designed for transformers, a dominant model in generative AI. The founders, Gavin Uberti and Chris Zhu, along with their team, are betting on a focused and streamlined design to push the limits of AI chip performance.

Let us introduce you to Etched AI’s journey and the unique aspects of their Sohu chip.

Get to know Etched AI

Etched AI’s inception is rooted in the vision of two Harvard dropouts, Gavin Uberti and Chris Zhu, who saw an opportunity to revolutionize AI hardware. Together with Robert Wachen and former Cypress Semiconductor CTO Mark Ross, they founded Etched AI in 2022.

Their goal was to address a specific need in the AI industry: creating a chip that excels at running transformer models.

Unlike many AI chips that are designed for general purposes, the Sohu chip is an application-specific integrated circuit (ASIC) focused solely on transformers.

Sohu is designed specifically for transformers, a dominant model in generative AI, unlike general-purpose AI chips (Image credit)

The significance of transformers in the AI landscape cannot be overstated. Since their introduction by Google researchers in 2017, transformers have become the backbone of numerous generative AI models. These include OpenAI’s video-generating model Sora, text generators like Anthropic’s Claude and Google’s Gemini, and art generators such as Stable Diffusion.

Recognizing the transformative power of transformers, Etched AI decided to concentrate all their efforts on optimizing their chip for this particular model.

What sets Sohu chip apart?

The Sohu chip is a testament to Etched AI’s commitment to specialization. Built using TSMC’s 4nm process, Sohu promises to deliver exceptional inferencing performance while consuming less energy compared to traditional GPUs.

According to Uberti’s talk with The Verge, Sohu is significantly faster and more cost-effective than Nvidia’s next-generation Blackwell GB200 GPUs when it comes to running text, image, and video transformers. He claims that one Sohu server can replace 160 H100 GPUs, positioning Sohu as a highly efficient and environmentally friendly solution.

Etched AI claims one Sohu server can replace 160 H100 GPUs, offering a more efficient and eco-friendly solution (Image credit)

The key to Sohu’s performance lies in its streamlined hardware and software pipeline. By focusing exclusively on transformers, the design team was able to eliminate unnecessary components and reduce software overhead. This focus allows Sohu chip to operate with greater efficiency and speed, offering a specialized solution in a market dominated by general-purpose GPUs. This approach also addresses the AI’s growing power consumption and environmental impact concerns associated with running large-scale AI models.

By offering a chip that is both faster and more energy-efficient, Etched AI aims to mitigate some of these environmental challenges, presenting Sohu as a viable alternative for companies looking to reduce their carbon footprint.

Market position is the obstacle

Etched AI’s journey is taking place in a highly competitive market, where the dominance of companies like Nvidia is formidable. Nvidia controls a significant portion of the AI chip market, with major cloud providers like Meta and Microsoft heavily investing in their GPUs. However, the reliance on a single supplier has created a desire among generative AI vendors and venture capitalists to seek alternative solutions.

Nvidia’s dominance in the AI chip market poses a challenge to Etched AI (Image credit)

Etched AI stands out among many startups by focusing on a single type of AI model. While other companies like Meta and Amazon have developed their inferencing chips, Etched AI’s unique selling proposition is its specialization in transformers. This niche focus has attracted significant investment, with Etched recently closing a $120 million Series A funding round, bringing their total raised to $125.36 million.

High-profile investors including:

• Peter Thiel
• GitHub CEO Thomas Dohmke
• Quora co-founder Charlie Cheever

have shown confidence in Etched AI’s potential.

Despite the challenges, including the possibility of new competitors and shifts in AI model preferences, Etched AI is optimistic about their future. They are preparing to launch the Sohu Developer Cloud, an online platform allowing customers to preview Sohu’s capabilities, which is expected to drive further interest and sales.

The success of Etched AI will depend on its ability to scale production and meet the demand for efficient and specialized AI chips. Uberti believes that the future of AI technology hinges on the availability of faster and more efficient infrastructure. His bold claims about Sohu’s capabilities and the company’s aggressive market strategy underscore Etched AI’s commitment to shaping the future of AI hardware.

Featured image credit: Etched AI