Is it possible that a small chip will shake the hegemony of AI giants? On June 27, Etched released an ASIC chip designed for the Transformer architecture, Sohu, claiming that its performance is 20 times that of the NVIDIA H100.
This news shocked the entire AI community, and everyone is discussing, can this new chip really beat the existing GPU supremacy?
On June 27, Etched officially announced the launch of the Sohu chip.
The release of this chip has set off a frenzy, as Etched claims that Sohu far surpasses Nvidia's B200 GPU in AI large language model inference performance, and the performance is 20 times that of the H100.
It sounds like a fantasy, but Etched seems to have come prepared.
They partnered with TSMC to produce Sohu chips using a 4nm process.
TSMC is the world's top semiconductor manufacturer, which undoubtedly provides a strong endorsement for the production of Sohu chips.
Even more impressively, Etched has secured a sufficient supply of high-bandwidth memory (HBM) and servers to be able to ramp up production quickly.
Early customers have already pre-ordered tens of millions of dollars in hardware, which is a testament to the high level of anticipation the market has for Sohu.
First, let's take a look at the specific performance of the Sohu chip.
Sohu is the world's first ASIC based on the Transformer architecture, which sets it apart.
Etched claims that servers equipped with 8 Sohu chips can process more than 500,000 Llama 70B tokens per second.
What is this concept? To put it simply, it's much more efficient than a server with 8 H100 or B200 GPUs, almost replacing the electric fan in your home with a typhoon.
Sohu's FLOPS utilization is over 90%, while GPU's FLOPS utilization is only about 30%.
Why is Sohu so powerful? Because it focuses on the Transformer architecture, removes the redundant control flow logic, and specializes in AI models.
For example, the GPU is like a Swiss army knife that can do a little bit of everything, but the Sohu is a specialized scalpel that is precise and efficient.
Memory bandwidth is also a big problem, and many chips will encounter memory bandwidth bottlenecks when processing large models.
The Sohu chip makes efficient use of memory bandwidth, and is no longer overstretched for memory bandwidth when processing large models.
It's like you go to the supermarket to do some shopping, and with the high-speed lane, the checkout speed doubles.
When it comes to software, Sohu also has the edge.
Developers only need to write software for the Transformer and do not need to deal with complex control flow logic.
What's more, Etched will open source the software stack for developers to customize.
It's like giving you a master key that can be unlocked at will, greatly simplifying the development process.
On top of that, Sohu chips cost less.
You know, the procurement and operating costs of AI data centers are a big one, and Sohu chips can significantly reduce these costs.
It's like buying double the happiness for half the money.
Nowadays, the improvement of GPU performance mainly depends on the increase in process technology and chip area, but the growth of computing power per unit area is slow.
It's like squeezing toothpaste, it's getting harder and harder to squeeze out more performance.
And the rise of Transformer architecture has changed all that.
The Transformer model market is growing rapidly and becoming mainstream, and the model architecture tends to be stable.
ASIC chips for specific algorithm models have natural advantages in speed and cost, especially after the AI algorithm architecture is stable, ASIC chips will become the key to improving computing power.
It's like a specialized tool is more efficient than a one-size-fits-all tool.
Of course, it is better to use a Swiss army knife, but when it comes to critical moments, it is better to use a scalpel.
Etched is confident in the market outlook.
They predict that within the next five years, AI models will become smarter than humans in standardized tests, and that scale is the key to the future.
The hardware optimization direction will also continue to be optimized for the Transformer architecture, which means that the Transformer architecture will become the dominant AI computing market.
The reason why Etched was able to launch the Sohu chip was actually a big gamble.
As early as when the Transformer architecture was not yet popular, Etched invested in the development of Sohu chips in advance.
The current situation proves that they are right to make a bet.
Sohu is expected to become an important hardware project and lead the development of AI hardware.
With the launch of the Sohu chip, other Transformer "killer" chips will also appear.
But to gain a foothold in the market, they must run faster than Sohu on GPUs, otherwise it will be difficult to succeed.
It's like a race, and you have to run faster to win the market.
Trends across the industry are also changing.
As the evolution of AI algorithms stabilizes, dedicated ASIC chips will be more advantageous.
Major cloud service providers have launched self-developed cloud AI chips, which undoubtedly conforms to this trend.
It is foreseeable that the AI computing market will usher in a new revolution in the future.
The launch of Sohu chips is not only a technological breakthrough, but also a reshuffle of the market structure.
It poses a new challenge to GPU manufacturers such as Nvidia, forcing the entire industry to rethink the direction of future development.
The competition for AI hardware will become more intense, and the ultimate benefit will be the entire industry and the majority of users.
The success story of Sohu chip also shows us a truth: the courage to innovate and dare to challenge can stand out in the fierce market competition.
With the Sohu chip, Etched is on this path of innovation.
In the future, we will wait and see who will lead the development trend of AI hardware.
(Disclaimer) The process and pictures described in the article are from the Internet, and this article aims to advocate positive social energy and no vulgar and other bad guidance. If it involves copyright or character infringement issues, please contact us in time, and we will delete the content as soon as possible! If there is any doubt about the incident, it will be deleted or changed immediately after contact.