Nvidia's latest AI chip reveals design flaws and delayed shipments

Sealing is the key to the performance of Blackwell B200 and GB200
According to Nvidia's press conference information, the Blackwell B200 chip is defined as the strongest AI acceleration chip currently on the surface, built on TSMC's 4nm process. Inside the chip, two Blackwell GPUs are interconnected to form a dual core design, with a single chip having up to 208 billion transistors.
Advanced technology and packaging make the Blackwell B200 chip a performance monster, with AI performance up to 20PFLOPS, which is 5 times that of the H100 chip and can bring a 30 fold efficiency improvement to LLM (Large Language Model) inference. The performance of the Blackwell GB200 chip is even stronger, with AI performance 7 times that of the H100 and training performance 4 times that of the H100.
It is not difficult to see that due to the failure of Moore's Law, advanced packaging technology has become a key means for Nvidia to improve chip performance. In order to enable two GPUs to be packaged together and function as a single GPU, Nvidia used a high-speed bandwidth interface of 10 TB/s NV-HBI between two Blackwell GPUs, and the entire chip was also packaged with 192GB of high-speed HBM3e memory.
However, insiders revealed that there were some issues with the circuit design between the two Blackwell GPUs, which prevented the B200 chip from being mass-produced as originally planned. The Blackwell package is the first package designed for mass production using TSMC CoWoS-L technology, utilizing RDL interlayers with local silicon interconnects (LSI) and embedded bridge chips. Compared to the previously popular CoWoS-S, CoWoS-L is more complex, but it also brings higher performance benefits.
Google, Meta, and Amazon are most affected
The Blackwell chip was originally scheduled to start mass production in October 2024. If it is postponed to April 2025 due to the delay, it will directly affect Nvidia's quarterly earnings. The three companies that have been most affected on the customer side are Google, Meta, and Amazon.
According to reports, Google, Meta, and Amazon have placed orders worth over $60 billion with Nvidia. According to the estimation of renowned investment bank Morgan Stanley, the connection technologies used in a single Nvidia supercomputer card are NVL36 and NVL72, which means that 36 or 72 Blackwell B200 chips are installed on a computing card. Calculated at a unit price of 30000-40000 US dollars per chip, even if there are 36 chips, the price of the chips alone exceeds 1 million US dollars. If it is 40000 US dollars, it is 1.5 million US dollars. Therefore, the price of a computing card connected with NVL36 will be set at 2 million US dollars, and the price of one connected with NVL72 will be set at 3 million US dollars, which meets Huang Renxun's standard of buying more and getting cheaper.
If Google, Meta, and Amazon all place orders for NVL72 computing cards worth $3 million, then the demand for Blackwell B200 chips will reach up to 1.4 million. If there are continued discounts for large-scale purchases, this quantity will be even greater. The fact is indeed true. According to sources close to Google, Google will invest approximately $10 billion to acquire 400000 Blackwell B200 chips and related server facilities. Therefore, the $60 billion budget may create a demand gap of approximately 2.4 million Blackwell B200 chips.
Meanwhile, Huang Renxun revealed during the release of the Nvidia Blackwell B200 chip that Dell, Microsoft, OpenAI, Oracle, Tesla, and xAI have plans to adopt Blackwell products. If we only look at the Blackwell B200 chip, its delayed release may affect Nvidia's revenue target for this year.
However, currently, the construction of cutting-edge data centers relies almost 100% on computing chips from NVIDIA. NVIDIA's compensation measure is to continue to extend the shipment of Hopper series chips and strive to supply Blackwell B200 chips as much as possible this year. Once this is achieved, NVIDIA may sprint to $200 billion in revenue by 2025 and $47.5 billion by 2024.
epilogue
According to statistics from semiconductor research firm TechInsights, the total global GPU shipments in data centers reached 3.85 million units in 2023, an increase of 44.2% compared to 2.67 million units in 2022. Among them, Nvidia ranks first with a market share of 98%. Based on such dominance, the high demand for Nvidia Blackwell B200 chips is imaginable.
However, the Blackwell B200 chip is a complex chip that uses TSMC's CoWoS-L packaging technology. It is currently facing some production challenges, but we believe that with the capabilities of Nvidia and TSMC, it will be resolved quickly.