Storing thousands of NVIDIA chips for floating profits is over 100 million yuan, making it even more difficult to go on the path of domestically produced large models

Li Peng, a domestic entrepreneur engaged in large model development, has recently experienced firsthand the craziness of the AI chip industry.

In April of this year, Li Peng placed an order for thousands of NVIDIA A800 chips at a price of nearly 80000 yuan per chip. After receiving the goods in mid October, he immediately faced further restrictions from the United States on high-performance AI chips. The price of domestic A800 chips has risen in response, and as of October 20th, their individual price has risen to about 300000 yuan. This means that the batch of A800 chips in Li Peng's hands, with a floating profit of over 100 million yuan.

Behind the skyrocketing prices of NVIDIA chips is the new regulations on AI chip export controls introduced by the US Department of Commerce's Bureau of Industry and Security (BIS).On October 17th Eastern Time, BIS officially announced the requirement to limit performance density as a new standard and provided a 30 day buffer period, based on the limitations imposed on AI chip bandwidth speed in October last year.

Under the influence of the new regulations,Nvidia originally launched a customized version of the A100/H100 chip A800/H800 for the Chinese market, but it will also be completely banned due to exceeding the standard.

In addition, BIS also hopes to use new restrictions on chip performance density to prevent Chinese manufacturers from using Chiplet packaging technology to splice multiple chips, thereby achieving the goal of improving overall chip performance. The 5G chip returned by Huawei on Mate60Pro was once speculated to have achieved process performance comparable to 7nm through chiplet packaging technology.

According to US Commerce Secretary Raymond,The export control regulations for AI chips will be updated at least once a year to try to close the loopholes in China's acquisition of advanced AI chips as much as possible.

As Li Peng witnessed the sharp rise in A800 chip prices, Nvidia's RTX4090 graphics card, released for the gaming market, also embarked on a path of skyrocketing value.

As the most configured and powerful graphics card in the global consumer product industry, the RTX4090 has been banned due to its excessive performance density. Affected by this news, starting from October 18th, RTX4090 graphics cards in domestic e-commerce channels showed a shortage at one point, and the price quickly soared from around 15000 yuan to 30000 to 50000 yuan.

However, under the continued implementation of the "small yard, high fence" policy in the United States, BIS has locked restrictions within a limited range and still allows the export of chips for consumer applications such as laptops, smartphones, and gaming applications. However, for a small number of high-end game chips, licensing notice requirements have been established.

There are still uncertainties regarding whether NVIDIA can successfully apply for an export license for RTX4090 graphics cards.Alphabet sought confirmation from one of NVIDIA's domestic partners, Rainbow Seven, who stated that it is currently unclear whether they can continue to sell RTX4090 graphics card products in the future.

But the market has already given optimistic feedback first. As of October 20th, RTX4090 graphics cards in domestic e-commerce channels rarely show out of stock status, and the price has also dropped to 20000 to 30000 yuan.

Compared to the possible "false alarm" of the ban on RTX4090 graphics cards and the ban on Nvidia A800/H800 chips, it is almost a confirmed fact.

In response to the impact of the US upgrade on China's AI chip regulatory measures, Nvidia mentioned in the announcement that "considering the strong global demand for Nvidia products, it is not expected to have a short-term meaningful impact on financial performance

But in the long run, the absence of the Chinese market will inevitably have an impact on NVIDIA. At the second quarter financial report this year, NVIDIA CEO Huang Renxun stated that if the ban on long-term sales of data center GPUs to China is implemented, NVIDIA's industry will lose the opportunity to compete and lead in one of the world's largest markets.

Nvidia is not the only one facing long-term pain. For a large number of domestic model manufacturers, the supply interruption of Nvidia chips may become the biggest obstacle for them to catch up with ChatGPT and even GPT-4.

A

Although some voices believe that the era of giant artificial intelligence models is coming to an end, in the eyes of OpenAI CEO Ultraman, the scaling laws of model performance are still effective,Increasing the size of the model will continue to improve performance, and Ultraman even proposed the "Moore's Law" of the Big Model era, where chip demand doubles every three to four months.

According to Mustafa Suleiman, co-founder of DeepMind and current CEO of InflectionAI, OpenAI is secretly training the GPT-5, which requires an estimated number of Nvidia chips ranging from 30000 to 50000 H100.

Even though major domestic manufacturers such as Alibaba, Tencent, Baidu, etc. have stated that they had already purchased a batch of chips in advance and deployed them in place before the new regulations were introduced, they are still facing OpenAI, which is still iterating in large model technology,Whether the chip reserves of domestic giants who do not want to fall behind in the big model competition are sufficient to cope with future competition remains a question.

In August, Baidu, Tencent, Alibaba, and Byte were revealed to have placed a cumulative delivery order of $1 billion with Nvidia, purchasing a total of approximately 100000 A800 chips. The value of AI chips delivered next year will reach $4 billion. The approximate order rate for next year is no longer expected, even if 100000 A800 chips are successfully delivered within the year, the shared share of each major factory is only 25000, which is far from enough to support them to keep up with the pace of GPT-5.

And from the 30 day buffer period reserved by BIS for NVIDIA, it also cuts off the illusion of domestic large factories rushing to buy before the ban takes effect.

According to CoreWeave co founder and CTOBrian Venturo, obtaining Nvidia GPUs was relatively easy in the first quarter of this year, but since April, the market has suddenly become extremely tight, and delivery times have already needed to be scheduled for the first or even second quarter of 2024.

The letter list seeks confirmation from Li Peng, and the above-mentioned delivery time has been verified. According to him,After placing an order with NVIDIA, it will take six to seven months to receive the goods.

Even worse, Nvidia will continue to launch a new AI chip every six months under the guidance of Huang's Law. At that time, the gap in hardware configuration between domestic manufacturers who cannot even obtain customized versions and American competitors may widen.

Once NVIDIA chips are completely out of supply, the alternative solution left for major manufacturers may only be to replace domestic GPU manufacturers.

At the recent Q3 financial report press conference, Jiang Tao, Vice President of iFLYTEK, stated that the iFLYTEK Spark model is not affected by the supply outage of NVIDIA chips, and its adopted Huawei Ascension 910B capability has basically achieved benchmarking with NVIDIA A100.

But Wu Wei, a model research and development engineer at Toutou Cloud Factory, told Alphabet that from a single chip perspective, some domestic GPU chips may already be comparable to A800 or even A100, but there is still a significant gap in the parallel computing process of multi card series connection.

Tao Ran, General Manager of Microsoft's Azure Business Unit in Greater China, also holds the view that many manufacturers do not have the ability to achieve multi card connections, but multi card parallel computing directly determines the time and cost required for large model training. "For the same set of models, if training takes one week, the cost is $1 million, but if the bandwidth is too small and ultimately takes two weeks, the cost will be multiplied by two

In the big model track that advocates efficiency, time is sometimes more important than money. Similar cost-effectiveness issues also exist when using self-developed chips from large manufacturers.

In practical applications, according to Wu Wei,Large factories' self-developed chips are more commonly used to perform inference tasks, that is, on large models trained by NVIDIA chips, large factories use self-developed chips to ensure the daily operation of large models.

Even during the process of replacing domestic GPUs, major manufacturers are currently facing the challenge of not being able to use more advanced process chips. BIS has included domestic GPU companies such as Bi Ren and Moore Thread in the physical list, which means that without permission, they cannot import any technology or products containing American companies, nor can they produce advanced chips below 14nm through OEM factories such as TSMC.

Another significant impact that cannot be ignored is that more advanced chips often help reduce the operating costs of large models through continuous technological iterations. Huang Renxun once said, "H100 can reduce the processing cost of large language models by an order of magnitude

Once the training cost of the large model becomes lower, the price for customers to call the API interface of the large model will also become cheaper. At that time, domestic large manufacturers will not only face the reality of outdated chip performance, but also bear higher operating costs for large models than their overseas competitors.

B

Domestic manufacturers are not the party most affected by the NVIDIA chip supply outage.Compared to the initiative of large manufacturers to reserve chips in advance with abundant funds, the shocks that domestic large model entrepreneurs will face will be even more severe. Among them, entrepreneurs who have developed universal large models have been hit harder.

Recently, some startup companies that aspire to become general models have announced financing news: Baichuan Intelligent has announced the completion of a new financing of $300 million; Zhipu AI announced the completion of a cumulative financing of over 2.5 billion yuan. But at present, these financing funds that could have been used to purchase Nvidia chips have lost their usefulness for a while.

Even startups that focus on the large models of vertical R&D industries will also be affected. According to Li Peng, purchasing cloud resources from large factories is theoretically sufficient to meet the demand for industry models with some parameter quantities maintained at the billions level. But for startups who want to achieve the ultimate experience, stable computing power is still an essential condition. Buying cloud resources from a large company is likely to encounter situations where the company's computing power needs expand and the other party is unable to provide timely support.

But compared to large manufacturers, in the process of replacing domestic GPUs, large model startups not only need to pay a higher cost of time, but also may create a larger catch up gap due to their shortcomings in algorithms.

In the opinion of artificial intelligence expert Dr. Ding Lei, compared to taking one month or three months to train a large model well, the greater challenge lies in the fact that some current large model startups "are not so good at training large models... If the training method is wrong, no matter how many GPUs you have, it's useless

In addition to model level entrepreneurship, application level entrepreneurship based on large models may also affect business development due to new sanctions imposed by the US government.

According to media reports, the US government is still considering restricting the rental of cloud services to Chinese AI companies, in order to block these companies from leasing Microsoft Cloud and avoid the export ban on advanced process chips.

C

At a time when a large number of domestic model manufacturers are facing challenges or impacts in their business, Nvidia, as the center of the storm, is also unable to escape losses.

When the US government announced that it would implement new export restrictions on semiconductor technology, further limiting the Chinese market's access to A800 and H800, Huang Renxun said in an interview in July, "If we are deprived of the Chinese market, we have no emergency measures. Because there is no other China in the world, only one China

In 2022, the Chinese market accounted for approximately 40% of NVIDIA's revenue. According to data from the Semiconductor Industry Association of the United States, in 2022, China's semiconductor procurement volume was $180 billion, accounting for more than one-third of the global semiconductor procurement total of $555.9 billion, making it the largest global single market.

In order to recover the losses as much as possible, Huang Renxun once tried to lobby the US government to change its strategy.In July of this year, Huang Renxun, along with Intel CEO Kissinger and Qualcomm CEO Amon, held talks with US government officials in Washington. These chip giants hope that the US government should further evaluate the impact of "restrictions on semiconductor exports to China", believing that export controls will undermine the US leadership in the semiconductor industry, and hope to stop implementing new restrictions on China.

Huang Renxun

Huang RenxunHuang Renxun

Although this lobbying campaign to some extent delayed the introduction of new regulations on chip exports from the US government to China, it ultimately failed to completely change the US government's mind.

30Huang Renxun

Considering that after the US government issued an export ban in October last year, Nvidia quickly adjusted its chip bandwidth speed and promptly released a customized version of the A800 chip that met the ban. Ivana Delevska, founder of SpearInvest, believes that "over time, there will be some similar solution, rather than completely prohibiting themNvidia may once again fine-tune its chips to meet the new ban requirements, in order to mitigate the revenue decline and other impacts caused by this AI chip ban.

Affected by the new regulations, Nvidia's stock price has plummeted from $446 on October 17th to $414 on October 20th, with a total market value plummeting by $76.9 billion (approximately 560 billion yuan) within three days.

A800/H800AIHuang Renxun