Huawei is working on gaining a large part of the AI market in China, which is currently led by Nvidia. It offers local companies the ability to use their AI chips for “inference” tasks.
To “train” LLMs, major AI companies in China depend on GPUs developed by Nvidia. The company stands at 3.4 trillion and is seen as an essential part of improving AI technology.
According to a Financial Times report, Huawei is promoting its new Ascend AI processors for Chinese groups that are considering “inference” as an option. Inference is the function that LLMs use to create prompt responses.
The senior AI researcher of inference acceleration for Huawei’s Zurich lab, Georgios Zacharopoulos, says, “Training is important, but it only occurs a few times.” He said the company can serve more customers by focusing on inference.
Huawei is seen as a strong competitor for Nvidia in China
Nvidia’s prominent rival in the local market is using AI models trained on Nvidia’s products on the Ascend chips. According to employees and customers, Huawei’s direction has fewer technical challenges and can generate high profits.
The company has also provided a tool to fill the gap between the different software used by Nvidia GPUs and Ascend.
The Chinese government is also fully supporting Huawei’s promotions. Officials are asking local tech companies to use Huawei AI chips instead of Nvidia.
One source associated with Nvidia’s operations in China said that the company is also seen internally as the country’s most serious rival since its chip design is more “advanced.”
Unlike US tech giants OpenAI and Google, Chinese companies face the challenge of accessing the best-quality GPUs. Due to US limitations on exports, Nvidia only provides China with lower-quality H20 chips. Nonetheless, these chips remain in high demand in China, and they are still better than the ones manufactured locally.
Huawei chips are not ready for inter-chip connectivity
An analyst at Bernstein’s China Semiconductor, Lin Qingyuan, said that while Huawei chips’ performance is remarkable on a per-chip basis, they still face a challenge when it comes to inter-chip connectivity. He said, “When training a big model, you must break it into smaller tasks. If one chip fails, the software needs to figure out a way for the other chips to take over without delay.”
On the other hand, developers still need convincing from Huawei to transition from Nvidia’s developer-friendly “secret sauce” software, Cuda, which has the power to run vast amounts of data processing. Reflecting on the challenge, an incognito Huawei employee said that the company is expecting the updated Ascend 910C “with improved software that makes it more accessible for developers.”
The company is speeding up its manufacturing capacity
Financial Times also reported SemiAnalysis data, which says in 2024, Nvidia made $12 billion in China, including 1 million sales of H20 chips in the region. This is double the amount made by Huawei’s Ascend 910B.
Dylan Pater, chief analyst at the research firm, said, “Nvidia’s China-specific H20 GPUs make up the majority of AI chips sold in China.” However, the difference is decreasing as Huawei speeds up its manufacturing.
Lin at Bernstein also believes that Chinese companies can reach commercialization faster because of their “attention to inference” more than the US. Therefore, “it is possible to make big efficiency gains even with less powerful chips.”
He said Chinese companies are relying on cheaper inference costs to stay competitive in AI and reduce the cost of running AI applications.
Cryptopolitan Academy: How to Write a Web3 Resume That Lands Interviews - FREE Cheat Sheet