DeepSeek v3, a Chinese AI model, outperforms GPT-4o and Claude 3.5 while costing only $5.5 million to train, 15 to 57 times cheaper than competitors. U.S. chip export controls forced Chinese labs to innovate more efficient training methods, which they then open-sourced.
Chapters
Have US sanctions on China led to more efficient LLMs? Let me explain. Deepseek, which is one of China's frontier LLM labs, have just released DeepSeek v3, an open source 671 billion parameter mixture of experts model that outperforms GPT4O, Claude 3.5 and Llama 3.1 across many benchmarks. The kicker though is that it only costs $5.5 million to train. In contrast, the GPT, Claude and Llama family of models are estimated to cost 15 to 57 times more. Experts point to US sanctions on China's chip industry as the primary reason for this newfound efficiency. You see, over the past three years the Biden administration has imposed export and investment controls over the Chinese chips industry. This forced Deepseek to find efficiencies in its training process leading to state of the art training performance. This is a prime example of how resource constraint can breed innovation and ingenuity. And since the paper is open source, other LLM providers can also learn about these training techniques. What do you think? Let us know in the comments.