Greetings from a world where… policy is policy and businesses is business [政策是政策，生意是生意] …***We’ve hit a bit of a lull in paid subscriptions lately, so please consider subscribing here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors). As always, the searchable archive of all past issues is
I'm trying to get a summary view of China's competitiveness in AI. Part of that is their access to compute (I understand that's not the full story), and how effective the US export restrictions have been. Wonder if you know of a high level look at this?
It seems to me China's a clear second but still has enough resources to produce GPT-4 level models. ByteDance's has 100k high-end chips seems like a clear 2nd to the compute available to leading US companies, but still enough to produce, say, a GPT-4 equivalent, trained on 25k A100s over 90 days (https://klu.ai/blog/gpt-4-llm). Compared to the half a million chips NVIDIA sold in Q3 2023, it seems to me the US has a strong advantage though far from complete dominance.
US companies seem to be receiving around 1x order of magnitude new chips relative to ByteDance. If there's two OOMs more compute required for GPT-4 vs. GPT-3, and China can keep up that supply ratio, it seems like China might only be half a generation behind the US companies?
I'm trying to get a summary view of China's competitiveness in AI. Part of that is their access to compute (I understand that's not the full story), and how effective the US export restrictions have been. Wonder if you know of a high level look at this?
It seems to me China's a clear second but still has enough resources to produce GPT-4 level models. ByteDance's has 100k high-end chips seems like a clear 2nd to the compute available to leading US companies, but still enough to produce, say, a GPT-4 equivalent, trained on 25k A100s over 90 days (https://klu.ai/blog/gpt-4-llm). Compared to the half a million chips NVIDIA sold in Q3 2023, it seems to me the US has a strong advantage though far from complete dominance.
US companies seem to be receiving around 1x order of magnitude new chips relative to ByteDance. If there's two OOMs more compute required for GPT-4 vs. GPT-3, and China can keep up that supply ratio, it seems like China might only be half a generation behind the US companies?
Interesting!