I'm trying to get a summary view of China's competitiveness in AI. Part of that is their access to compute (I understand that's not the full story), and how effective the US export restrictions have been. Wonder if you know of a high level look at this?
It seems to me China's a clear second but still has enough resources to produce GPT-4 level models. ByteDance's has 100k high-end chips seems like a clear 2nd to the compute available to leading US companies, but still enough to produce, say, a GPT-4 equivalent, trained on 25k A100s over 90 days (https://klu.ai/blog/gpt-4-llm). Compared to the half a million chips NVIDIA sold in Q3 2023, it seems to me the US has a strong advantage though far from complete dominance.
US companies seem to be receiving around 1x order of magnitude new chips relative to ByteDance. If there's two OOMs more compute required for GPT-4 vs. GPT-3, and China can keep up that supply ratio, it seems like China might only be half a generation behind the US companies?
I'm trying to get a summary view of China's competitiveness in AI. Part of that is their access to compute (I understand that's not the full story), and how effective the US export restrictions have been. Wonder if you know of a high level look at this?
It seems to me China's a clear second but still has enough resources to produce GPT-4 level models. ByteDance's has 100k high-end chips seems like a clear 2nd to the compute available to leading US companies, but still enough to produce, say, a GPT-4 equivalent, trained on 25k A100s over 90 days (https://klu.ai/blog/gpt-4-llm). Compared to the half a million chips NVIDIA sold in Q3 2023, it seems to me the US has a strong advantage though far from complete dominance.
US companies seem to be receiving around 1x order of magnitude new chips relative to ByteDance. If there's two OOMs more compute required for GPT-4 vs. GPT-3, and China can keep up that supply ratio, it seems like China might only be half a generation behind the US companies?
Interesting!