ChinAI #236: The LLM Implementation Gap
Baidu's plan to bridge the implementation gap in large models
Greetings from a world where…
for the rest of the college football season, this status update will be devoted to tracking the Iowa Hawkeye offense’s march to mediocrity
…As always, the searchable archive of all past issues is here. Please please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors).
Feature Translation: Who can get Chinese companies to use large models first
Context: Earlier this month, the Chinese government approved large language models (LLMs) from eight labs to be released to the general public. The most impressive and popular was Baidu’s Ernie Bot, which responded to more than 33 million questions within 24 hours of its release. This week, however, we focus on business-end users of large models. The central motivating question, posed by a Leiphone article: Chinese labs have released over 100 large models AND “large models are deemed a new generation of infrastructure, but on the industrial side, many companies do not really use large models.” What gives?
Key Takeaways: Why do many businesses fail to adopt LLMs? In other words, why is there an LLM implementation gap?
Here’s the view of Wang Feng, who leads a company that specializes in consumer content and marketing services. On his firm’s efforts to fine-tune an existing LLM: “But in the process, they found that they didn’t know which model to use: There are currently too many large models published in China. If they want to verify all the models and compare the effects of the models one by one, then the labor cost will be very high.”
Some good details on how corporate customers have become more discerning when choosing between LLMs: “It used to be that when corporate customers were trying to comprehend the basic capabilities of large models for a large model producer, they just looked at the rankings. Now when they choose a large model, they have to optimize it based on their own scenarios and data, and they will consider many things, such as the performance of the model, the efficiency of development, and usage costs.”
The article states: “There is currently a high wall between domestic large models and the industry…large models trained based on open data sets are not good at professional knowledge, and enterprise users who have a grasp on industry data cannot participate in the construction of large models.”
To address this implementation gap, Baidu has developed a Qianfan large model platform, which aims to help businesses quickly build application-specific large models
To give businesses options, the Qianfan platform has access to 42 mainstream LLMs (from both Chinese and international labs), including Meta’s open source model Llama 2.
Qianfan also enhances these models in a few ways, by improving their performance on Chinese-language tasks, reducing training and inference costs, and improving their contextual Q&A and reasoning skills.
Crucially, they also make another enhancement in terms of “content security” to ensure that the output of third-party large models and their own Ernie models meet China’s censorship requirements.
If you can’t tell by now, this Leiphone article seems pretty bullish that “Baidu has obvious advantages in the competition among infrastructure operators in the era of large models.” At times, in my opinion, the language leaned too far into PR-messaging.
Still, the article raises an interesting comparison: “Every company that has the ability to independently research large models wants to be an infrastructure operator in the era of large models. After a few months, whether large models are really the smart operating system of the AI era can only be determined by industry movements. When it comes to societal infrastructure such as water and power grids, no matter how many existing industry players there are, only a few will succeed in the end.”
One advantage that Baidu holds in this competition is that it can integrate its LLM services with its cloud offerings, which allows it to help companies make “AI-native applications.”
For more details, see FULL TRANSLATION: Midfield contest between large models: who can get companies to use large models first
ChinAI Links (Four to Forward)
Must-read: Dead Reckoning
One of my other research interests is the effect of technology on military accidents. In Atavist, Robert Kolker provides a longform accounting of the largest peacetime disaster in American naval history, which resulted in the death of 23 sailors. In the investigation, one point of contention was the degree to which the bilateral radio compass technology (a precursor to radar) was to blame. H/t to Joelle Brown for sharing.
Should-listen: The AI ‘Race’: China vs. the US with Jeffrey Ding and Karen Hao
Thanks for the Centre for Humane Technology team for having me on to talk about some of my recent research, including the diffusion deficit paper and the co-authored report on China’s large language models. For me, most valuable part of this conversation was listening to Karen discuss her work covering China tech — one of the best on the beat.
Should-apply: “Craftsmen” in the age of technology, genius programmers change the world (in Chinese)
For AItechtalk, Yangxia Li reports on China’s first programmer-centered reality TV show! The program tracks 16 contestants as they prepare for a programming competition in the Ant Technology Exploration Conference (ATEC).
Should-apply: Assistant Professor of Political Science and International Affairs at GW
My department is hiring! Here’s the position summary: The George Washington University’s Elliott School of International Affairs and the Department of Political Science invite applications for a tenure track position in the field of Political Science with a specialization in International Security, beginning in Fall semester 2024. We look to hire a scholar at the Assistant Professor level who will teach and produce scholarship in areas such as (but not limited to) the causes, conduct, control, and termination of armed conflict between or within states; deterrence and compellence; nuclear weapons; alliances and security institutions; grand strategy; cybersecurity; and military intervention—as well as international relations theories that are relevant to assessing these issue areas.
One cool feature of this position is the affiliation with the Elliott School’s Institute for Security and Conflict Studies, a great community to share work-in-progress and hear from external speakers.
Thank you for reading and engaging.
These are Jeff Ding's (sometimes) weekly translations of Chinese-language musings on AI and related topics. Jeff is an Assistant Professor of Political Science at George Washington University.
Check out the archive of all past issues here & please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay for a subscription will support access for all).
Any suggestions or feedback? Let me know at chinainewsletter@gmail.com or on Twitter at @jjding99