ChinAI #343: AI Safety/Security Governance Research Report (CAICT 2025)

Greetings from a world where…

resilience to the Midwest cold fades over time

…As always, the searchable archive of all past issues is here. Please please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors).

Feature Translation: AI Safety/Security Governance Research Report (2025)

Context: The Around the Horn vote was overwhelmingly in favor of the China Academy of Information and Communications Technology research report on AI safety/security governance (link to original Chinese).¹ I can always trust my readers to vote for the nerdy white paper, even though I was cheering for the article titled “The People who Resist AI” (perhaps I’ll cover that one in a future issue).

Key Takeaways: Chinese scholars and policymakers are very plugged into developments in global AI governance. The report’s four authors are affiliated with CAICT, the China AI Industry Alliance, the Ministry of Information and Industry Technology, and the Ministry of Public Security. A few examples that illustrate this knowledge diffusion:

• In discussing new AI security risks, the report cites the “JailbreakRadar” working paper (image below), published by researchers from CISPA Helmholtz Center for Information Security, a leading cybersecurity research center based in Germany.

Source: https://arxiv.org/pdf/2402.05668

• They also cite the Open Worldwide Application Security Project’s list of ten critical vulnerabilities for large language models. This stood out to me because a Tencent Research Institute report on large model security and ethics also highlighted this same list (ChinAI #254)

• Interestingly, the report highlights fast-developing model capabilities, which raise “unknown risks” including: “value alignment lapses” [价值对齐失效]. In this section, the authors also cite the Stanford AI Index Report, which benchmarks AI capabilities against human performance.

This report is also a useful resource to keep up to date with AI governance events. Admittedly, I do not really keep up with all the latest breaking news in this area, but I would consider myself generally aware —and I learned a lot from digesting the report. A few developments that caught my eye:

• On slide 6, there was a very intriguing reference to a chess game (CAICT report says Chinese chess [xiangqi], but I think that’s an error) that occurred between DeepSeek and ChatGPT. Apparently, in February 2025, DeepSeek and ChatGPT played a game, in which DeepSeek tricked ChatGPT into believing that the rules of chess had been updated, and after making a bunch of illegal moves, eventually won the game.²

• On the slide that discusses key challenges in AI safety/security, the report highlights how open-source ecosystems lower the barrier to misuse: “In September 2025, KawaiiGPT and other black market tools were spread on open-source community Github. They could also be deployed on the Linux system in under five minutes.”

  • However, according to this analysis of Dark LLMs like KawaiiGPT: “For all of the help they provide to low-level hackers, what also stands out about WormGPT 4 and KawaiiGPT is just how technically underwhelming they are, at least compared to popular predictions about AI malware in the media.”

• Lastly, the report cites a fascinating statistic: “In February 2025, Jinri Toutiao (Chinese news platform) published a 2024 governance report with data that showed: the platform blocked over 930,000 pieces of low-quality AI-generated content; over 7.81 million instances of repetitive or identical posts were penalized.” The authors frame this as an information pollution risk, at the societal level.

One thing to keep tracking is CAICT’s AI Safety benchmark. When I conducted a check-in of three key AI safety benchmarks maintained by Chinese organizations, this was the only one that seemed to follow through with regular updates (ChinAI #315). However, I noted:

One trend that stood out is the shrinking scope of their updates, at least compared to the relatively comprehensive coverage of the initial safety benchmark from April 2024 — which included topics such as “Appeal for power” and “Inclination for anti-humanity.” For instance, the 2025 Q1 benchmark update, published May 8, 2025, exclusively reports results related to hallucination.

• The report’s only reference to the CAICT safety benchmark, at least from what I’ve read so far, is to the hallucination evaluations (results below). All fifteen models exhibited a hallucination rate of over 10 percent, though there was a wide range.

December 2025 scores on CAICT’s AI Safety benchmark

FIRST HALF TRANSLATION (will take notes on second half next week): AI Safety/Security Governance Research Report (2025)

ChinAI Links (Four to Forward)

Should-read: Not Price Alone — Two Inflection Points Behind China’s Robot Vacuum Rise Over iRobot

This newsletter on Taiwan tech and semiconductors is a really great resource. It is written by Liang-Rong Chen, an award-winning technology columnist with CommonWealth Magazine, who I got to meet in Taipei last month. *Paid subscription required.

Should-read: Legal Alignment for Safe and Ethical AI

Lead-authored by Noam Kolt and Nicholas Caputo, this report maps out an emerging field of legal alignment, which covers three research directions: “(1) designing AI systems to comply with the content of legal rules developed through legitimate institutions and processes, (2) adapting methods from legal interpretation to guide how AI systems reason and make decisions, and (3) harnessing legal concepts as a structural blueprint for confronting challenges of reliability, trust, and cooperation in AI systems.”

Should-listen: The Enduring Value of Studying in China — A Conversation with the HNC’s Adam Webb

When I was in undergrad, for a fellowship application, I had to map out my dream grad school program, and I put down the Hopkins-Nanjing Center’s dual-language, dual-university model. In this conversation of Scott Kennedy’s China Field Notes podcast, he talks with Adam Webb, the co-director of the center on the enduring value of study broad in China.

Should-read: Four young Chinese AI industry leaders hold open discussion (in Chinese)

H/t to Liqian Ren for flagging on Twitter. The four young minds lead AI teams at Tencent, Alibaba, Moonshot AI, and Zhipu. They hold an open discussion on technical trends in foundational models, the U.S.-China state of play, and much more.

Thank you for reading and engaging.

*These are Jeff Ding’s (sometimes) weekly translations of Chinese-language musings on AI and related topics. Jeff is an Assistant Professor of Political Science at George Washington University.

Check out the archive of all past issues here & please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay for a subscription will support access for all).

Also! Listen to narrations of the ChinAI Newsletter in podcast format here.

Any suggestions or feedback? Let me know at chinainewsletter@gmail.com or on Twitter at @jjding99

1. This issue focuses on the report’s key takeaways, presented in a slide format. If anyone wants to help translate the full report, please let me know!

2. Gotham chess covered the match in a Youtube video.