Welcome to the ChinAI Newsletter!
These are Jeff Ding's (sometimes) weekly translations of writings on AI policy and strategy from Chinese thinkers. I'll also include general links to all things at the intersection of China and AI. Please share the subscription link if you think this stuff is cool. Here's an archive of all past issues. *Subscribers are welcome to share excerpts from these translations as long as my original translation is cited.
I'm a grad student at the University of Oxford where I'm based at the Center for the Governance of AI, Future of Humanity Institute.
ChinAI – Let’s Debate!
***Back from a hiatus with a jam-packed issue. Welcome to all the new subscribers – we’re up to 4000+. Introducing a new segment of ChinAI where I’ll make a case for or against some proposition. High school policy debate was the single most formative activity in my life, and I truly believe testing one’s arguments in direct opposition to others can result in more effective policy. I’ll always try to keep the arguments directed at an idea rather than the person behind the idea. That’s why in what follows I’m blunt and sarcastic at times toward Financial Times reporting on Western-Chinese AI research partnerships not because I don’t respect its reporting (FT is second only to the New York Times in terms of China coverage in my opinion) but rather because I think “iron sharpens iron” should apply to argumentation.
**I think there should be public debates on issues like these in policy debate format (with cross-examination, multiple rebuttals). You can watch the tape on me back in the glory days of my policy debate career (“Google me, Chuck”) – and I’m sure there are countless people smarter than me who would be up for more structured debates about these issues.
*We previewed a prototype of this segment in the previous newsletter, where I laid out five lessons learned from a year of translating Chinese writings on AI-related issues. Other outlets are welcome to excerpt from these rantings and ramblings as MIT tech review did or contact me about re-publishing them with light editing as technode did (I also did a podcast with technode on the subject).
This week’s proposition: The recent Financial Times (FT) pieces on Microsoft working with a Chinese military university on AI and Western AI researchers partnering with Chinese surveillance firms — which have led to Republican senators Cruz and Rubio publicly calling out Microsoft’s China work — were deeply flawed. Specifically, they:
1) contained uninformed reporting on technical details of the co-authored papers
2) overhyped the relevance and significance of these collaborations with respect to their potential to aid China’s military and surveillance efforts, and
3) failed to provide appropriate context to prevent the misuse of this reporting by others who want to question all collaborations with Chinese counterparts and promote the meme of a US-China Cold War.
Points 1 and 2: Uniformed and overhyped reporting
The first FT piece highlighted three papers co-written by academics at Microsoft Research Asia and researchers with affiliations to China’s National University of Defense Technology. As of May 5, nearly a month after the article was first published, the article did not even link to the actual papers themselves, so we have no way to even properly judge their significance and relevance for surveillance and censorship (Note: I sent a heads-up about this post to Madhumita, Yuan, and Christian, so links to articles may be edited into the article now; I’ll provide rebuttals from Madhumita from our exchange at the end of this post so as to hopefully give fair representation to FT’s views but welcome further open exchanges). It’s a seemingly minor thing to point out about the piece, but the three papers are the central component of piece so there’s no excuse for not even linking to them in the original article. This is reflective of possible haste and lack of attention to detail in the publication process.
As Miles Brundage, a research scientist on OpenAI’s policy team, who has probably read more AI articles on arxiv than anyone, wrote on Twitter, “The recent FT story on China/AI and the reaction to it tells us more about the state of AI journalism than it does about China/AI: - overhyping the significance of specific papers… Panicked coverage of cherrypicked (public) papers that say things one could have already known about from other public sources, all packaged together into a scary narrative that makes this random batch of papers sound like the most important thing ever.”
The only details we are given about the three papers include that one of them describes a “new AI method to recreate detailed environmental maps by analyzing human faces,” which can have a “variety of vision applications” (cue ominous vibes). From my read of it — and I’m happy to entertain push back as I’m not nearly as qualified as reading AI arxiv articles as others — this paper is focused on increasing photorealism in augmented reality apps like Snapchat, with limited applications for, say, matching a person in real life better to a surrounding environment.
We’re also told that Microsoft’s Beijing lab collaborated on “two other papers with NUDT researchers, including in the area of machine reading.” (a very awkward phrasing that we’ll investigate later). First, let’s look at the machine reading comprehension article (which was shared with me by Yuan as, again, these weren’t linked in the original article) which was on building systems that can abstain from answering “unanswerable questions” given a particular passage. For example, if given a passage that gave background on Normandy, a region in France, previous models would respond to the “What is France a region of” (an unanswerable question in the context of the given passage) with “Normandy,” but the correct answer is “no answer.” If you’ve been following ChinAI, specifically issue #45 on China’s NLP landscape, one could easily deduce that the likely application of this research would not be for boosting censorship or military operations (unless one believes that the Communist Party and the PLA are very concerned with making sure not to answer unanswerable questions) but rather for something like, I don’t know, maybe say, improving Microsoft’s Xiaoice, its hugely popular chatbot and one of its crown jewel AI-based products? Cue next week’s FT headline: BREAKING NEWS THAT YOU HAVE TO CLICK BELIEVE: CHINESE MILITARY UNIVERSITY AIDS AND ABETS US TECH GIANT
Now, let’s circle back to that awkward phrasing. Curiously, the FT piece doesn’t ever specify what that third paper is about. It’s on the subject of how facial features, dress, and voice collectively affect the human sense of beauty. Um. Excuse me, as I need to brush up on how a better sense of human beauty is the next key revolution in military affairs and surveillance.
At least in a FT follow-up piece on Western AI researchers partnering with Chinese surveillance firms, there are links to the articles. These include ones on 3D scene reconstruction for robots and machine comprehension. The former is about reconstruction models of large indoor environments, so I am not sure how FT concludes this has “applications in unmanned drones, and autonomous underwater vehicles” (emphasis mine). The latter is about reducing the time costs by training more compact neural language models on mobile devices – this could enable model training on decentralized data, which may actually undercut surveillance/censorship applications, in contrast to what FT cites experts as saying. Also, from my read of things, none of the corpuses used even contain Chinese-language data.
Please, please, please: before writing these pieces, talk to people who regularly read and can understand AI articles on arxiv (not me please). Jack Clark, policy director for OpenAI, provides an essential weekly ImportAI newsletter as an invaluable service toward this end – he highlights what he views as significant AI research and lists affiliations for pretty much everything he covers, giving a sense of how multi-disciplinary and multi-country AI research is. Do the work to find people like Miles and Jack and get their opinions, in order to make informed, specific claims about the connection between the technical details of the paper to the dual-use applications.
Tech journalism in these times is a damn hard job and a lot of faithful public servants are doing their best. In one sense, the issue here was FT found people who could speak to the technical side of things but not the China side of things. They also found people who could speak to the China side of things but not the technical side. They didn’t find anyone who could speak to both. Those people who can are, unfortunately, few in number. I don’t count myself among this camp but there are good folks (including many cited in these articles) working hard to get there, and I think in some newsletters we get close.
Also, maybe read the articles closely enough to get to the end. If they did, the FT reporters would have found some interesting tidbits in the “acknowledgements” sections such as the fact that one piece, Fu et al., was sponsored by IARPA (an organization within the Office of the Director of National Intelligence) and another, Shi et al., was supported by the National Science Foundation. Either these bodies deemed the research appropriate or the review process was not stringent enough. Did anyone look closely enough to ask?
Point 3: Failure to Provide Appropriate Context to Prevent Misuse of Reporting
This is not to say that the piece does not raise valid and important concerns. The Fu et al. paper on person re-ID involving a researcher at IBM and coauthors at CloudWalk is an example of concerning collaboration that could enable the Chinese government’s adoption of high-tech surveillance, especially in targeting and persecuting Uighurs in Xinjiang. And the commentators in both pieces are right to push for more oversight and due diligence when it comes to academic collaborations linked to human rights abuses.
But it’s also important not to swing too far in the other direction and problematize all academic collaborations. In my opinion, FT has a duty to understand the context on the issues their reporting concerns. There is a climate of hostility against international researchers in the US now, with many university professors being warned to stay away from international students, and this goes from the President on down. Trump himself has noted of Chinese international students in the U.S., “almost every student that comes over to this country is a spy.” FT’s reporting failed to provide the context to guard against how others have used it to problematize all US-China academic collaboration and push forward the dangerous meme of a US-China Cold War.
The FT piece raises several important issues facing a world that has seen significant increases in international research collaborations but one that is still separated into nation-states. One issue is what level of engagement, if any, should Western researchers have with researchers at military institutes of a strategic rival. Microsoft makes the point that everything their researchers did was transparent and in compliance with US and local laws. I think many would be surprised to know that the U.S. Army conducts international S&T collaborations through its international technology centers in Tokyo, Chile, and the UK. A subtly different issue is what level of engagement, if any, should Western researchers have with researchers at research institutions that receive Chinese-military funding. Does every researcher at Johns Hopkins University, which receives substantial funding from the DoD, become a member of “US military-funded academia?” FT calls out Microsoft Research’s “long-running links to Chinese military-funded academia” but again doesn’t link to where they are getting this evidence from (Who is doing the editing for these pieces? Am I just misunderstanding journalistic practice? Isn’t there an obligation to link to external sources and/or quote experts as evidence for claims like these?)
A third issue is research collaborations pertaining to facial recognition and surveillance. In one of the FT pieces Elsa Kania, a fellow at the think-tank Center for a New American Security said, “When research pertains to facial recognition or person re-identification, considering the known abuse of these technologies by the Chinese government, no [organisation] that values ethics or human rights should collaborate with Chinese counterparts, let alone those that are known to profit from marketing surveillance capabilities to China’s party-state.” I disagree with Elsa on one aspect of this point. I think there’s some space for collaboration with Chinese counterparts in research that pertains to facial recognition, but it’s a topic worthwhile for debate. For instance, does that mean international collaboration on research that pertains to making “pay-with-face” mobile repayment systems more secure (the subject of this week’s feature translation) is also off-limits?
FT didn’t address the subtleties of any of these issues. Writing things in a way to score political points by citing Senators and naming and shaming people will not convince AI researchers to more strictly scrutinize collaborations nor help with the situation in Xinjiang. FT failed to get anyone to present opposing arguments, of which there are many. Here’s a brief sampling: a) allowing collaborations on emerging technologies with Chinese military-affiliated folks may actually increase US net assessment capabilities to track Chinese progress in these domains; b) stricter regulations could send a broader chilling effect that cuts against the openness of the US academic enterprise and especially the unique openness ethos of the ML research community, one of the key driving forces of our technological dominance; c) it is really really hard to prevent AI advances from diffusing – so we should focus our efforts (as the consensus has been with respect to export controls) on “running faster” -- none of the info contained in any of these articles was groundbreaking and could have easily been found in any number of other articles.
These were important pieces that, understandably and predictably, caused waves. There was a version of this story that a) limited over-hyping of specific papers by better technical understanding, b) provided more context about how to judge the relevance and significance of these papers to potential applications, and c) still emphasized lack of oversight in academic collaborations with implications for human rights abuses. In this case, it’s unfortunate the FT failed to take the time to tell that version.
I reached out to Madhumita, who led FT’s reporting on the pieces, with excerpts of an incomplete draft of this section. She was kind and responsive enough to provide rebuttals to some of my points and give me permission to cite them. They are linked here in this Google doc.
ChinAI Translations of the Week
1. Fascinating longform article about how someone lost all their money because a roommate scanned their face while they were sleeping and transferred all the money out of their mobile payment system. Goes into the vulnerabilities of the “pay by face” functions of Alipay and Wechat pay
READ FULL TRANSLATION: How safe do you think it is to "pay by face" on mobile phones? We measured an accuracy rate of less than 70% | Exclusive In-Depth Survey
*Too much writing this week, so we’ll come back next week to dive deeper into this case as well as finish out the second part of the translation.
2. Speaking of NUDT, how does one of the researchers connect the recent Chinese sci-fi blockbuster “Wandering Earth” to the need for early-warning systems of AI rebellions as well as calling out the loss of control of military drones by the US military on the Iraqi battlefield?
READ FULL TRANSLATION: In “The Wandering Earth” Do We Encounter a Defection by MOSS? Artificial intelligence is advancing rapidly, intelligent command and control systems need early warning of rebellion
Phew, don’t worry – my neck doesn’t usually feel good enough for this much typing each week. Here’s this week’s ChinAI links:
I really think the world’s collective understanding of China’s AI development would improve so much if we looked outside DC and Silicon Valley for China expertise. While these places attract really talented and knowledgeable individuals and benefit from network effects, they are also bubbles vulnerable to groupthink. Two organizations come to mind but there are countless others. MERICs in Berlin, which recently put out a great paper on how China’s Digital Rise will affect Europe; and MacroPolo in Chicago, which recently published a data-packed piece on why American should open, rather than close, its doors to Chinese AI talent.
Speaking of orgs based outside of DC and SV, my home base GovAI recently published two technical reports on AI governance:
1. Stable Agreements in Turbulent Times: A Legal Toolkit for Constrained Temporal Decision Transmission (2019) - Cullen O’Keefe – on crafting long-term agreements, ex ante, in the face of radical changes from Advanced AI.
2. Standards for AI Governance: International Standards to Enable Global Coordination in AI Research & Development (2019) - Peter Cihon - The case for further engagement in the development of international standards for AI R&D are detailed in this report. It explains the global policy benefits of AI standards, outlines the current landscape for AI standards around the world, and offers a series of recommendations to researchers, AI developers, and other AI organizations. Peter’s piece is especially relevant in light of the US government’s recent call for comments on federal involvement in AI standards-related activities.
FT reports that Chinese facial recognition unicorn SenseTime has sold out of a joint venture with Xinjiang security company Leon – marking “the first time a major Chinese technology has opted out of operations in the region.” ChinAI first flagged this partnership in a previous issue in September.
New York Times on how China’s targeting of Uighurs is not limited to Xinjiang; facial recognition technology is used to identify Uighurs in eastern cities like Hangzhou and Wenzhou and across the coastal province of Fujian.
Thank you for reading and engaging.
Shout out to everyone who is commenting on the translations - idea is to build up a community of people interested in this stuff. You can contact me at jeffrey.ding@magd.ox.ac.uk or on Twitter at @jjding99
