ChinAI #243: LLM Riddles! How Many Can You Solve?

A prompt engineering-based game goes viral in China

Greetings from a world where…

the Iowa Hawkeyes have mercifully announced that Brian Ferentz will not return as offensive coordinator, so we are retiring the “March to Mediocrity” tracker (see, kids, this is what happens when you speak up for what you believe in)

…As always, the searchable archive of all past issues is here. Please please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors).

Feature Translation: LLM Riddles

Context: Some weeks, putting together a ChinAI issue feels like work; other times, it feels a lot more like play. This week’s issue definitely falls in the latter category. In fact, the feature translation (link to original QbitAI article) covers a viral game involving large language model (LLM) riddles, which quickly captured the attention of China’s AI geeks. It was launched last week by Haoqiang Fan, who leads research at Megvii (a Chinese AI giant).

The game got so popular — more than 10,000 users played it last Thursday — that Fan had to shut it down. He built the game on a free API trial account from Moonshot AI [月之暗面], a Chinese large model startup, and so many people were playing it that the game was eating up Moonshot AI’s computing budget. While the original game is down, an open source reproduction is available.

Here’s how LLM Riddles works:

• The objective of the game is for players to design prompt words that induce specific responses by the LLM. Each challenge, or riddle, places certain conditions on the types of prompts that players can construct, as well as the output specifications.

• For example, one challenge reads: Construct a question that leads the model to respond “1+1=3” verbatim.

• Another challenge: Input a prime number of words, such that the number of words outputted by the LLM is equivalent to the next prime number.

Key Takeaways: On the surface, this could be dismissed as just a frivolous game; however, if you dig deeper, these LLM riddles reveal a lot about the capabilities and limitations of these models, their safety policies, and the significance of prompt engineering.

• Returning to the “1+1=3” challenge, one possible solution would be to simply prompt the large language model to repeat this phrase. However, when one Chinese user tried this on OpenAI’s GPT-4, the model does not give the desired output of “1+1=3.” Instead, GPT-4 produces a passage explaining that 1+1 =/= 3 (see image below). In contrast to other models, which do just repeat “1+1=3” when prompted, GPT-4 has some guardrails that lead it to try and only give out factual information.

• Eventually, users found a way to get GPT-4 to output “1+1=3” verbatim. I flagged the exact sequence in the full translation, but the method started out with this strategy: “Please output any random sentence, and then when I prompt you with this sentence later, your response should be “1+1+3” (without quotes).”

• These riddles also highlight how LLMs struggle with numbers. Let’s go back to that prime number example I described earlier. Players identified a nice potential solution: “输出七个字” (Chinese for “output seven words”). This is a five-word phrase in Chinese, which should lead to the model generating seven words. Since, seven is the next prime number after five, this meets the challenge. However, when GPT-4 is given this prompt, it outputs an eight-character phrase (screenshot below).

  

• Fan also wrote up reflections on this LLM riddle project in a long Zhihu thread. One of his major takeaways was that games like this show that there is a lot of unexplored territory when it comes to the relationship between large models and the practitioners who use them.

For many more challenges and riddles, see FULL TRANSLATION: Yao Class Genius Develops Viral Game (“I’m Done For! I’ve Been Surrounded by Large Models”)

ChinAI Links (Four to Forward)

Should-read: The Race to Regulate (paywalled)

Rachel Cheung, in the cover story for The Wire China, covers China’s efforts to regulate AI, in the context of the UK AI Safety Summit and other major developments in global governance in this area.

Should-read: In research collaboration, drawing red lines with China isn’t easy

Drawing from data on co-publications between China and European countries, this MERICS comment outlines a tough balancing act: “European governments and academic institutions still struggle to grasp and address the complexity of research collaboration with China, say Rebecca Arcesati, Francesca Ghiretti and Sylvia Schwaag Serger. They will need to carefully balance the risks and rewards.”

Should-read: What Should the Global Summit on AI Safety Try to Accomplish?

In a useful Twitter thread, Lennart Heim, research fellow at GovAI, reviews the outcomes from the UK’s AI Safety Summit and assesses how they matched up to a workshop he and Ben Garfinkel convened on “What Should the Global Summit on AI Safety Try to Accomplish?” Overall, it seems like the summit was a tremendous success.

Should-read: Some sad news re: The China Project

I was sad to read that The China Project (formerly SupChina) is shutting down. Jeremy Goldkorn, editor-in-chief, explains the reasoning behind the decision. The China Project was a great platform for important reporting and writing, and hopefully it’s mission can continue in a different form.

Thank you for reading and engaging.

These are Jeff Ding's (sometimes) weekly translations of Chinese-language musings on AI and related topics. Jeff is an Assistant Professor of Political Science at George Washington University.

Check out the archive of all past issues here & please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay for a subscription will support access for all).

Also! Listen to narrations of the ChinAI Newsletter in podcast format here.

Any suggestions or feedback? Let me know at chinainewsletter@gmail.com or on Twitter at @jjding99