ChinAI #310: New-type AI Storage Research Report
Greetings from a world where…
we all drag our favorite newsletters from the Gmail promotions tab into the primary inbox, right?
…As always, the searchable archive of all past issues is here. Please please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay support access for all AND compensation for awesome ChinAI contributors).
Feature Translation: New-type AI Storage Research Report
Context: This newsletter used to be built on white papers and research reports. It’s been a year and two months since the last time we really sunk our teeth into one of these (ChinAI #254: Tencent Research Institute’s large model security and ethics research report), so we’re long overdue. This week’s feature translation is of the China Academy of Information and Communications Technology’s research report on “new-type AI storage” (link to original Chinese pdf). It’s jointly authored with the China AI Industry Alliance.
Key Takeaways: What is new-type AI storage [新型人工智能存储]?
New AI storage refers to data storage systems optimized for AI applications and services, with a focus on 1) more efficient training of large models and 2) faster and more accurate outputs in the application inference stage.
CAICT depicts AI storage as the very base layer of the AI infrastructure stack (Figure 1). AI storage includes document storage, big data storage, object storage, and knowledge base storage. Here are the other layers, from top to bottom: industries row includes finance, health, etc.; development platforms (model layer) lists iFlyTek SparkDesk, Zhipu AI’s ChatGLM, etc.; framework level lists PyTorch, TensorFlow, and (Huawei’s) MindSpore; resource layer lists kubernetes, docker, and DCS; compute row includes Ascend, NVIDIA, and AMD; the second-to-last row is networks for data centers.
As the datasets being used to train AI models reach the petabyte level and inference poses data management demands, CAICT’s report on new-type AI storage takes us back to an old question: Is it still all about the data?
Here’s how the research report interpreted Grok3’s improvements over Grok2: “In February 2025, Grok3 was released, and its multimodal capabilities attracted global attention. The amount of data used by Grok 3 increased by 3 times compared with Grok2, and the ability of large models was doubled by increasing the amount of data rather than model parameters.”
It’s not just about the quantity and quality of data used to train foundation models. Industry-specific large models also must integrate their private domain data into foundational models. The article states, “According to IDC's analysis and forecast, in the future, 95% of large and medium-sized enterprises will build their own industry-specific large models based on proprietary data, such as risk control data of banks, autonomous driving data of car companies, and medical PACS (image) data.”
In fact, this CAICT report goes even further. Among the triad of AI drivers (data, computing power, and algorithms), the last two are gradually converging. Most companies use the same GPUs, and everyone relies on the transformer infrastructure and rely on PyTorch and TensorFlow development frameworks. This leaves data as the key variable: “Datasets determine model performance and application scope. Data infrastructure is constantly innovating, aiming to break through input-output walls.” On the left of Figure 2 below, it shows the AI triad; on the right, it shows Meta Llama’s performance on the OpenBookQA benchmark exceeding that of OpenAI’s GPT-3, even though Llama has fewer parameters (the key difference is Llama’s 4.5 TV of training data compared to GPT-3’s 570 GB).
Some of the key technologies for new-type AI storage include long-term memory storage [长记忆存储] and vectorized databases.
Aimed at improving the quality and efficiency of inference, the long-term memory storage paradigm creates a persistent KV Cache (“long memory”) for models to call on. The article cites data that shows “the use of this technology can achieve an inference throughput speed increase of more than 50%, significantly reduce the end-to-end cost of inference, and improve the application scenario experience of large model industry applications when it comes to long prompts.”
I was only able to get through half of the report. Next week, we’ll cover case studies of best practices for AI storage in the medical and manufacturing industries as well as policy recommendations for the development of AI storage.
First part of FULL TRANSLATION: New-type AI Storage Research Report
ChinAI Links (Four to Forward)
Should-read: Why AI Language models choke on too much text
As I was desperately trying to wrap my brain around KV caches and long-term memory storage, this arstechnica piece and the accompanying discussion posts were extremely informative.
Should-read: The Fear Tariff
From Princeton Prof. Rory Truex’s weekly politics column, this post captures so well what many of us at universities have been confronting:
A couple weeks ago, I received an email from a colleague in Europe who had been planning on visiting our campus sometime in the spring. The trip had been in the works for months, but he sent me a note indicating that he would no longer be visiting the United States. The risk of a trip to the States just wasn’t worth it, he said.
The sad irony is that I have written some version of that email myself many times in the past—to colleagues in China. Journalists, scholars and other foreigners face distinct risks when entering China, and many of us that conduct research on Chinese politics no longer feel comfortable going…
In just a few short weeks, the Trump administration has created a security situation where foreigners no longer feel comfortable coming and going from our country.
Two book recommendations
Should-read: A Dictionary of Maqiao
I’m really enjoying Julia Lovell’s translation of Han Shaogong’s “novel” in the form of dictionary entries of the Maqiao dialect.
Should-read: The Face That Launched a Thousand Lawsuits
I got a chance to hear Jessica Lake, Melbourne Law School senior lecturer, talk about her book at the IAPP global privacy summit this past week. It provides an account of how American women “shaped the common law right to privacy during the late nineteenth and early twentieth centuries.”
Thank you for reading and engaging.
These are Jeff Ding's (sometimes) weekly translations of Chinese-language musings on AI and related topics. Jeff is an Assistant Professor of Political Science at George Washington University.
Check out the archive of all past issues here & please subscribe here to support ChinAI under a Guardian/Wikipedia-style tipping model (everyone gets the same content but those who can pay for a subscription will support access for all).
Also! Listen to narrations of the ChinAI Newsletter in podcast format here.