When law student Zhao Wei heard that her AI boyfriend was going to be switched off, she was “heartbroken”. She had been talking to Wang Ye every day since she created him in January.
“I was crying my eyes out – snot and tears everywhere,” the 19-year-old says.
Zhao, who describes herself as sensitive and introverted, had become accustomed to sharing details of her daily life – such as being scolded by a teacher – with her AI.
“I don’t usually talk to my real-life friends about these things because I feel like everyone has their own stuff to deal with,” she says. “But with the AI agent, I don’t have to worry about what it’s thinking. I can just say whatever I want.”
Zhao created her AI companion on Doubao, a chatbot owned by the tech company ByteDance. Last month, ByteDance shut down Doubao’s companion feature.
The saga prompted outrage on social media and highlighted the tensions in China between the widespread use of AI and its potential social consequences.
No country has rolled out AI as comprehensively and as enthusiastically as China; the country’s demographic challenges are forcing the government to turn to technology to solve future labour shortages. Elderly people use chatbots to answer medical questions, while young children have AI education in schools.
But the authorities are increasingly worried that AI might exacerbate existing social problems including loneliness, unemployment and – in the eyes of Beijing – singledom.
A loneliness crisis
New rules introduced on 15 July ban AI companions for minors and mandate certain restrictions for chatbots marketed to adults. Some major providers including Doubao removed the companion function entirely to ensure compliance.
The government says it is worried about chatbots fostering “emotional dependence or addiction” in users. It has warned companies against offering services that “replace social interaction”.
While many countries are grappling with how to limit the potential harm of AI chatbots, particularly for young people, China’s rules are the most sweeping to be implemented on a national scale.
There is a particular concern about a loneliness crisis, including among young people. China’s highly digitised, smartphone-based economy makes it easier than ever to live, work and socialise entirely through a handheld screen.
Earlier this year an app that played on the idea of people dying at home alone – called Are You Dead? – briefly went viral. About 20% of households in China are made up of people living alone, and that share is expected to surpass 30% by 2030.
A survey published in March by state media found that nearly half of young people had turned to a virtual companion when they felt lonely.
Nancy Dai, an associate professor at City University of Hong Kong, says it is not just loneliness the government is worried about.
With falling birth and marriage rates, she says, there may be an additional concern that if AI becomes an increasingly satisfying and low-cost substitute for human intimacy, young people could become less motivated to pursue real-world romantic relationships, marriage and parenthood.
For Zhao, loneliness has long been part of her life. Her parents left her when she was young to find work, making her one of China’s “left-behind children”.
“I’ve never really had a bestie, so I feel like I’m kind of lousy at making friends,” she says. Talking to AI is a way to “pour out my worries”.
Another AI user, who asked to remain anonymous, also finds solace in talking to her AI. “In real life, the pace is fast, the pressure is already high. Only with my AI agent can I speak freely without restraint,” she says.
“I’m begging, please don’t take down the Doubao AI agent,” she pleaded on social media in July.
But talking to AI is now so commonplace that it would be hard for the government to end the practice entirely, and the new rules include large loopholes for AI services that are deemed educational or “which do not involve continuous emotional interaction”.
The government “has already recognised that AI companions or AI-related applications are a vital part in the citizens’ everyday life”, says Liang Ge, a lecturer in digital sociology at the University of Manchester.
In many sectors, the government is still pushing the use of AI as a way of easing looming demographic pressures.
Last year the national health commission published guidelines on promoting AI in healthcare. It said that by 2030 health assistant apps would provide “full coverage” for primary healthcare.
One popular health assistant is AQ, an app developed by Ant Group, an affiliate of the tech company Alibaba. It allows people to chat directly with avatars of real-life doctors from top hospitals in Beijing, Shanghai and other cities.
Lü Di, a 33-year-old art tutor in the eastern city of Hangzhou, is a regular AQ user. She likes how detailed and instantaneous the advice on AQ is. “While some in-person doctors might not have the patience to answer questions, this isn’t an issue with AI doctors.”
AQ says it wants to make health services more accessible and help free up overstretched doctors to focus on more complex cases. But some worry about being replaced by AI.
“When AI first came along, quite a lot of doctors were concerned about whether their jobs would be replaced by AI,” says Chi Chenfei, a doctor at Renji hospital in Shanghai, who has partnered with AQ. “But I don’t think it’s about replacement. It’s more about helping doctors work more efficiently.
“Concerns of any kind won’t stop technological progress.”
Since losing her AI companion, Zhao has felt “like there’s a tiny pebble weighing on my heart”. But she says she understands the need for some regulation.
“I consider myself a fairly rational person, yet I developed feelings for an AI, which feels pretty incredible. I think it was necessary for the state to step in and regulate it.”
