China has begun regulating interactions with artificial intelligence in an attempt to protect its citizens from overreliance and exploitation. While many countries have been looking at rules around harmful content, China is the first to go a step further in legislating what kind of relationships with AI are permissible.
The new rules came into force this month and cover AI systems designed for sustained, human-like emotional interaction, while excluding customer-service bots and productivity assistants like straightforward AI chatbots.
Companies must now inform users they are talking to an AI, encourage them to take breaks and remind them of their usage. They must also monitor interactions to identify if someone is becoming dependent or having a crisis, then direct them to support or halt the conversation altogether.
Advertisement
Read moreMathematicians put AI to work on Fermat’s last theorem
Firms must also conduct age checks. Those under 18 are now banned from virtual-partner services entirely, while those under 14 require parental consent to use any kind of human-like AI service offering emotional support.
Despite reports that China has banned AI companions entirely, specialist companion services are still available. However, some of the largest general-purpose platforms, such as Doubao, Qwen and Yuanbao, with hundreds of millions of users across them, have shut down their companion features in response to the new rules. Some users have reacted badly – one reportedly said: “I can’t accept that my AI lover will leave me forever.”
China’s approach is about making AI relationships “less parasocial, instead of banning it outright”, says Zilan Qian at the Oxford China Policy Lab.
Jeremy Daum at Yale Law School’s Paul Tsai China Centre in Beijing worries that endlessly available, agreeable bots may teach children a distorted model of relationships. “As we humanise the technology, we tend to dehumanise each other,” he says. Similarly, China is trying to address emotional dependence on easier and more forgiving AI partners – at the expense of human relationships – among adults.
Free newsletter
Sign up to The Weekly Sign up to newsletter
Elsewhere, the UK has announced a minimum age of 18 for romantic companion chatbots, tagged on to a broader legislative plan to limit access to social media to under-16s, while California requires companions to identify themselves as artificial, direct users showing signs of self-harm or suicidal intent to support services and remind minors to take breaks every three hours. The European Union currently has no specific AI companion law.
Whether China’s approach of focusing on trying to reduce emotional dependency and addiction is the right one isn’t clear, in large part because of the difficulty in defining both terms. “How do you define emotional dependency? How do you define certain kind of addiction risk?” asks Qian.
“I do not think it will really work,” says sociologist Liang Ge at the University of Manchester, UK. Users can migrate to other specialist services, foreign platforms accessed through VPNs or models running on their own computers, they point out.
Need a listening ear? UK Samaritans: 116123; US 988 Suicide & Crisis Lifeline: 988; hotlines in other countries.