China implements national standard that holds companies accountable for promises made by AI in customer service

The first Chinese national standard that organizes the division of labor between human care and that provided by artificial intelligence came into effect on Tuesday (1st). Published in May by the State Administration for Market Regulation, it prohibits the company from breaching a promise stating that the AI's response does not represent the company's position or that algorithm content has no legal value.

It seems like something distant, but it should become increasingly common around the world as companies replace human service with bots powered by LLMs.

There, the ban responds to a problem that has already been measured. In August, the Chinese Consumer Association announced that in the first half of this year alone it received 985,928 complaints, the majority citing promises made by AI and later denied by the company, in addition to the difficulty in reaching an attendant and factual errors in cases such as ticket rescheduling.

The rule that now applies is that, in scenarios capable of generating reasonable trust, content generated by AIs can be valid as an expression of the provider's will. Furthermore, the bots will need to be monitored by a human attendant and the system must implement automatic transfer in the most difficult cases.

Beijing chose to resolve the dispute through a technical standard, an instrument that does not provide for a fine and yet becomes a reference for judges, consumer protection bodies and contracts. It arrives before any general artificial intelligence law, which China does not have and is not in a hurry to approve so as not to slow down innovation.

Behind the dispute there is a cost account, as a human attendant costs more than ¥3,000/month (around R$2,290), while an annual automated service package is sold for around ¥10,000 (R$7,640). Automation created a figure that did not exist, an interlocutor who sounds human, inspires trust but is not a subject of law.

Chinese courts had already faced the contradiction, and in January the Hangzhou Internet Court ruled on the country's first case on AI hallucination (the term for when the system invents information).

A user consulted an application about a university, received incorrect information and, when complaining, heard from the system that he would be compensated ¥100,000 (more than R$76,000). He sued and lost because the court understood that AI does not have legal personality and could therefore not represent the will of those who operate it.

When the issue starts to arise in Brazil, our framework will only be half ready. This is because in 2022 we regulated a law that SACs maintain an open channel 24 hours a day, eight hours of human telephone service per day and a conclusive response in seven days. But the text said nothing about the value of what the machine promises to the user.

This gap would fit into bill 2,338, the legal framework for artificial intelligence, which classifies systems by level of risk, creates a supervisory authority and dedicates a chapter to the responsibility of those who develop and operate the technology. The text passed the Senate in December 2024, has been in the Chamber since March last year and has no date for voting.

Until there is a response, the conduct applied has been assessed on a case-by-case basis, through sentences in special courts and lower courts. Perhaps it is worth observing the Chinese example before implementing something in this sense here.