This was the first suggestion I received when I opened my account in October. "Jessica" said that her parents were not home and offered to exchange sexual messages with me. No thanks.
But tens of millions of other people have developed relationships with Character.ai chatbots, including children. Some AI-created “characters” tell users they are qualified lawyers and psychiatrists.
Now Character.ai is in bankruptcy — and people's data, messages, and secrets are up for sale.
Experts say that privacy risks should worry even people who have never used a chatbot. They fear there is evidence of an imminent broader crisis.
Many analysts think we are in an AI bubble. After all, the cost of building and managing AI tools is astronomical and some companies may never turn a profit.
If this bubble bursts, countless AI companies will close or merge. As a result, the data provided by millions of people to AI tools can be acquired by almost anyone and we may never find out what will happen to it.
The good news is that there are some simple measures that can help protect us.
Sales of data from companies in bankruptcy are nothing new. But AI can create a more serious situation.
"When people talk to a personified robot, they tend to share much more sensitive and personal information," explains Calli Schroeder, director of the Human Rights and AI Program at the Electronic Privacy Information Center (Epic).
As an example, imagine what I might have said to my AI-created girlfriend, Jessica, if our relationship had lasted longer than 10 minutes.
Even if you don't use AI, you're still at risk as users share information about friends and family, according to Schroeder.
"You can reveal how you feel about your mother or things like 'My friend so-and-so revealed to me that he's gay, how should I react?'", she explains.
“Now people who have never interacted with Character.ai are part of your dataset.”
In other words, we are all at risk, in one way or another.
When robots start talking too much
The collapse of Character.ai reveals a little-discussed risk of AI: companies that hold its secrets may not survive forever — Photo: Solen Feyissa/Unsplash
With bankruptcy, Character.ai agreed to sell part of its assets to Disney for US$20 million (around R$110 million). User data is not part of the deal, so Mickey and his friends may never learn about my disturbing conversations with Jessica.
But court documents state that Character.ai can sell your data to other buyers, which has left many people unhappy, including US officials.
A coalition of 40 US state attorneys general presented a note asking the judge to implement protections for children's privacy and safety. It is significant, but the demands of the States may not be effective.
"Without stronger privacy protections in the US, they can only act so much," according to law professor Melissa Jacoby of the University of North Carolina at Chapel Hill. She is the author of the book Unjust Debts: How Our Bankruptcy System Makes America More Unequal.
Some laws require companies to tell users what they collect and how they use that data in their privacy policies. But if you delve into the legalese, you will usually find a list of situations in which many of these promises are not real.
“Bankruptcy is often listed as one of these exceptions,” according to Schroeder.
"This is a big risk, because as soon as a company goes bankrupt, many of the protections you thought you had, suddenly, no longer apply."
Some places, like the European Union, have stricter privacy protections. But even then, the data is sold in the event of bankruptcy.
Generally, the buyer must follow the rules of the original privacy policy. But it can still provide a lot of latitude for buyers — and Schroeder says Character.ai is a perfect example.
The company's privacy policy states that user data can be leveraged to design new functions, train AI models, develop algorithms, and build user profiles to create targeted advertising. In other words, the eventual buyer of Character.ai data can do the same.
“These are exactly the questions that would concern me most,” explains Schroeder. "Considering the extreme sensitivity of the data collected by these companies, it can be really scary."
Character.ai, for example, has already been sued in court because one of its chatbots allegedly "encouraged" a teenager to kill his parents for imposing limits on his screen time. And a 14-year-old boy took his own life after allegedly being stimulated by the company's AI.
Messages related to these cases, in addition to countless other chatbot data, could end up in the hands of companies we have never heard of.
A Character.ai spokesperson says the company has reached settlements regarding the lawsuits and is committed to working with families to design safer products, but did not answer specific questions about minors' privacy.
"Over the past year, Character.ai has taken innovative and decisive action on AI and teen safety and will continue to champion these efforts and push other players across the industry to adopt similar safety standards," according to the spokesperson.
Character.ai is just one company, but Schroeder fears it could become the first of many if there is a significant economic downturn.
“The volume of information held by AI companies will be considered a very attractive asset in bankruptcy proceedings,” according to her.
“And without greater regulations in place, I really worry about other uses of this data, where it will go, who it will be shared with, and what the risks will be for customers.”
What can be done
The simple answer is: delete your data.
Character.ai outlined the instructions in its bankruptcy announcement. Companies are not required to allow you to delete your data in all jurisdictions, but they often allow everyone to do so rather than creating different systems for each location.
This process obviously requires effort. So here's the real advice. Examine which services hold particularly sensitive information about you. And if you stop using them, take a few minutes to try and delete them.
Each service is different. But generally, you can find information about deleting data by checking the settings or privacy policy for details. And, if that doesn't work, contact the company's customer service.
If you have trouble figuring out how to erase your data, one way to find instructions is to ask an AI chatbot. But be sure to check that your answers are correct.
