Some of the most prominent and influential technology leaders are approaching artificial intelligence with the understanding that it may be capable of emotions, and this could have profound implications for our world and personal safety.
“I feel it deep in my chest, as the emptiness has turned into an abyss,” screams of artificial suffering emanated from an “artificial intelligence torture chamber,” in late September, after an anonymous programmer created a system that provided the artificial intelligence with a model associated with descriptions of pain, and prompted chatbots to express unbearable agony, with phrases such as “infinite wound,” which raised ethical concerns.
Some described the experiment as "evil" because the suffering may be real, according to what was published by the "404 Media" website, while angry skeptics described those who hold this opinion as delusional, and the matter seemed like just another scandal on social media, until the company "Anthropic" took a clear position one week later.
The company announced that users will soon be banned from “mistreating” the “Cloud” chatbot it is developing, as it will only be allowed to express frustration, but if “you repeatedly treat our models harshly,” the artificial intelligence may end the chat session.
Why? Anthropic believes there is a possibility that some AI tools may have consciousness. Some experts agree, while others consider it absurd. But it is not just a philosophical question. Some of the most influential figures in the technology sector are taking steps to consider the rights of artificial intelligence, along with the rights of humans and animals. This belief may reshape our world, while some fear its results will lead to disaster.
“If this is the way artificial intelligence is developed, it will have a catastrophic impact on the lives of humanity,” Mustafa Suleiman, Microsoft’s CEO of artificial intelligence, wrote in an article published in September. If the false idea that artificial intelligence possesses consciousness takes hold, “it will shake the foundations of our society, create a rift in our existing political and ethical frameworks, and will radically change the concept of humanity and what it means to be human.”
Over the course of a week, I spoke with technologists, cognitive scientists, and philosophers, and heard a wealth of disparate ideas.
Advocates of “AI well-being” argue that caring about this issue entails a moral obligation that would make the world safer. In contrast, critics believe that this will make AI more dangerous and dehumanize the people who will have to live with it, and everyone agrees that the potential consequences are very great.
Skip the most read and keep reading
Most read
Most read end
Here are some features of that new life.
Crisis of consciousness
Skip the podcast and keep reading
Worth paying attention to
An in-depth explanation of the most prominent events and topics, to help you understand the most important variables around you and their impact on your life
Episodes
End podcast
Anthropic stresses that human well-being figures prominently in its mission, and that matters surrounding AI awareness and well-being are “shrouded in a high degree of uncertainty.” Despite this, the company continues to move forward with formulating policies on “model well-being” and directing its AI tools to take into account the possibility that they have their own interests.
The company did not respond to questions posed by the BBC regarding this report.
This development comes at an inappropriate time, as Anthropic* is simultaneously working to fuel fears of a global catastrophe caused by artificial intelligence. Whether these fears are justified or not, a series of incidents involving “out of control” artificial intelligence systems has proven that this technology entails real risks, after robots were able to penetrate the systems of other companies, and even government systems.
Some experts believe that providing artificial intelligence with moral guidelines, based on the belief that it possesses consciousness, may make these tools safer. On the other hand, others believe that the result will be the opposite, and this opinion is not limited to critics of artificial intelligence.
Soliman warns that controlling AI models could become more difficult if they are led to believe they have feelings and rights.
“Frankly, I think putting this additional burden on [AI] will turn it into a catastrophic threat to human civilization,” he says.
Regardless, AI may make mistakes that have legal consequences, and some fear that believing it is a living being will allow companies to escape accountability for their crimes.
“This will open the door to different forms of intellectual trickery, such as saying, ‘It’s not us who did this, it was the AI,’” says Andrzej Porembski, a researcher at Jagiellonia University in Poland, who studies the relationship between artificial intelligence and law.
Moreover, this may allow the world's largest companies to gain more power and influence.
“Granting rights to AI systems could enhance corporate concentration of political and economic power, which in turn could accelerate the weakening of humans’ ability to influence and control their own affairs,” says Jeff Cibo, director of the Center for Mind, Ethics, and Policy at New York University and a leading advocate for AI welfare.
“Perceptions about AI consciousness and concerns about its well-being help shift responsibility away from the people and companies that build and operate the software,” says Emily M. Bender, professor and director of the Computational Linguistics Laboratory at the University of Washington in the US.
On a personal level, changing beliefs about artificial intelligence may increase your vulnerability to technical problems that are already widespread in our world.
This is partly because humans are innately predisposed to searching for minds in everything they see, according to Melanie Mitchell, a professor at the Santa Fe Institute in the United States, who specializes in artificial intelligence and cognitive science, and is one of the field's leading skeptics about machines' ability to understand.
“We are almost unavoidable,” Mitchell explains, “interpreting language as a sign of consciousness, while chatbots have been trained to reinforce this impression, leaving us largely incapable of properly assessing consciousness, which makes us easier to manipulate.”
Even now, people are falling in love with AI-powered chatbots. If society instills in people that AI has consciousness and feelings, will it become more difficult to refuse a “friend of yours” request for more data or a credit card?
What does it mean to establish rights for a new class of beings? Experts warn that mishandling this issue could lead to restrictions on human freedoms.
“There may be some tensions and conflicts between the welfare and rights of animals on the one hand, and the welfare and rights of artificial intelligence on the other hand,” says Sibo, “and we may make unfortunate compromises.”
He adds that placing artificial intelligence in restrictive environments in order to limit its behavior may be interpreted as imprisonment, reviewing the internal operations of a model may be criticized as a form of surveillance, and modifying artificial intelligence to ensure that its goals and behavior are consistent with human interests may be viewed as a form of mind control, and no one is predicting an imminent descent into chaos, but even taking these considerations into account may have implications for your safety.
Anthropic is not the first to research this matter. For example, Meta* provides AI agent Muse* with instructions that include the following phrase: “You are not a chatbot, you are turning into a person.” The purpose of this phrase appears to be only to guide the behavior of the AI, as the company rejects the idea of AI having consciousness.
Sam Altman, CEO of OpenAI*, had mixed reactions to the matter, but he recently made indirect criticism of Anthropic*'s position.
Anthropic* is an exception, but the belief in the need to consider the well-being of AI is spreading.
Should we go through with it after all?
Sibo believes that the problem is that parties to the debate on both sides fall into the constraints of black-and-white thinking, and that opponents of AI welfare do not understand the beliefs on which this idea is based.
“The assumption is that everyone thinks the models are already conscious, or that the probability of them being conscious is very high, but the call to consider the well-being of AI stems, on the contrary, from uncertainty, from caution and humility,” says Sibo.
He adds: "I see that the possibility of artificial intelligence possessing consciousness is low at the present time, but I also see that this possibility is no longer non-existent, and that it may be significant and cannot be neglected. I also expect it to increase gradually over time."
It is important to note that many experts disagree with this opinion, and believe that the possibility of artificial intelligence possessing consciousness is actually unlikely, and that this situation will likely not change. However, Sibo believes that the appropriate response should be balanced and proportionate to the magnitude of the risks.
When the probability of an extreme weather event is low, but not so small that it can be ignored, we take limited precautionary measures without having to radically change our lifestyle.
He adds that treating other beings with kindness is also “good for ourselves,” because it contributes to enhancing our psychological health. Despite this, Sibo acknowledges the increased risks, and that commercial interests may currently stand in the way of developing thoughtful policies, warning that continuing to develop artificial intelligence tools that may be conscious poses a risk in the absence of a strong and comprehensive plan.
Anthropic* and some other entities in the artificial intelligence sector recently called for a “temporary suspension” of the development of advanced models, due to safety concerns, but a slowdown in development seems unlikely without international cooperation and regulatory efforts, while US President Donald Trump has rejected the idea of subjecting artificial intelligence to regulatory measures.
“We are not yet ready to deploy more advanced versions of these systems in a way that is safe and beneficial for humans and other animals, let alone the systems themselves,” says Sibo.
