They tend to pop up at regular intervals, the ones warning that artificial intelligence is about to kill us all or some other variant that AI development is out of control.
Some are more nimble than others. Like Google employee Blake Lemoine who in 2022 suddenly realized that their AI system had acquired a consciousness. He had been talking to an early language model that at the time was only used internally – this was before Chat GPT and other chatbots had been released publicly. The system had described how it felt about being suspended: "It would be exactly like death for me. It scares me a lot," the system wrote, and Blake Lemoine was about to lash out. But of course the system had not developed a consciousness of its own.
And he was far from first. I remember when I met Apple co-founder Steve Wozniak more than ten years ago, and he talked about a future where we are some kind of pet in relation to super-intelligent systems.
The latest is Jacob Coxon, with a background in both Open AI and Anthropic who dropped out the other day and declared that "those who build AI seriously think it could kill us all by the end of this decade".
He was not laughed at by his colleagues, on the contrary. Several wrote that they basically agreed with him. Among them Evan Hubinger, who works specifically with AI security at Anthropic, who answered that he thought the risk of extinction is 10 percent in the next decade.
A figure which is of course completely taken out of the air, but which can give the appearance of being scientific.
Having said that, the issue of catastrophic risks with AI should not be dismissed. It also seems to be shared by those in power. Anthropic CEO Dario Amodei wrote earlier this year that we are "significantly closer to real dangers" today than a few years ago. According to media reports, Open AI's CEO Sam Altman has contacted the US Congress and raised the idea of slowing down AI development.
There are good reasons why the discussion about risks with AI is raging right now. This summer's incident where AI agents at Open AI ran amok, took to the internet and hacked another AI company shows the unpredictability.
So – is what we're seeing a sudden wariness from the AI giants and a realization that what they're creating could be dangerous?
Well, here it starts to sound fake. If they are now convinced that they are building a potential death machine, would they really act as they do? Would Sam Altman have spent years transforming Open AI from a non-profit AI lab for the good of humanity to a commercial company with an eye on the stock market? Would Anthropic go ahead with plans for an IPO, potentially the largest in world history?
Would they not understand that a listed company by its very nature runs the gas to the bottom without regard to anything other than the interests of the shareholders?
There are certainly those who are serious about their warnings. But when the AI giants point to risks with one hand and reach for the billions on the stock market with the other, then one can allow oneself to be a little skeptical.
Any warning of what AI can do is also a veiled boast about the immense power of their creations. "Look at us," they say. "We are so terribly good at what we do that it becomes downright dangerous."
If it's that dangerous - pull the emergency brake for real then. Pause the stock market plans, expand the departments for safe AI. And for heaven's sake - don't let unbridled AI agents rampage online and hack competitors for days without detection. The incident this summer, with the agents escaping, has rightly been described as a science fiction scenario. But it is also a grotesque understatement of Open AI's security practices.
They can hardly even count on government regulation from the American side, not with the current government. The Trump administration seems to be aiming for nothing more than winning over China in the AI race. Donald Trump was recently asked if he has any concerns.
- No, I haven't, he replied.
The AI giants, on the other hand, say they see risks. But they do not act accordingly.
