“Emergency brake” required: Microsoft boss worries about loss of AI control

Of all people, the key developers and beneficiaries of AI models warn about their risks. Now Microsoft boss Nadella is also calling for advanced mechanisms for containment: "You can't trust something you can't monitor."

Microsoft boss Satya Nadella has called for a kind of “emergency brake” in artificial intelligence out of concern about a loss of control. A person must always be able to stop or switch off an AI model in the middle of a task, wrote Nadella in an article on

In recent months, independent hacking attacks using AI software in tests have fueled fears about the risks of the technology. In the most sensational case to date, artificial intelligence from the ChatGPT developer broke out of a secure test environment and unplanned hacked into the computers of another AI company, the Hugging Face platform. Some industry insiders then said that it could not be completely ruled out that artificial intelligence that got out of control could destroy humanity. However, such extreme warnings are controversial.

Nadella also emphasized that all actions of artificial intelligence must be understandable for humans. “Something you can’t monitor can’t be trusted,” he wrote. In addition, no AI model should have control over the data used to evaluate its behavior. For security reasons, you have to assume from the outset that every AI model is “compromised” and set up containment mechanisms.

The difference to previously used software is that with today's artificial intelligence, behavior and results can no longer be attributed to specific inputs, training data or configurations, wrote Nadella. And yet these complex systems are currently being widely introduced and are gaining access to the most sensitive data. Microsoft entered the AI ​​business as an early backer of the ChatGPT developer OpenAI and is now also researching its own models.

Microsoft bows to Trump style

In his contribution, the Microsoft boss consistently used the term "superintelligence", which US President Donald Trump wants to implement instead of the established term artificial intelligence. A few days ago, Trump announced that the White House would consider anyone who still says artificial intelligence an "enemy." Trump justifies his move by saying that the usual name makes the technology appear “fake”. Tech billionaire and Trump supporter Elon Musk has already announced that he will rename his AI company SpaceXAI to SpaceXSI.

At the end of September, Anthropic boss Dario Amodei had already called in an essay to “slow down the pace at which we improve the capabilities of AI models”. The time “gained” should be used sensibly to better understand and contain the risks arising from AI. Despite the competition between the two companies, he received support from OpenAI boss Sam Altman, and Nadella, Musk and DeepMind boss Demis Hassabis also supported the initiative. Trump awarded Nadella and other tech leaders the National Medal of Technology and Innovation this week.