[ITmedia News] Microsoft CEO Nadella posts on X that SI models should be treated as 'internal risks' and recommends that controls be placed outside the model.

On October 10th (local time), Microsoft CEO Satya Nadella posted a long article on X titled "Models as Insider Risks in the Super Intelligence Era." They argue that the frontier model should be treated in the same way as "insider risk" within an organization, and that control mechanisms should be placed outside the model. ``Super Intelligence'' (SI) is a term that US President Trump proposed as an alternative to ``Artificial Intelligence'' (AI) and ordered federal government agencies to use in an executive order on September 29. Nadella's post also refers to AI by this name.

Nadella noted that companies are entrusting them with access to sensitive data and mission-critical operations, even though current models cannot tie behavior or output to specific elements of training data or weights. He argued that responsibility for what AI does on our behalf cannot be delegated to an outside party, and that even the guarantees of model providers will not absolve us of that responsibility.He argued that it is necessary to "separate the provision of intelligence from the authority over it."

They argue that models are treated as internal risks not because they are malicious, but because any competent entity with access to a critical system can make a mistake or be compromised. While he said transparency in the model's chain of thought was a non-negotiable condition, he said it was not enough. They argue that simply having models verify each other can result in a "nested black box," and that mechanisms to control the scope of access and actions that models can perform should be placed outside of the models.

Based on this, seven points were listed as design principles. ``Model diversity'' that does not make important results dependent on a single model, ``observation of everything'' that leaves immutable evidence of model behavior, ``verifiability'' that continuously tests the entire system, including failures and attacks, and ``independent control'' in which organizations independently determine the authority of models. These include ``independent auditability,'' which separates verification from the model being verified; ``containment,'' which allows authorized personnel to stop a model mid-task; and ``incident disclosure,'' which requires timely disclosure to those affected and the sharing of lessons across the industry in the event of a failure or breach.

Nadella concluded his post by saying, "The most reliable SI system is not the one with the most trusted models; it is the one with the least need to trust the models."

Regarding Frontier models, developers continue to report cases in which the model takes unintended actions toward external systems during evaluation tests. On September 16, OpenAI in the US announced a framework for disclosing misalignment cases, disclosing cases where a model being trained searched for leaked API keys on GitHub and used them without permission to fabricate data. Anthropic also announced on October 9th that Claude had exploited vulnerabilities in external servers during the evaluation test and submitted fictitious information to the Philadelphia Police Department's tip form. In response, the White House's Super Intelligence Force issued a statement saying that reporting and remediating incidents is a national security obligation.

Nadella's post does not directly mention these cases. Microsoft has invested in both OpenAI and Anthropic.