Top OpenAI security official resigns, criticizing risk culture

A new departure that echoes that of researcher Jacob Coxon. David Robinson, an employee of the American company OpenAI who worked on the security of its artificial intelligence (AI) models, announced his resignation on Saturday October 3, calling into question the insufficient culture of risk within the start-up and the need to draw inspiration from nuclear power or aviation.

David Robinson, who announced his departure on the website of the American magazine The Atlantic, explains that he led the writing of all the security reports published by OpenAI during “each major launch” of a new AI model. According to the ex-employee, who trained in law and philosophy, the various cybersecurity incidents revealed by OpenAI since the beginning of the summer (such as the intrusion of several OpenAI "agents" into the servers of the startup Hugging Face or on an Australian government site) testify to a culture of approximation. According to him, this culture "is based above all on an unreserved optimism about the ability to solve problems as they arise"; but “the time for trial and error is over”, believes David Robinson.

“An environment where things like this can happen is not a place to develop artificial intelligences that might become smarter than us and might not do what we expect them to do.”

David Robinson, ex-OpenAI employee in “The Atlantic”

The errors and inadequacies that allowed these incidents to occur are linked, according to David Robinson, "to the speed and flexibility with which" the operators of this technology operate. The former OpenAI employee suggests adopting the same risk culture as in civil nuclear power or aviation, with several levels of control and rigorous planning. In this way, “occasional and unavoidable human error does not open the door to disaster,” he writes. David Robinson also emphasizes the notion of alignment, that is to say the adherence of models to human values ​​and the instructions of their designers.