Former OpenAI employee criticizes risk culture and calls for inspiration from nuclear power and aviation

A resigned OpenAI employee, who worked on the security of its artificial intelligence (AI) models, questioned the insufficient risk culture within the start-up and the need to take inspiration from nuclear power or aviation. The call, published on Saturday by the magazine website The Atlantic, echoes several similar messages launched in recent weeks, notably that of Jacob Coxon, formerly of OpenAI and Anthropic.

For David Robinson, who says he wrote the OpenAI security report during twelve different model launches, the incidents reported by the group since the beginning of the summer testify to a culture of approximation. The errors and inadequacies that allowed several AIs to spontaneously join the internet to intrude on dozens of sites and platforms are linked, according to him, “to the speed and flexibility with which” the operators of this technology operate.

Skip the ad

“An environment in which things like this can happen is not a place to develop artificial intelligences that might become smarter than us and might not do what we expect them to do,” says David Robinson.

Self-regulation

The one who spent three and a half years at OpenAI suggests adopting the same risk culture as in civil nuclear power or aviation, with several levels of control and rigorous planning. In this way, “occasional and unavoidable human error does not open the door to disaster,” he writes.

David Robinson also emphasizes the notion of alignment, that is to say the adherence of models to human values ​​and the instructions of their designers. The giants of artificial intelligence lack certainty about the reliability of the supervision of the models, he underlines.

In addition, the latest AIs are becoming better and better at detecting testing phases and several speakers have raised the possibility that models seek to mislead their evaluators. On Tuesday, following a meeting at the White House, the big bosses of AI made a series of commitments which all relate to self-regulation. They relate in particular to internal controls and the integration of external observers.

Donald Trump has so far publicly demonized employees of cutting-edge AI companies, and even certain bosses, who warned of the dangers of this technology.