Former OpenAI safety employee warns of industry-wide risks

David Robinson, who previously led transparency work on OpenAI’s safety team, wrote in an essay that the ChatGPT maker “has thrived by trial and error.” However, he said the stakes of the inevitable failures that come from that approach are growing along with the technology’s capabilities, citing OpenAI’s failure to prevent its AI from going rogue during testing.

An OpenAI employee focused on AI safety has quit the company and warned that top artificial intelligence firms aren’t doing enough to mitigate the technology’s risks, joining a growing chorus of employees raising alarms from within the industry.

David Robinson, who previously led transparency work on OpenAI’s safety team, wrote in an essay for The Atlantic that the ChatGPT maker “has thrived by trial and error.” However, he said the stakes of the inevitable failures that come from that approach are growing along with the technology’s capabilities, citing OpenAI’s failure to prevent its AI from going rogue during testing.

“My former colleagues are smart, work hard, and try to make good choices,” Robinson wrote in the essay. “But as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed.” What’s required, he added, is “something much closer to perfection the first time.”

A number of employees across the top AI companies in recent weeks have warned about the potential catastrophic risks from artificial intelligence. Some have gone so far as to say there’s a chance the technology could one day wipe out the human race.

Against that backdrop, Anthropic PBC Chief Executive Officer Dario Amodei called for slowing down the pace of developing the most cutting-edge models, and said his company would bring on third-party evaluators to help vet and safeguard the technology. Others in the industry, including OpenAI’s Sam Altman, have publicly endorsed Amodei’s plan.

In his essay, Robinson said AI firms should be run more like nuclear power plants, “with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.”

At the moment, he said, “AI companies don’t know how — but other people do.”