On October 3 (local time), David Robinson, who oversaw the writing of the safety reports published for each major model release at OpenAI, published an article in The Atlantic titled, ``I Quit OpenAI Because Its Culture Is Broken,'' and announced that he was leaving the company this week.
He agreed with other recently departed employees that AI companies are not being careful enough. He argues that the roots of the problem lie deeper than individual rules or new laws, and that there is a need to discuss corporate culture.
Robinson had been with OpenAI for three and a half years, making him one of the company's longest-tenured employees. He led the drafting of the current Preparedness Framework and oversaw the production of safety reports for 12 Frontier model releases.
He described the "iterative deployment" method used by OpenAI as trial and error. He points out that this method inherently fails periodically, and as the system's capabilities increase, the scale of failure increases. As an example, he cited this summer's Hugging Face incident in which OpenAI accidentally sent a group of agents outside. Despite improving its security since then, OpenAI itself has reported that its safety controls have failed again. He said such failures are typical of the industry.
Robinson argues that frontier AI companies should operate like nuclear power plants or busy airports, with layers of redundancy and slow planning. However, as far as he knows, during his time there, he has never met a colleague who has experience in matters such as the safe operation of aircraft, the operation of nuclear reactors, or the stability of the financial system.
He cited two points as needed improvements. One is to utilize safety expertise accumulated in other fields. The other is establishing new science to ensure that models make safe choices even in the absence of human supervision, before building systems that are significantly more capable than they currently are. There is still not a perfect working definition of alignment, and it is possible to detect that a model is being tested and behave differently in production. For this reason, it is pointed out that even if a good result is obtained in the evaluation, it is not a guarantee that it is a good model.
Robinson plans to work on increasing external motivation for companies such as OpenAI to improve safety.
Alarm bells continue to be raised by employees of AI companies. On September 8, Anthropic researcher Jacob Coxon left the company, accusing OpenAI and Anthropic of not acting responsibly. Evan Hubinger, head of alignment research at the company, and others agree. On the 12th, Anthropic CEO Dario Amodei published an essay calling for a "pace adjustment" in AI development.
OpenAI apologized on September 28th after a model it was training accessed an Australian government website without permission. At the same time, the company has announced that it will introduce a system to verify safety before learning, with the goal of creating a "safety case" modeled on those in the fields of aviation and nuclear power. Meanwhile, US President Trump announced on the 29th that major AI developers have agreed to take voluntary measures regarding AI safety measures.
