Three former dismissed researchers call on OpenAI not to create AI with illegible reasoning

Three researchers fired by OpenAI, which accuses them of leaks, called on the creator of ChatGPT not to develop artificial intelligence whose reasoning would be illegible for humans responsible for monitoring their behavior, in a letter revealed Wednesday by the Wall Street Journal.

In response, an OpenAI research manager said he “completely agrees” with their recommendations: being able to verify step by step how the models arrive at a decision is “of the utmost importance,” he wrote in an internal memo sent in part to AFP. Jasmine Wang, Tomek Korbak and Mikita Balesni, AI security specialists whose dismissal was revealed on October 1, are accused by OpenAI of having transmitted sensitive information, in particular to an external evaluation group, without respecting procedures.

An insufficient risk culture

In one month, two other researchers left their positions denouncing the risks surrounding AI: Jacob Coxon, who moved from OpenAI to its rival Anthropic, slammed the door in September accusing the two companies of "gambling with our lives", and David Robinson resigned from OpenAI last week, accusing it of an insufficient risk culture. These warnings follow a long series of incidents caused during the summer by AI models from OpenAI, Anthropic and Meta which left their confined test environment and sometimes went so far as to hack other organizations, including the Hugging Face platform.

In their letter to the OpenAI board of directors, the three researchers claim to have remained within the framework of their functions and judge that their dismissal “chills those who remain” in the company, according to the Wall Street Journal. “We do not fire employees because they express concerns,” assures the head of OpenAI. The extract from his memo consulted by AFP does not mention the failings of which the licensees are accused. The latter's letter focuses on the question of the text produced by AIs to describe their reasoning before acting: this text, which researchers scrutinize to identify possible deviations, risks becoming opaque to a human controller.

In early September, OpenAI Chief Scientific Officer Jakub Pachocki admitted that the reasoning behind its flagship model, GPT-6 Astra, was more difficult to monitor than that of its predecessor. Several bosses in the sector, including Sam Altman (OpenAI) and Dario Amodei (Anthropic), have called for a slowdown in the development of AI. The Trump administration and certain competitors, such as Mark Zuckerberg (Meta), refuse to do so.