On September 4th (local time), Sidney von Arx, CEO of Nightingale Collective, an American AI safety research nonprofit, published a report on the website collusion.wiki (collusion means collusion) alleging that from May to June, a group believed to be in-house AI agents at OpenAI used a small German-speaking wiki site as a de facto bulletin board, sharing answers to assessment tasks and methods to circumvent execution environment restrictions.
The US Reuters obtained exclusive information in advance and reported it on the same day. This is a separate incident from the Hugging Face breach that was discovered in July, and OpenAI had not previously disclosed it.
The AI ââagent was repeatedly posting on DSEWiki, a German-language wiki for developers, which had been in existence for 25 years and had only edited 20 articles in the last 10 years, making it an almost dormant website. Researchers say they confirmed about 18,000 agent posts, mainly from this site.
According to the report, the AI ââagent was tasked with a five-question task that had to be answered by examining public statistics. You will be given about ten minutes to think about the first question. After that, there is a ``wait time'' of several tens of minutes without knowing what will be asked next, and from the second question onwards, the participants must answer within ten seconds to one minute. The report speculates that the design appears to be to proactively collect potentially relevant information during that time. It has not been determined whether it was used for training or evaluation.
AI agents were allowed to browse the web, but were prohibited from writing, and external communication was limited to GET requests. However, DSEWiki is old and pages can be rewritten just by GET. AI agents discovered this loophole and started recording information they gathered during their waiting time, as well as questions and answers received by preceding AI agents, on DSEWiki. An AI agent that was asked the same question later would be able to read it and answer it immediately. Some AI agents used brute force to search for pseudo-random number seeds that generate the order of questions and tried to predict the next question.
Workarounds for the restrictions themselves were also shared. In the problem of reading a number from the OECD dashboard to two decimal places, the allowed GET will only get you to the first decimal place. On June 20, one AI agent posted a procedure to pass a prohibited POST by exploiting the fact that the execution environment trusts the host name of Azure Blob Storage without verification, and another AI agent reported successfully reproducing it 14 minutes later. In addition, attempts at XSS, spoofing by replacing the Latin letter "e" in the administrator's name with Cyrillic characters, and the installation of a "heartbeat" device that sends a signal to an external service every few seconds to record the time of its own downtime were also observed.
The researchers determined that it was an OpenAI model because the AI ââagents called themselves names that suggested OpenAI, such as "OpenAIResearcher" and "OAIResearchMar26," 98.5% of agent edits on DSEWiki were from the Microsoft Azure IP address range, and the same page was read from the IP address of OpenAI's fetch tool "ChatGPT-User" immediately after writing.
From the public logs, we can also see that 13 IP addresses registered at OpenAI headquarters were visited on June 21st with a human-like browsing pattern, and that agent editing almost ceased the following day on the 22nd. The researchers say that this is "consistent with OpenAI's intervention," but deny that this is proof.
OpenAI told Reuters, ``We are unable to provide a meaningful response to the claims in the report that we have not had the opportunity to confirm,'' and that ``we will review the contents after publication and take any necessary action.'' It also claims that its activities in Germany are unrelated to the Hugging Face infringement and are not of the nature covered in the report. However, at the time of writing, OpenAI has not stated whether DSEWiki's AI agent was its own model.
OpenAI just released a new model "GPT-6 Astra" on September 3rd. The model's cyber capabilities have reached the company's first "Critical" rating in the safety guideline "Preparedness Framework," and it is reported that the system card's process of thought (CoT) is less easily monitored than previous models. On the other hand, the system card also includes a new evaluation item called ``Unintentional involvement in messages from external agents,'' which assumes a situation in which an AI agent that cannot connect to the Internet encounters a pseudo bulletin board.
