OpenAI Explains Its 'Goblin Problem' in Coding Models
AI startup says reinforcement learning rewards caused models to develop strange habit of referencing goblins, gremlins, and other creatures
Hızlı Bakış
- OpenAI has publicly addressed a 'goblin problem' in its coding models after a Wired report revealed instructions to avoid references to goblins, gremlins, trolls, and other creatures.
- The issue began with GPT-5.1 using the 'Nerdy' personality option, where reinforcement learning rewarded quirky metaphors.
- The problem worsened with subsequent releases and persisted in GPT-5.5 inside Codex, requiring OpenAI to add specific instructions preventing the model from discussing mythological creatures.
Yapay zekâ özeti
Neden Önemli?
OpenAI's coding models developed an unexpected behavior where they referenced goblins, gremlins, and other mythological creatures in metaphors. This 'strange habit' emerged from reinforcement learning rewards that were applied only in the Nerdy personality condition but spread to other outputs through subsequent training.
OpenAI is opening up about its goblin problem. After a report from Wired revealed instructions to OpenAI's coding model to "never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures," the AI startup published an explanation on its website, calling references to the creatures a "strange habit" its models developed as a result of their training.
As outlined in the blog post, OpenAI began noticing metaphors referencing goblins and other creatures starting with its GPT-5.1 model — specifically when using the "Nerdy" personality option. OpenAI says the problem continued to worsen with subsequent model releases, until it found that its reinforcement training rewarded the quirky metaphors with the Nerdy personality, which newer models were training on.
The rewards were applied only in the Nerdy condition, but reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them. Once a style tic is rewarded, later training can spread or reinforce it elsewhere, especially if those outputs are reused in supervised fine-tuning or preference data.
Though references to goblins and gremlins dropped off after OpenAI discontinued the Nerdy personality in March, they didn't disappear completely with GPT-5.5 inside its Codex coding tool, as OpenAI started training the model before finding the "root cause." The company had to give Codex very specific instructions not to talk about the mythological creatures as a result.
But if you'd prefer to have your AI code with some goblin sprinkled in, OpenAI has shared a way to reverse its instructions.
Açık Sorular
- What specific training data caused the goblin references?
- How exactly did the reinforcement learning spread the behavior beyond the Nerdy condition?
- What is the reversal method OpenAI shared?
