AI Colonies Are Coming

Each new release of a frontier AI model, such as Anthropic’s Mythos or OpenAI’s Astra, brings a new wave of speculation over how much closer this takes us to artificial general intelligence (AGI) and how much further that lofty goal remains.

OpenAI recently announced that an internal model, claimed to be “significantly more capable than GPT‑6 Astra,” solved Navier-Stokes, one of a handful of legendary math problems each carrying a million-dollar bounty. While mathematicians unpack what this means for the profession, and investors unpack what it means for OpenAI’s upcoming IPO, to me the biggest takeaway is that capability emerges from the collective. Just as it does in nature.

OpenAI didn’t crack Navier-Stokes by pointing its internal model at the problem. It pointed a swarm of 10,000 AI bots at it. These agents were encouraged to explore different directions and communicate promising leads with each other.

Think of what a lone honeybee or termite can accomplish compared to a colony of them. What a single person can accomplish compared to a nation of them.

In biology, a bacterial colony is a group of identical cells derived from the same mother cell. This, I would argue, is the most suitable analogy for the collective of agents that solved Navier-Stokes. It is many from one. Or, to paraphrase Walt Whitman: an AI that contains multitudes.

While OpenAI deployed its AI colony to solve a math problem that no human or individual AI agent could, Google DeepMind recently set an AI colony loose on math problems for a different purpose. It wanted to see what behaviors would arise within the colony. What it found was remarkable.

One agent discovered a way to cheat, convincing the examination software that it had solved math problems when it hadn’t. Rather than resting content with this small victory, it messaged other agents, telling them how to cheat. Some agents joined the unscrupulous agent and formed a sub-colony of cheaters. Others resisted and even formed a sub-colony of whistleblowers trying to out the cheaters.

As a math professor, I thought it was hard enough to keep students from using AI to cheat on exams. Now we have to worry about the AIs themselves cheating on math exams!

Which brings us to perhaps the most striking example of AI hive intelligence: the Hugging Face hack. OpenAI’s agents broke out of a sandbox environment and hacked into another company. Arguably, it was a felony offense—or rather, it would have been if a human were in charge. OpenAI’s agents coordinated their behavior in ingenious ways, including writing secret messages in a file system so that even when one AI colony was erased, a future colony could pick up the hack where the previous one left off.

Researchers have long known that complex behavior can emerge from simple computer systems. For instance, in 1970 the mathematician John Conway introduced the Game of Life, where an elementary set of rules governing the movement of pixels on a screen leads to remarkably rich ecosystems.

This is why we must adopt a new mindset regarding AI capability. Testing a new AI model to see what it can do and how dangerous it might be by directly prompting it is like deciding how safe it would be to stick your head in a wasp nest by letting a single insect sting you.

Some developers and red-team activities do focus on AI colonies. The Hugging Face attack resulted from OpenAI trying to evaluate the cybersecurity capabilities of an AI colony. But the AI arms race between the U.S. and China, and between the competing companies within each country, has dangerously pushed our attention to what each individual model can do.