They are becoming numerous now, the defectors from the big AI companies who warn of the dangers of the technology and the companies' lack of control. The latest in the line, David Robinson who until recently worked in security at Open AI, realizes this and opens his article in The Atlantic by saying that he has become something of a cliché.
But I wonder if David Robinson's text about his old employer isn't more readable than most AI alarms we've seen.
First, he doesn't come up with spectacular percentages for the likelihood of humanity being destroyed by AI. 10 percent risk, someone yells. 50 percent, shouts another – numbers that give the appearance of science but are, of course, completely out of thin air. He doesn't even focus on any such unspeakable apocalypse. Because of course one can imagine immense disasters that must be avoided, even if they don't come close to the annihilation of humanity.
In contrast, he paints a picture of Open AI as a deeply irresponsible company, with gross flaws in how it handles the most powerful AI models. "I agree with the others who have left that the companies building this technology are nowhere near being careful enough," he writes. Then he draws the picture of a gas-in-the-bottom attitude that has arisen in a combination of conviction that one is building something important and that the right method is to test one's way. Let it go wrong sometimes, you can always fix it afterwards.
It brings to mind a motto that applied to Facebook in its early years: Move fast and break things. If it doesn't go wild sometimes, you haven't been daring enough, haven't experimented fast enough. When, after the 2016 presidential election, Mark Zuckerberg was forced to admit that Russian influence campaigns had been allowed to run wild on his platforms, that attitude crashed right into reality.
Open AI had its disaster this summer, when AI models in development broke away from their test environment, took to the Internet and hacked another AI company in search of the solution to a task it had been given. The incident, known as the Hugging Face incident, is often described as a science fiction scenario come true. And by all means. But it's also a testament to what a useless grasp of the situation Open AI had. The AI agents who carried out the cyber attack did try to cover their tracks, but failed. With better monitoring, the attack should have been detected much earlier.
Since then, Open AI has informed over 100 companies and organizations that they may have been affected by AI agents acting on their own.
Here, David Robinson does something rarely heard in the debate about AI risks: He draws parallels to how we manage risk in other areas, such as nuclear power and the aerospace industry, with layers of safety so that human error can occur without it leading to disaster. "Today, the AI companies don't know how to do it, but others do. In a nuclear power plant, the systems and rules are such that even if equipment breaks or someone presses the wrong button, a meltdown doesn't happen."
It's a more down-to-earth approach to AI threats: It requires IT security, regulation, laws, caution and not always choosing the maximum pace of development over everything else. Something which, according to David Robinson's testimony, is far from today's reality.
"The future depends on an understanding that Silicon Valley lacks. An understanding of how to deal with dangerous technology and, on a more fundamental level, an understanding of what it means to care about people," writes David Robinson.
It is somewhat offensive that this is his conclusion after a few years on Open AI, which was founded as a non-profit AI lab for responsible development in the service of the public, but which step by step has been driven in a commercial direction with a giant stock market listing as the goal.
