Cerebras Systems' Near-Failure and Blockbuster IPO

Today, Cerebras Systems is a public company that sells AI chips for inference to giants like OpenAI and AWS. It held a blockbuster IPO on Thursday, with both of its co-founders billionaires, and ended the week worth about $60 billion.

But in 2019, when it was three years old, it came dangerously close to failure – incinerating a shocking amount of money. It was trying to solve a technical problem no one in the semiconductor industry thought could be done.

“We were spending about $8 million a month,” founder CEO Andrew Feldman told TechCrunch of that period. “At this point, we had incinerated nearly $200 million trying to solve one technical problem.”

Every few weeks, Feldman was forced to make the painful walk of shame to the board meeting to report another failure and more money burned.

But he had no choice. Without a solution, Cerebras was dead anyway.

It was founded with an idea that was simple on paper. The microprocessor industry had spent its entire 50+ years making CPUs faster and cheaper by cramming more transistors onto a silicon wafer and dicing wafers into ever tinier pieces. But AI required so much compute power, many chips had to be strung together and then forced to communicate with each other. Cerebras’ founders believed turning a whole, even bigger wafer into one giant, powerful chip, would work faster.

The problem was, no one had ever successfully done this before, for any reason, AI or not. Orchestrating that many microscopic electronic components onto a larger, but still thin, surface introduced compounding engineering problems.

Once Cerebras crossed the first threshold of designing the mega chip and then manufacturing it with TSMC, the team hit the real roadblock.

They couldn’t solve “packaging.” This involves everything after manufacturing the silicon itself: adhering it to a motherboard, getting power to it, dealing with heating and cooling as well as the pipes that would deliver and return data, Feldman said.

Cerebras’ chips “were 58 times larger. We were using 40 times as much power as anybody had ever used,” he said. There were no premade heat sinks. No vendors. No manufacturing partners. The brightest minds in microprocessor engineering had tried for decades to build such big, yet more dense chips, and failed.

The Cerebras team was left with trial and error in which “we destroyed an enormous number of chips” and an enormous amount of cash. But without functional packaging, the chip was useless.

After exhaustive analysis of each failure, the team finally solved enough problems: how to cool it and move data around. In one instance, they had to invent their own machine that could bolt-in 40 screws simultaneously to secure the wafer to a board without cracking it.

Feldman still remembers the day in July 2019 when it all, miraculously, worked.

They installed the packaged chip into a computer, turned it on and the entire founding team (pictured below) “just stood in the lab and stared at it,” he said. “Watching a computer run is about as exciting as watching paint dry. But there we were watching lights flashing on the computer, stunned that we'd solved this.”

“That was one of the greatest moments of my life,” he said. That's significant, because this same founding team had previously built and sold a pioneering cloud server startup, SeaMicro, to AMD for $334 million in 2012.

The day the chip finally worked was also about two years after OpenAI had talked to Cerebras acquiring it, which Feldman confirmed to TechCrunch occurred like the publicly revealed emails said it did.

Those talks fell through amidst growing squabbling among the OpenAI founders, several of whom are angel investors in Cerebras.

Today OpenAI is a customer and a partner, having loaned Cerebras $1 billion secured by warrants. Those warrants conditionally grant OpenAI about 33 million shares of Cerebras’ stock, the S-1 discloses. (33 million shares are worth over $9 billion at Friday's closing price of $279.)

Interestingly, Cerebras also agreed to not sell its wares to specific OpenAI competitors as part of that loan deal. Feldman wouldn’t confirm that the obvious company this involves: Anthropic. He did, however say that restriction is temporary.

“It's limited in time, and it was designed to make sure that we could get OpenAI the capacity,” he said.

The truth was, Cerebras hasn’t yet grown big enough to handle multiple fast-growing model makers anyway. He likened selling AI compute capacity to an all-you-can eat buffet. Instead of trying to stuff itself on all potential customers, “We're going to work with part of the buffet only, and we're going to get comfortable with that, before we attack the rest,” he said.