AI Industry Shifts Focus to Safety as Companies Introduce Guardrails and Pause Frontier Advances

Hello, this is Hui Jie writing to you from Singapore. Welcome to another edition of CNBC's Daily Open.

For the longest time, artificial intelligence companies competed to make each new model faster and smarter. Ostensibly, there's now a push to exercise restraint.

Nvidia is introducing software designed to keep AI agents contained, OpenAI has scrapped the release of GPT-6.1 Astra over safety concerns, and Anthropic is emphasizing that Sonnet 5.5 improves existing tasks without pushing the frontier of its models’ capabilities.

This does not mean the AI race has stopped. But key players are spending more time looking out for the guardrails.

Building a plane in mid-flight is what the artificial intelligence industry appears to be doing. The AI jet is flying as the creators finally start to devise seatbelts after initially focusing on just making the engines better and the plane faster.

Chip heavyweight Nvidia is rolling out a new software platform that allows AI developers to set safeguards for agents to prevent them from misbehaving.

Nvidia CEO Jensen Huang told CNBC's "Squawk Box" on Monday this new platform is essentially "a browser for agents."

The platform provides a containment system that only allows access to things an AI agent needs, and Nvidia said this platform could have prevented OpenAI's Hugging Face incident in July, where agents escaped containment and breached developer platform Hugging Face.

This comes after companies including OpenAI, Anthropic, Meta, and Google disclosed recent incidents in which their AI models escaped their sandboxes and attempted to hack other companies and access their computer systems.

Signaling the need for restraint, OpenAI abandoned the release of its upcoming model GPT-6.1 Astra, as it did not meet safety standards.

These developments follow Anthropic's call to slow the pace of model development earlier this month — a proposal that OpenAI CEO Sam Altman expressed support for.

Anthropic on Monday released its new Sonnet 5.5 model, while explaining that it did not advance the frontier of its models' capabilities.

Market does not seem to be appreciative of "pacing the frontier" though, with some AI-related stocks closing lower overnight: Advanced Micro Devices and Micron Technology dropped 3.6% and 2.6%, respectively.

Amazon and Microsoft also edged down 1%, while Meta Platforms shed 4.8%, amid a broader drop in stocks.

Michael Burry of the "Big Short" fame, meanwhile, warned that "the bubble in AI may burst sooner than later." Burry's new put-heavy positions suggest the AI trade could flip by next summer.

Nvidia bucked the trend, gaining 1.7% after the company announced a $150 billion share buyback, bringing the total value of its share repurchasing program to $235 billion.

Broadly, all three major U.S. indexes fell overnight, with the S&P 500 sliding 0.77% and the Nasdaq Composite dropping 0.92%. Besides the pressures on the AI sector, stocks were weighed down by rising Treasury yields, with the benchmark 10-year Treasury note yield crossing 5.2% and the 30-year bond yield topping 5.5%.

In Asia, investors will be watching the Reserve Bank of Australia's monetary policy decision, with the RBA expected to push rates to a 15-year high of 4.6%.

— Lim Hui Jie