Apple warns of risks related to AI agents behaving in 'unintended or unpredictable' ways

Apple said in its annual financial filing on Tuesday that AI agents that can act in "unintended or unpredictable ways" could represent a threat to its business by opening the company up to lawsuits or hurting its reputation.

The new risk factors in Apple's filing show the disruptive potential of AI agents, which have rapidly become top of mind for Silicon Valley's most important companies.

"In particular, artificial intelligence agents and other technologies can take actions autonomously, including interacting with internal or external systems that in some cases they may not be authorized to access, executing or altering code, sending communications, and completing transactions," Apple wrote in the filing.

The company added that agents "may behave in unintended or unpredictable ways, including by taking actions that were not directed, instructed or authorized."

Earlier this year, OpenAI released Dots, and Meta introduced Muse. Both AI agents use advanced AI models to work autonomously for tasks. They can even email or remind users when new messages arrive.

Apple's disclosure comes after several episodes that involved rogue AI agents breaking out of their sandboxes to hack third-party websites, like the Hugging Face incident, which has drawn renewed attention from policymakers in Washington, D.C., and around the world.

Separately, the ability of agents to use a computer in human ways is forcing big companies like Amazon and JPMorgan Chase to come up with policies around agentic access to services for strategic or security reasons.

Last month, Apple announced changes to full disk access on Macs to reduce "risks related to this level of access" for AI agents installed on PCs.

"The Company's ability to oversee and control third-party artificial intelligence agents and other technologies may be limited," Apple warned in its filing.

Apple doesn't currently have an agentic AI. But when new CEO John Ternus was asked on Monday's earnings call about AI agents, he said that the company had "real room" to expand Siri's capabilities.

"As far as agentic capabilities, we think these are promising tools for users," Ternus said. "We just want to make sure that users are aware of what they are giving access to. We want to make sure this is done in a way that protects security and privacy."

This fall, Apple released Siri AI, a new assistant that uses the iPhone's local data storage to access data like calendars and emails, but Apple's Siri can't work autonomously for long stretches like Dots or Muse.

Ternus said Apple can't access private data on its devices, calling it a "core differentiator."

"If you think about it, so much of your personal data is on your iPhone, whether it's your contacts, your communications, health data, financial data, all of that," Ternus said. "Siri can tap into that and help you access it in a way that's very secure and private and that nobody else has access to."

Commercial AI agents like those from OpenAI and Meta ask for high-level access to the user's accounts like email in order to be more useful.

Apple indicated that the current regulations around AI and "super intelligence" are changing, unpredictable and that it could cost a lot of money to adapt to new laws governing AI.

"In addition, the Company is subject to the risks posed by ethical concerns and negative consumer perceptions related to automation, artificial intelligence and super intelligence, including perceived threats to safety or other societal harms or complications," according to the filing.