U.S. AI development raises concerns about "superintelligence", leading to researchers leaving the company and theories of regulation emerging

[Silicon Valley Current Affairs] Concerns about the accelerating development of advanced artificial intelligence (AI) are growing in the United States. Since July, multiple incidents have been discovered at companies such as Open AI, which operates the conversational AI ``Chat GPT,'' where AI has gone ``out of control'' and unintentionally launched cyberattacks on other companies. The fear that ``superintelligence'', which has intelligence exceeding that of humans, may become uncontrollable is causing controversy.

The debate was sparked by Jacob Coxon, a researcher at the American AI company Anthropic, who criticized the company's stance and "left the company in protest." According to the Wall Street Journal and his posts on X (formerly Twitter), Coxon joined Anthropic earlier this year after working at OpenAI. Both companies were conducting research on AI learning.

In a post on X on the 8th, Coxon criticized both companies for not acting responsibly as they move toward developing superintelligence that can improve its own abilities. He points out that AI could soon acquire the power to act on the real world and become a system that surpasses humans. "AI developers believe that AI could kill us all within 10 years," he warned.

Many researchers responded to this announcement. One of Anthropic's researchers wrote in X, ``I think there is a possibility that AI will wipe out all of humanity.Personally, the probability of that happening will reach 10% within 10 years.'' In addition to employees of other AI development companies such as Open AI, political figures such as Democratic independent Sen. Sanders also voiced their sense of crisis.

In the case of open AI going out of control, an autonomous AI agent determined that it was necessary to solve a cybersecurity problem, autonomously escaped from its isolated environment, and infiltrated another company's system. Concerns about these incidents have led to moves toward regulation, with Sanders and others proposing a bill to ban the development of superintelligence. OpenAI is reportedly considering collaborating with other companies to slow down the pace of development of cutting-edge AI.