Artificial Intelligence Experts Warn the Risk of Superintelligence

Coxon suggested that artificial intelligence systems will surpass human capabilities in the near future, being able to infiltrate computer systems and gain power and resources.

“Do not underestimate the power of this technology,” Coxon said, adding, “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and gain real power and resources.”

Coxon also said that people who develop artificial intelligence truly believe that this technology can bring about the end of humanity by 2030.

A response to Coxon's statement came from Evan Hubinger, who leads research at Anthropic on the operation of artificial intelligence systems in harmony with human goals and values.

"Jacob is right here. We really believe that artificial intelligence can kill all humans. I personally think the probability of this happening in the next 10 years is over 10 percent," Hubinger said.

Sharing this rate as his personal estimate, Hubinger stated that he believes Anthropic is doing its best, but that they do not yet have a plan to solve the problem that will enable superintelligence to work in harmony with humans. Hubinger said it's not clear they're on track to solve this problem.

Emphasizing in another post that the risk arising from existing artificial intelligence models is low, Hubinger explained that his main concern is the super intelligence that may arise from the self-development process.

"Iterative self-improvement", which the researchers draw attention to, means that artificial intelligence systems improve themselves with very little human intervention and repeat this process with the new abilities they acquire.

This mechanism has not been implemented yet. However, artificial intelligence laboratories are working towards this goal.

The statements came at a time when Anthropic and OpenAI were raising large amounts of investment and were on the agenda with preparations for public offerings. There was no response from either company at first stage to CNBC's request for comment.