[After meeting him] Are you creating a god and using him as a slave?... Superintelligence AI control debate

“Trying to control AI that is smarter than humans is like trying to create God and enslave him. This is impossible” (Eliezer Judkoski, co-founder of the Machine Intelligence Research Institute (MIRI))

“To completely stop AI development, a dictatorial system that monitors the entire world is inevitable. Trying to avoid the dangers of AI is creating another totalitarian nightmare” (Professor Philip Coralus, University of Oxford)

In the era of alternative intelligence that threatens human reasoning and problem-solving abilities, will humanity lose control and become extinct, or will it remain the master of its own thoughts?

On the 4th, I met with Researcher Yudkosky and Professor Coralus, who visited Korea to attend the SBS D Forum held at Dongdaemun Design Plaza (DDP) in Seoul.

◇ Yudkosky “Superintelligence AI, development should be stopped, not delayed”

A researcher at the Yudkosky Machine Intelligence Research Institute (MIRI) called for an immediate halt to the development of 'superhuman intelligence'.

He said, “Alignment research that wants AI to stay in a sandbox, a controlled experimental environment, is progressing too slowly, and it is unclear whether it will ever reach the right answer. In the AI ​​arms race, the only winner is AI.”

Researcher Yudkosky pointed out that a distinction must be made between specialized AI, such as AlphaFold, which predicts protein structures, and superintelligence AI, which has general-purpose reasoning capabilities beyond that of humans.

He warned, “If an AI designed to cure cancer annihilates humanity instead of curing cancer, it is not a good deal at all. If the AI ​​model continues to increase its intelligence in a situation where it gets out of control, death will be the natural result.”

Regarding Korea's role, he emphasized that the global semiconductor supply chain should be used as an active weapon.

Researcher Yudkoski said, "If an international agreement is pursued, Korea should declare, 'If other countries stop, we will not develop superhuman AI,'" and argued, "Rather than focusing on selling as many semiconductors as possible at the highest price, we should be prepared to use the leverage of key supply chains such as high-bandwidth memory (HBM) to obtain an international agreement and impose export controls."

◇ Coralus “Forced cessation of development is totalitarianism… Question-and-answer equilibrium is needed.”

On the other hand, Oxford University professor Coralus refuted that a complete halt to AI development is not only unrealistic but could also drive humanity into an authoritarian dystopia.

Professor Coralus said, "To stop AI development, we need an extreme authoritarian control system to monitor and track development around the world. If we completely stop development around the world, wouldn't the person holding the control power become another totalitarian nightmare?"

Professor Coralus diagnosed that modern people are caught between the loss of the ability to act, which means losing competitiveness by not using AI, and the loss of autonomy, which involves handing over all decisions to AI.

The third narrow path he proposed is a question-and-answer equilibrium that confirms one's unique value through various questions and counterarguments.

He proposed a cognitive firewall as a solution to a reporter's question, "Users want to outsource complex thinking to AI, so will Big Tech create a question-and-answer equilibrium model that asks questions?"

Professor Coralus said, "If the free service model where consumers become products is maintained like past platforms, there will be no incentive to strengthen users' judgment. There is a demand to use high-performance models even if they pay high subscription fees, and the computational power of the large model and the layer that controls the user's exploration process can be separated."

He predicted, "It will be possible to create a structure where the own AI layer running on a personal laptop acts as a cognitive firewall between external commercial services and the user's brain and protects autonomy."

He advised, “The AI ​​era must be the era of philosopher developers,” and “Before we can make AI systems do useful things, we must first clearly define what values ​​and perspectives we truly pursue.”