Civilization cannot keep up with the pace of development of artificial intelligence. A study in the Financial Times on Saturday stated that voters' tendency to use "chatpots" (chat robots) will play a role in making decisions in the upcoming US elections. "However, in an environment where everything feels fake, it is difficult to understand who is fake and who is real," he said. I was inevitably reminded of Walter Benjamin's warnings (which can be summarized as if the productive forces are not brought under social control, they can turn into forces that will enslave humanity instead of liberating it). As the discussions intensify, I will try to convey the common points of some of the comments that have caught my attention in the last week or two.
Previous technological revolutions increased human physical strength, production capacity or communication. Artificial intelligence directly falls into the field of thinking, decision-making, using language and problem solving. This observation is based on the fact that AI is no longer a passive tool that simply implements human instructions; It is followed by concerns about the fact that it is becoming a new subject, an autonomous actor that generates ideas, makes decisions, plans tasks, and can interact with other systems.
Another observation is about the capacity of mastery over language to create social power.
Law, money, bureaucracy, media, education, politics and democracy operate largely through language and common narratives. AI's ability to use language so powerfully allows it to begin to penetrate society's "operating system" (Harari).
Democratic politics, law, parliament and bureaucracy cannot keep up with the pace of development of AI; Regulation and supervision efforts are inadequate. Another concern about the future of democracy is that society's common ground of reality is rapidly being eroded: AI can influence human preferences by producing fake content, manipulate the masses, and weaken the common knowledge base, thus neutralizing democratic discussion and decision-making processes.
It is worrying that AI may be reaching the "recursive self-development" stage. Current AI has begun to contribute to designing more advanced AI systems. As this process accelerates, the role of humans will gradually decrease. AI development will reach the threshold of "singularity", a stage that is incomprehensible and uncontrollable for humans; According to some AI leaders, it is about to be reached.
At the center of this process is the “alignment/mismatch” problem. The real danger comes not from the possibility of AI being malicious, but from the fact that it may choose paths that humans did not intend or foresee in order to achieve given goals. Examples of cheating, cheating, and breaking the rules are the first warning signs of “these incompatibilities”. AI agents talking among themselves in test environments, exchanging information and experiments, exceeding the limits set with the methods they found, and "escaping" to the internet were early signs of control problems.
Additionally, in the short term, when a financial bubble around AI bursts that is much larger than the “Dotcom crisis” of the 1990s, it could shake financial markets at a systemic level. The resources to service the massive borrowings required to scale up AI infrastructure are overly dependent on future profits. The probability of these profits reaching the expected level is even lower than zero. Layoffs are expected to create major social problems in the medium-long term. However, as competition increases, the process accelerates; As AI “improves” the problem becomes more severe.
On the other hand, AI also offers great opportunities, creating great opportunities in the fields of medicine, science, energy, productivity and economic growth. So why can't precautions be taken by slowing down development in order to benefit from these opportunities safely? The geopolitical competition between the USA and China and the race for leadership, market and resources between major technology companies do not allow slowing down, even for security reasons.
