[AI Prism] “As AI agents become smarter, inference infrastructure becomes the key.”

The importance of memory and inference efficiency is highlighted in the proliferation of AI agents.

Physical AI, lack of data and high reliability standards are technical challenges

(Seoul = Yonhap News) Reporter Hayoung Kwon = It is predicted that as artificial intelligence (AI) agents become more sophisticated, the importance of inference infrastructure to run them quickly and efficiently will increase.

While open source AI models stand out in the cost-performance competition, it has also been pointed out that Korean models are failing to enter the major competitive landscape.

These infrastructure efficiency challenges are not limited to software agents. Analysis suggests that physical AI models, which serve as the brains of robots, may also face similar problems in the process of processing complex information and performing tasks.

Jeon Byeong-gon, CEO of Friendly AI, said at the 'Physical AI Tech Day' held at COEX in Seoul on the 14th, "As AI agents become smarter, the key will be how quickly and efficiently they operate."

◇ The importance of inference efficiency is highlighted in the proliferation of AI agents.

CEO Jeon defined this year as the era of AI agents that make their own decisions and perform actions. In particular, citing the coding field as an example, he predicted that the proportion of work currently handled by developers is around 50%, but this will decrease to 10% next year and the rest will be performed by agents.

The technical basis supporting the spread of such AI agents was the improvement in the performance of open models that are disclosed to the outside and can be used by anyone.

CEO Jeon explained that the open model has recently begun to show high performance. When comparing the cost-performance ratio of each model in a benchmark that evaluates software development capabilities, open models form the 'Pareto Frontier', an optimal curve that delivers high performance at low cost.

However, he pointed out that this competitive landscape is mainly dominated by Chinese models, and that no Korean model has yet been named.

Memory burden was cited as a technological challenge that is emerging as a result of the spread of AI agents.

Unlike general chatbots, agents must continue to include previous conversations and work history in new input values ​​as they continue to work, so the amount of information to be processed increases exponentially.

In fact, the amount of input information for a single processing of the coding agent reaches 116,000 tokens (the minimum unit for AI to process text), but the resulting output is only 184 tokens, showing a large imbalance between input and output.

This accumulated work content is stored in a temporary storage space called 'Key-Value (KV) Cache' and is loaded and utilized whenever necessary. However, as the work history becomes longer, the required memory capacity increases, which causes the problem that the number of agents that can be processed simultaneously decreases.

To solve this problem, the industry is increasing the efficiency of models and systems by using the 'Sparse Attention' technique, which selects and calculates only certain important sections rather than all previous records, and compression techniques that reduce the amount of stored information itself.

CEO Jeon added that Friendly AI's self-developed 'Continuous Batching' technology, which bundles and processes requests from multiple users, has now become an industry standard technology.

CEO Jeon predicted that these challenges will be repeated in physical AI as well. In order for a robot to understand the situation like a human and continue work through multiple steps, a model with a structure similar to that of current AI agents is needed. As image and sensor information increases, the memory burden inevitably increases, just like a software agent.

◇ Physical AI, securing data and work reliability are key

In the subsequent presentation, technological challenges were also highlighted along with the enthusiasm for investment in the physical AI industry.

Kim Sang-bae, professor and meta-robotics architect at the Massachusetts Institute of Technology (MIT), who participated via video, recently divided AI technology into 'virtual intelligence', which deals with digital data such as text, images, and sounds on the web, and 'physical intelligence', which moves the body in reality.

He explained that language and image data are created by humans and are relatively easy to secure, but physical information such as touch, force, and movement handled by robots does not exist on the Internet, so it is difficult to collect them.

The difficulty in securing data is directly related to the reliability of the robot. Professor Kim said, “In the field of virtual intelligence, a 99% success rate is great, but a 99% success rate is useless for robots,” emphasizing that it is difficult for robots to tolerate mistakes in each task.

He explained that while Tesla's autonomous driving data has accumulated about 15 million miles (about 375 million hours), 'Manipulation' data, which deals with robot hand movements, is now only about a year's worth.

In addition to the difference in data scale, the degree of freedom, or number of directions in which a robot arm can move, is much greater than that of a car, and the purpose of work changes every moment, so the difficulty of the problem to be solved technically is high.

◇ Government strengthens support for physical AI verification and commercialization

Based on this AI evolution trend, the government plans to strengthen support for the entire industry to create a physical AI ecosystem.

Ryu Je-myeong, Second Vice Minister of Science and ICT, said, “Physical AI is one of the three mega projects that will lead the future of Korea, along with semiconductors and AI data centers,” and emphasized, “It is a key technology in the AI ​​revolution that increases productivity and safety in various industries such as manufacturing, logistics, agriculture, medical care, and mobility, and creates new products and services.”

He continued, “As it cannot be completed by just one technology or company, close cooperation between industry, academia, and research is important,” and added, “We will actively support industrial application and expansion so that excellent technologies can lead to field verification and commercialization, not just R&D.”

Park Yun-gyu, president of the National IT Industry Promotion Agency (NIPA), said, “If the AI ​​industry has been focusing on developing better technologies and verifying their usability, now is the time to go beyond verification and focus on implementation.”

He said, "Based on our country's world-class manufacturing capabilities, excellent AI technology, and abundant industrial sites, we will expand the use of physical AI and lead to strengthening industrial competitiveness and creating practical results."

Meanwhile, this event was organized as part of 'Physical AI Tech Week' and followed the 'Physical AI Alliance Day' held the previous day. Experts from industry, academia, and research at home and abroad participated to share global technology trends and future development directions for physical AI and discuss ways to cooperate to strengthen domestic technological competitiveness and expand the industry.