Transformation of AI data center competition logic: from hardware competition to Token production efficiency

The competitive logic of the AI ​​Data Center (AIDC) is changing. The Institute of Industrial Intelligence (MIC) pointed out that as AI inference and application accelerates commercialization, the standard for measuring the efficiency of AI infrastructure is shifting from focusing on the number of GPUs and chip theoretical computing power (FLOPS) in the past to further shifting to the token output per watt and the cost per unit token (CPM) under fixed power supply. This means that AIDC competition is gradually shifting from hardware to "computing power production efficiency."

The Information Policy Council MIC recently held a 2026 MIC FORUM seminar. Experts pointed out at the seminar that token production capacity and liquidity are rewriting the AIDC competition logic, and "heterogeneous computing integration" and "cabinet division of labor" will become the key solutions to improve the bottleneck of token output.

This change is also related to the rapid development of agent AI (Agentic AI). Chen Yiling, a senior industry analyst at MIC, pointed out that agent-based AI requires repeated inference, tool calls, and long-context access, which makes the AI ​​inference bottleneck further shift from pure chip computing power to memory capacity and bandwidth, CPU-GPU interconnection, and data movement efficiency of the overall system.

In order to improve the efficiency of agent-based AI computing, computing platforms have begun to turn to "heterogeneous integration", extending from the chip all the way to the system level, allowing different computing components to perform their respective duties and operate collaboratively, creating a heterogeneous computing power pool, thereby improving resource utilization, hoping to maximize Token production at the lowest cost and highest utilization under limited power conditions.

In addition to heterogeneous computing, AIDC's cabinet architecture may also see a clearer "division of labor" in the future. MIC stated that in the future, AIDC will be composed of cabinets with different roles such as high computing power, large-capacity cache, and high-speed storage. The asset utilization management of heterogeneous cabinets will replace the competition of single hardware specifications.

Senior industry analyst Chen Yi-ling said that this competition from chip performance to system efficiency will also bring upgrade opportunities for Taiwanese manufacturers. Taiwan has established itself as a hub in AI server manufacturing. Facing the trend of heterogeneous computing and customized ASIC (Application Special Integrated Circuit), Taiwanese manufacturers need to have cross-platform rapid introduction, co-design and rapid mass production capabilities. The next stage of value improvement will lie in extending from hardware manufacturing to solutions that can improve system-level token efficiency.