(Ulsan = Yonhap News) Reporter Kim Yong-tae = A technology has been developed that can evaluate the performance of an artificial intelligence (AI) model that extracts highlights from sports game videos.
On the 13th, the Ulsan National Institute of Science and Technology (UNIST) announced that Professor Kim Tae-hwan's team at the Graduate School of Artificial Intelligence developed 'SVHighlights' (Sport Video Highlights), a large-scale benchmark that evaluates AI models for extracting sports video highlights.
A benchmark is a type of common test that compares which AI is better at getting the right answer. It must have not only the problems that the AI must solve, but also the answers to judge its performance.
Most videos included in existing benchmarks are around 2 to 4 minutes long. In order to create the correct answer, a person had to watch the video from beginning to end and individually mark which sections were the highlights, which was a limitation as the cost and time of the work rapidly increased as the video became longer.
On the other hand, the benchmark created by the research team consists of 320 videos from 8 sports, including soccer, baseball, basketball, volleyball, American football, ice hockey, rugby, and racing, and is massive, with a cumulative total of over 640 hours.
The average length of one video is 2 hours, which is 30 to 60 times longer than existing datasets.
The human role in building such a massive benchmark was limited to marking the start and end points of the game once per video and scanning the automatically paired screens as grid images. The error rate during this verification process was only 0.18%.
The research team was able to cost-effectively create a long and massive dataset by using official sports highlight videos already released on the Internet. This is because the highlight scenes selected by professional editors themselves serve as reliable answers.
The problem was that there was no timestamp information about 'minutes and seconds' of the original highlight video, but the research team created a matching algorithm to automatically find this.
This is an algorithm that compares the frames of the original video and highlight video on a pixel-by-pixel basis to pair the most similar scenes.
In addition, when designing the algorithm, not only the similarity of the screen but also the temporal sequence with the previously found scene was considered. This is to prevent errors that incorrectly match visually identical replay scenes with highlights instead of the actual time of the game.
The research team also developed an AI model called 'TF-SELECTOR' that can effectively extract highlights from long videos by combining the existing scene segmentation model, speech recognition model, vision language model, and large-scale language model.
When tested with the SVHighlights benchmark, the research team explained that it performed better than existing models in most indicators.
Professor Kim Tae-hwan said, "By using highlights already created by broadcasters, we have replaced the work that humans had to do for hours to answer the question. In the same way, we can continue to increase data, which will help us objectively evaluate video analysis models over a long period of time and develop models with better performance."
The results of this study, supported by the Ministry of Science and ICT and the Korea IT Planning and Evaluation Institute, were announced at 'ACM KDD', an international conference in the field of data mining held in Jeju on the 9th of last month.
