“Real-time sports video analysis for video content viewing with haptic information” by Takahashi, Azuma, Handa, Ishiwatari, Sano, et al. …
Conference:
Type:
Entry Number: 02
Title:
- Real-time sports video analysis for video content viewing with haptic information
Presenter(s)/Author(s):
Abstract:
Video content mainly conveys visual and audio information, but it would be more immersive by adding haptic information. The sense of presence when watching video could be enhanced through tactile stimuli from haptic devices, enabling video content to provide immersive experiences.
We developed a haptic perception system that can be used at public sports viewing events. The live sports broadcast video is displayed on a large screen in front of audiences, and haptic stimuli that are synchronized with the video are supplied to the audiences in real time.We targeted Judo content because it is a physical sport, and haptic stimuli related to athletes’ bodily motions would improve audiences’ sense of immersiveness.
Figure 1 shows an overview of our system. First, broadcast video is analyzed and type and timing of haptic stimuli, which are related to athletes’ motions are measured using the video-analysis module. This module is key to enabling public viewing with haptic devices because the devices have to be automatically actuated in real time to synchronize with live broadcast content. The haptic devices, which audiences have and sit in, are then automatically controlled via wireless WiFi. Video analysis requires processing time, so the original video is delayed by about one second, and the haptic stimuli and audio/visual information are synchronized.
We also developed two types of haptic devices, i.e., handheld and chair. The handheld device mainly conveys the intensity of athletes’ motions, and the chair device conveys the timing and impact of a specific action event, such as throwing trick and ground fighting. These devices contribute to a viewer understanding the game situation and improving the sense of immersiveness during a public viewing event.
We evaluated the performance of the video-analysis module of our system, and the results indicate that it has sufficient accuracy of action recognition.We demonstrated our system at an exhibition event in June 2021, and plan to put it to practical use at a live sports public viewing event in July 2021. To the best of our knowledge, this live public viewing event will be the first attempt at using such a system.
