“Teleport to the Augmented Real-World with Live Interactive Effects (IFX)” by Rhee, Chalmers, Hoh, Roberts, Butcher, et al. …
Conference:
Type(s):
Title:
- Teleport to the Augmented Real-World with Live Interactive Effects (IFX)
Developer(s):
Description:
Augmented telepresence provides rich communication for people at a distance with interactive blended information between the virtual and real world [Rhee et al. 2017, 2020; Young et al. 2022]. We push the boundaries of augmented telepresence with a novel live media technology, including live capturing, modeling, blending, and interactive effects (IFX) to augment telepresence. Using our technology, people at a distance can connect and communicate with creative storytelling, augmented with novel IFX.
We achieve this with the following breakthroughs: 1) digitizing remote spaces and people in real-time, 2) transmitting digitized information across a network, 3) augmenting remote telepresence using real-time visual effects and interactive storytelling with live-blending of 3D virtual assets into the digitized real-world.
In this presentation, we will unveil several new technologies and novel IFX that can enrich telepresence, including:
• Real-time 360° RGBD video capturing: we will demonstrate capturing 360° RGBD videos using a 360° RGB camera and LiDAR sensor, including synchronization between the RGB and depth streams as well as depth map generation.
• IFX with live RGBD videos: we will demonstrate real-time blending of 3D virtual objects into the live 360° RGBD videos, showcasing real-time occlusion and collision handling.
• 6-degrees of freedom (DoF) tele-movement: we introduce our recent research [Chen et al. 2022] for volumetric environment capturing and 6-DoF navigation. We will demonstrate real-time navigation (movement and rotation) in captured real surroundings (beyond room scales).
We will showcase applications (Figure 1) where we can virtually teleport to and explore within a live stream of the augmented real world and communicate remotely with live IFX.
PDF:
Overview Page:
Submit a story:
If you would like to submit a story about this experience or presentation, please contact us: historyarchives@siggraph.org
