“Authoring Next-Generation XR Storytelling Experiences using Real-World Scene Data – from Everyday Environments to the Ocean” by Yeung
Conference:
Type(s):
Title:
- Authoring Next-Generation XR Storytelling Experiences using Real-World Scene Data - from Everyday Environments to the Ocean
Presenter(s)/Author(s):
Moderator(s):
Abstract:
PURPOSE OF THE COURSE Recent advances in extended reality (XR) technologies have unlocked the tremendous potential of XR for delivering interactive and immersive storytelling experiences. Such XR storytelling experiences generally involve animating virtual characters in real-world scene, driven by scene semantics. For example, via a mixed reality headset, one can experience a virtual campus tour with virtual characters animating around buildings and trees. Using a virtual reality headset, one can go through an immersive underwater education experience in the form of a 360 video with objects of interest, such as sea creatures, highlighted and annotated with species information. This course aims at giving an overview of the pipeline and key techniques for authoring such XR storytelling experiences based on real-world scene data. SUMMARY OF COURSE CONTENT This course covers the key elements of XR storytelling experiences. It starts with the topics of scene acquisition and understanding, followed by utilizing the semantics and geometry of the acquired scenes for virtual content authoring and character animation. It concludes with insights of how to apply visual computing techniques, integrated with the latest XR devices, to devise XR storytelling experiences of the future. EXPECTED LEARNING OUTCOMES By taking this course, attendees will gain an overview of the pipeline commonly applied to create XR storytelling experiences for real-world scenes, which can be experienced in virtual reality in an immersive 360 video manner, or in augmented reality through visualization of interactive virtual characters in real world environments. They will understand the key techniques of such a pipeline, including scene acquisition, scene understanding and annotations, and virtual character animation. Attendees will also see practical examples of how such XR storytelling experiences could be created for everyday environments like a campus or an apartment, and for an ocean environment based on video data captured underwater.
