“MR360 Interactive: Playing with Digital Creatures in 360° Videos”
Conference:
Experience Type(s):
Title:
- MR360 Interactive: Playing with Digital Creatures in 360° Videos
Description:
We present “MR360 Interactive”, interactive mixed reality (MR) experiences using both pre-recorded and live streaming 360° panoramic videos (360-video) shown in head mounted displays. We developed the MR360 toolkit, which provides an interface to create interactive MR content using 360-video. The toolkit detects salient lights in the 360-video and casts realistic shadows. Image based lighting is perceptually optimized to provide fast, believable results. Real-time differential rendering is used to obtain a composition between the virtual objects and the real-world background. We present two applications of our toolkit: a VR experience using pre-recorded 360-video, and the MR Stage, an MR experience using live streaming 360-video. In both applications, participants are interacting with digital creatures with high presence in 360-video.
References:
[1] Andrew Chalmers, Jong Jin Choi, and Taehyun Rhee. 2014. Perceptually optimised illumination for seamless composites. Pacific Graphics, The Eurographics Association (2014).
[2] Paul Debevec. 1998. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’98). ACM, New York, NY, USA, 189–198.
[3] Thomas Iorns and Taehyun Rhee. 2016. Real-Time Image Based Lighting for 360-Degree Panoramic Video. In Revised Selected Papers of the PSIVT 2015 Workshops on Image and Video Technology – LNCS, Volume 9555. Springer-Verlag New York, Inc., New York, NY, USA, 139–151.
[4] Taehyun Rhee, Lohit Petikam, Benjamin Allen, and Andrew Chalmers. 2017. MR360: Mixed Reality Rendering for 360° Panoramic Videos. IEEE Transactions on Visualization and Computer Graphics, Presented in IEEE Virtual Reality (2017).
