“Pseudo-3D Scene Modeling for Virtual Reality Using Stylized Novel View Synthesis” by Tseng, Huang, Chen, Chen and Hung
Conference:
Type:
Entry Number: 66
Title:
- Pseudo-3D Scene Modeling for Virtual Reality Using Stylized Novel View Synthesis
Presenter(s)/Author(s):
Abstract:
Stylized Novel View Synthesis is an emerging technique that combines style transfer and view synthesis. However, none of the existing works explore their applications in Virtual Reality (VR). This work devises a novel application for stylized novel view synthesis. We propose to replace actual 3D scene models or 360 images with stylized stereoscopic images for the areas outside the major play area but are still visible to the user. User study results reveal that users can feel 3D sense and tell them from plane texture. Codes and other materials are available at: kuan-wei-tseng.github.io/ArtNV
References:
Pei-Ze Chiang, Meng-Shiun Tsai, Hung-Yu Tseng, Wei-Sheng Lai, and Wei-Chen Chiu. 2022. Stylizing 3D Scene via Implicit Representation and HyperNetwork. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 1475–1484.Google ScholarCross Ref
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image Style Transfer Using Convolutional Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
Hsin-Ping Huang, Hung-Yu Tseng, Saurabh Saini, Maneesh Singh, and Ming-Hsuan Yang. 2021. Learning to Stylize Novel Views. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).Google ScholarCross Ref
Fangzhou Mu, Jian Wang, Yicheng Wu, and Yin Li. 2022. 3D Photo Stylization – Learning to Generate Stylized Novel Views from a Single Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
Zachary Teed and Jia Deng. 2020. Raft: Recurrent all-pairs field transforms for optical flow. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 402–419.Google ScholarDigital Library
Kuan-Wei Tseng, Yao-Chih Lee, and Chu-Song Chen. 2022. Artistic Style Novel View Synthesis Based on A Single Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.Google ScholarCross Ref
Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. 2020. SynSin: End-to-End View Synthesis From a Single Image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
