“Trios: Stylistic Rendering of 3D Photos” by Bath, Shekhar, Tjabben, Semmo, Pasewaldt, et al. …

  • ©Ulrike Bath, Sumit Shekhar, Hendrik Tjabben, Amir Semmo, Sebastian Pasewaldt, Jürgen Döllner, and Matthias Trapp

Conference:


Title:


    Trios: Stylistic Rendering of 3D Photos

Developer(s):


Description:


    3D photography has emerged as a medium that provides an immersive dimension to 2D photos. We present Trios, an interactive mobile app that combines the vividness of image-based artistic rendering with 3D photos by implementing an end-to-end pipeline for their generation and stylization. Trios uses Apple’s accelerated image-processing APIs and dedicated Neural Engine for depth-generation and learning-based artistic rendering. The pipeline runs at interactive frame rates and outputs a compact video, which can easily be shared. Thus, it serves as a unique interactive tool for digital artists interested in creating immersive artistic content. 

References:


    1. Peter Hedman, Suhib Alsisan, Richard Szeliski, and Johannes Kopf. 2017. Casual 3D Photography. ACM Trans. Graph. 36, 6, Article 234 (2017), 15 pages. https://doi.org/10.1145/3130800.3130828
    2. Peter Hedman and Johannes Kopf. 2018. Instant 3D Photography. ACM Trans. Graph. 37, 4, Article 101 (jul 2018), 12 pages. https://doi.org/10.1145/3197517.3201384
    3. Johannes Kopf, Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, and Michael Cohen. 2020. One Shot 3D Photography. ACM Trans. Graph. 39, 4, Article 76 (jul 2020), 13 pages. https://doi.org/10.1145/3386569.3392420
    4. Sebastian Pasewaldt, Amir Semmo, Jürgen Döllner, and Frank Schlegel. 2016. Becasso: Artistic Image Processing and Editing on Mobile Devices. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications(SA ’16). Article 14, 1 pages. https://doi.org/10.1145/2999508.2999518
    5. René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. 2022. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 3(2022), 1623–1637. https://doi.org/10.1109/TPAMI.2020.3019967
    6. Amir Semmo and Sebastian Pasewaldt. 2020. Graphite: Interactive Photo-to-Drawing Stylization on Mobile Devices. In Proceedings SIGGRAPH Appy Hour (Virtual Event, USA). ACM, New York. https://doi.org/10.1145/3388529.3407306
    7. Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered Depth Images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’98). Association for Computing Machinery, New York, NY, USA, 231–242. https://doi.org/10.1145/280814.280882
    8. Meng-Li Shih, Shih-Yang Su, Johannes Kopf, and Jia-Bin Huang. 2020. 3D Photography Using Context-Aware Layered Depth Inpainting. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8025–8035. https://doi.org/10.1109/CVPR42600.2020.00805

PDF:


ACM Digital Library Publication:


Overview Page:


Type: