“Shading atlas streaming” – ACM SIGGRAPH HISTORY ARCHIVES

“Shading atlas streaming”

  • 2018 SA Technical Papers_Mueller_Shading atlas streaming

Conference:


Type(s):


Title:

    Shading atlas streaming

Session/Category Title:   Acquisition, rendering and display for virtual reality

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Streaming high quality rendering for virtual reality applications requires minimizing perceived latency. We introduce Shading Atlas Streaming (SAS), a novel object-space rendering framework suitable for streaming virtual reality content. SAS decouples server-side shading from client-side rendering, allowing the client to perform framerate upsampling and latency compensation autonomously for short periods of time. The shading information created by the server in object space is temporally coherent and can be efficiently compressed using standard MPEG encoding. Our results show that SAS compares favorably to previous methods for remote image-based rendering in terms of image quality and network bandwidth efficiency. SAS allows highly efficient parallel allocation in a virtualized-texture-like memory hierarchy, solving a common efficiency problem of object-space shading. With SAS, untethered virtual reality headsets can benefit from high quality rendering without paying in increased latency.

References:


    1. John M. Airey, John H. Rohlf, and Frederick P. Brooks, Jr. 1990. Towards Image Realism with Interactive Update Rates in Complex Virtual Building Environments. In Proceedings ACM I3D. 41–50. Google ScholarDigital Library
    2. Tomas Akenine-Möller and Timo Aila. 2005. Conservative and Tiled Rasterization Using a Modified Triangle Set-Up. Journal of Graphics Tools 10, 3 (Jan 2005), 1–8.Google ScholarCross Ref
    3. Magnus Andersson, Jon Hasselgren, Robert Toth, and Tomas Akenine-Möiler. 2014. Adaptive texture space shading for stochastic rendering. Computer Graphics Forum 33, 2 (may 2014), 341–350. Google ScholarDigital Library
    4. Dan Baker. 2016. Object Space Lighting. Talk at Game Developers Conference. (2016). http://www.cogsci.rpi.edu/~destem/gamearch/gdc16/Object-Space-Lighting-Rev-21.pptxGoogle Scholar
    5. Paul Bao and Douglas Gourlay. 2004. Remote walkthrough over mobile networks using 3-D image warping and streaming. IEE Proceedings – Vision, Image and Signal Processing 151, 4 (Aug 2004), 329–336.Google ScholarCross Ref
    6. Louis Bavoil and Miguel Sainz. 2009. Multi-layer dual-resolution screen-space ambient occlusion. In SIGGRAPH 2009: Talks. ACM, 45. Google ScholarDigital Library
    7. Kevin Boos, David Chu, and Eduardo Cuervo. 2016. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proc. MobiSys. 291–304. Google ScholarDigital Library
    8. Mario Botsch, Andreas Wiratanaya, and Leif Kobbelt. 2002. Efficient High Quality Rendering of Point Sampled Geometry. In Proceedings of the 13th Eurographics Workshop on Rendering. 53–64. Google ScholarDigital Library
    9. Huw Bowles, Kenny Mitchell, Robert W. Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative Image Warping. Computer Graphics Forum 31, 2pt1 (2012), 237–246.Google Scholar
    10. Chris Buehler, Michael Bosse, Leonard McMillan, Steven Gortler, and Michael Cohen. 2001. Unstructured Lumigraph Rendering. In Proceedings SIGGRAPH. 425–432. Google ScholarDigital Library
    11. Christopher A. Burns, Kayvon Fatahalian, and William R. Mark. 2010. A Lazy Object-space Shading Architecture with Decoupled Sampling. In Proceedings HPG. 19–28. Google ScholarDigital Library
    12. Nathan A. Carr and John C. Hart. 2002. Meshed atlases for real-time procedural solid texturing. ACM Transactions on Graphics 21, 2 (apr 2002), 106–131. Google ScholarDigital Library
    13. Cem Cebenoyan. 2014. Real Virtual Texturing Taking Advantage of DirectX11.2 Tiled Resources. Game Developer Conference. (2014).Google Scholar
    14. Matthäus Chadjas, Christian Eisenacher, Marc Stamminger, and Sylvain Lefebvre. 2010. Virtual Texture Mapping 101. (2010).Google Scholar
    15. Chun-Fa Chang and Shyh-Haur Ger. 2002. Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering. Springer Berlin Heidelberg, Berlin, Heidelberg, 1105–1111. Google ScholarDigital Library
    16. Ka Chen. 2015. Adaptive Virtual Texture Rendering in Far Cry 4. Talk at Game Developers Conference. (March 2015). http://twvideo01.ubm-us.net/o1/vault/gdc2015/presentations/Chen_Ka_AdaptiveVirtualTexture.pdfGoogle Scholar
    17. Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the Latency of Cloud Gaming Systems. In Proceedings of the 19th ACM International Conference on Multimedia (MM ’11). 1269–1272. Google ScholarDigital Library
    18. Shenchang Eric Chen and Lance Williams. 1993. View interpolation for image synthesis. In Proceedings SIGGRAPH. 279–288. Google ScholarDigital Library
    19. Sharon. Choy, Bernard. Wong, Gwendal. Simon, and Catherine. Rosenberg. 2012. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Workshop on Network and Systems Support for Games (NetGames). 1–6. Google ScholarDigital Library
    20. Petrik Clarberg, Robert Toth, Jon Hasselgren, Jim Nilsson, and Tomas Akenine-Möller. 2014. AMFS: Adaptive Multi-Frequency Shading for Future Graphics Processors. ACM Transactions on Graphics 33, 4 (jul 2014), 1–12. Google ScholarDigital Library
    21. Petrik Clarberg, Robert Toth, and Jacob Munkberg. 2013. A sort-based deferred shading architecture for decoupled sampling. ACM Transactions on Graphics 32, 4 (jul 2013). Google ScholarDigital Library
    22. Daniel Cohen-Or, Yair Mann, and Shachar Fleishman. 1999. Deep Compression for Streaming Texture Intensive Animations. In Proceedings SIGGRAPH. 261–267. Google ScholarDigital Library
    23. Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. 2015. High-quality Streamable Free-viewpoint Video. ACM Transactions on Graphics 34, 4 (July 2015). Google ScholarDigital Library
    24. Robert L. Cook, Loren Carpenter, and Edwin Catmull. 1987. The Reyes Image Rendering Architecture. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’87). ACM, New York, NY, USA, 95–102. Google ScholarDigital Library
    25. Cyril Crassin, David Luebke, Michael Mara, Morgan McGuire, Brent Oster, Peter Shirley, Peter-Pike Sloan, and Chris Wyman. 2015. CloudLight: A System for Amortizing Indirect Lighting in Real-Time Rendering. Journal of Computer Graphics Techniques (JCGT) 4, 4 (15 October 2015), 1–27. http://jcgt.org/published/0004/04/01/Google Scholar
    26. Eduardo Cuervo, Alec Wolmany, Landon P. Coxz, Kiron Lebeck, Ali Razeenz, Stefan Saroiuy, and Madanlal Musuvathi. 2015. Kahawai: High-Quality Mobile Gaming Using GPU Offload. In Proceedings MobiSys. 121–135. Google ScholarDigital Library
    27. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2010. Adaptive Image-space Stereo View Synthesis. In 15th International Workshop on Vision, Modeling and Visualization Workshop. Siegen, Germany, 299–306.Google Scholar
    28. Karl. E. Hillesland and J. C. Yang. 2016. Texel Shading. In Proceedings of the 37th Annual Conference of the European Association for Computer Graphics: Short Papers. 73–76. Google ScholarDigital Library
    29. Martin Kraus and Thomas Ertl. 2002. Adaptive Texture Maps. In Proceedings ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware. 7–15. Google ScholarDigital Library
    30. Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime – Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. Google ScholarDigital Library
    31. Sylvain Lefebvre, Jerome Darbon, and Fabrice Neyret. 2004. Unified Texture Management for Arbitrary Meshes. INRIA Research report 5210. (2004).Google Scholar
    32. Marc Levoy. 1995. Polygon-assisted JPEG and MPEG Compression of Synthetic Images. In Proceedings SIGGRAPH. 21–28. Google ScholarDigital Library
    33. Gábor Liktor and Carsten Dachsbacher. 2012. Decoupled deferred shading for hardware rasterization. In Proceedings ACM I3D. 143–150. Google ScholarDigital Library
    34. Yair Mann and Daniel Cohen-Or. 1997. Selective Pixel Transmission for Navigating in Remote Virtual Environments. Computer Graphics Forum 16 (1997), C201–C206.Google ScholarCross Ref
    35. Marc Manzano, José A. Hernandez, Manuel Uruena, and Eusebi Calle. 2012. An empirical study of Cloud Gaming. In 2012 11th Annual Workshop on Network and Systems Support for Games (NetGames). 1–2. Google ScholarDigital Library
    36. William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proceedings of the 1997 Symposium on Interactive 3D Graphics. Google ScholarDigital Library
    37. Colt McAnlis. 2009. Halo Wars: The Terrain of Next-Gen. Talk at Game Developers Conference. (2009).Google Scholar
    38. Martin Mittring. 2007. Finding next gen: Cryengine 2. In ACM SIGGRAPH 2007 courses. ACM, 97–121. Google ScholarDigital Library
    39. Martin Mittring. 2008. Advanced Virtual Texture Topics. In Advances in Real-Time Rendering in 3D Graphics and Games Course SIGGRAPH 2008. 23–51. http://portal.acm.org/citation.cfm?id=1404435.1404438 Google ScholarDigital Library
    40. Yuval Noimark and Daniel Cohen-Or. 2003. Streaming Scenes to MPEG-4 Video-Enabled Devices. IEEE Computer Graphics and Applications 23 (01 2003), 58–64. Google ScholarDigital Library
    41. OculusVR. 2018. Rendering to the Oculus Rift. (2018). https://developer.oculus.com/documentation/pcsdk/latest/concepts/dg-render/ Visited on March 30, 2018.Google Scholar
    42. Dawid Pajak, Robert Herzog, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. Scalable Remote Rendering with Depth and Motion-flow Augmented Streaming. Computer Graphics Forum 30, 2 (2011), 415–424.Google ScholarCross Ref
    43. Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, and Frédo Durand. 2011. Decoupled sampling for graphics pipelines. ACM Transactions on Graphics 30, 3 (may 2011), 1–17. Google ScholarDigital Library
    44. Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided Image-based Rendering for Mobile Devices. Computer Graphics Forum 35, 7 (oct 2016), 353–362. Google ScholarDigital Library
    45. Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered Depth Images. In Proceedings SIGGRAPH. 231–242. Google ScholarDigital Library
    46. Bin Sheng, Wei-Liang Meng, Han-Qiu Sun, and En-Hua Wu. 2011. MCGIM-Based Model Streaming for Realtime Progressive Rendering. Journal of Computer Science and Technology 26, 1 (jan 2011), 166–175. Google ScholarDigital Library
    47. Shu Shi and Cheng-Hsin Hsu. 2015. A Survey of Interactive Remote Rendering Systems. ACM Comput. Surv. 47, 4, Article 57 (May 2015). Google ScholarDigital Library
    48. Shu Shi, Klara Nahrstedt, and Roy Campbell. 2012. A Real-time Remote Rendering System for Interactive Mobile Graphics. ACM Trans. Multimedia Comput. Commun. Appl. 8, 3s, Article 46 (Oct. 2012), 20 pages. Google ScholarDigital Library
    49. Markus Steinberger, Michael Kenzel, Bernhard Kainz, and Dieter Schmalstieg. 2012. ScatterAlloc: Massively parallel dynamic memory allocation for the GPU. In 2012 Innovative Parallel Computing (InPar). 1–10.Google Scholar
    50. Christopher C. Tanner, Christopher J. Migdal, and Michael T. Jones. 1998. The Clipmap: A Virtual Mipmap. In Proceedings SIGGRAPH. 151–158. Google ScholarDigital Library
    51. Eyal Teler and Dani Lischinski. 2001. Streaming of Complex 3D Scenes for Remote Walkthroughs. Computer Graphics Forum 20, 3 (2001), 17–25.Google ScholarCross Ref
    52. J. M. P. van Waveren. 2009. id Tech 5 Challenges – From Texture Virtualization to Massive Parallelization. In SIGGRAPH 2009 Course: Beyond Programmable Shading.Google Scholar
    53. Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13 (apr 2004), 600–612. Issue 4. Google ScholarDigital Library
    54. Lei Yang, Yu-Chiu Tse, Pedro V. Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L. Wilkins. 2011. Image-based bidirectional scene reprojection. In Proceedings of the 2011 SIGGRAPH Asia Conference. Article 150. Google ScholarDigital Library
    55. Ilmi Yoon and Ulrich Neumann. 2000. Web-Based Remote Rendering with IBRAC (Image-Based Rendering Acceleration and Compression). Computer Graphics Forum 19, 3 (2000), 321–330.Google ScholarCross Ref
    56. Cem Yuksel. 2017. Mesh Color Textures. In Proceedings of High Performance Graphics (HPG ’17). Article 17, 11 pages. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org