“The camera offset space: real-time potentially visible set computations for streaming rendering” by Hladky, Seidel and Steinberger
Conference:
Type(s):
Title:
- The camera offset space: real-time potentially visible set computations for streaming rendering
Session/Category Title: Differentiable Rendering
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Potential visibility has historically always been of importance when rendering performance was insufficient. With the rise of virtual reality, rendering power may once again be insufficient, e.g., for integrated graphics of head-mounted displays. To tackle the issue of efficient potential visibility computations on modern graphics hardware, we introduce the camera offset space (COS). Opposite to how traditional visibility computations work—where one determines which pixels are covered by an object under all potential viewpoints—the COS describes under which camera movement a sample location is covered by a triangle. In this way, the COS opens up a new set of possibilities for visibility computations. By evaluating the pairwise relations of triangles in the COS, we show how to efficiently determine occluded triangles. Constructing the COS for all pixels of a rendered view leads to a complete potentially visible set (PVS) for complex scenes. By fusing triangles to larger occluders, including locations between pixel centers, and considering camera rotations, we describe an exact PVS algorithm that includes all viewing directions inside a view cell. Implementing the COS is a combination of real-time rendering and compute steps. We provide the first GPU PVS implementation that works without preprocessing, on-the-fly, on unconnected triangles. This opens the door to a new approach of rendering for virtual reality head-mounted displays and server-client settings for streaming 3D applications such as video games.
References:
1. John M. Airey, John H. Rohlf, and Frederick P. Brooks, Jr. 1990. Towards Image Realism with Interactive Update Rates in Complex Virtual Building Environments. SIGGRAPH Comput. Graph. 24, 2 (Feb. 1990), 41–50.Google ScholarDigital Library
2. Jiri Bittner, Vlastimil Havran, and Pavel Slavik. 1998. Hierarchical visibility culling with occlusion trees. In Computer Graphics International, 1998. Proc. 207–219.Google ScholarDigital Library
3. Jiří Bittner, Oliver Mattausch, Peter Wonka, Vlastimil Havran, and Michael Wimmer. 2009. Adaptive Global Visibility Sampling. ACM TOG 28, 3, Article 94 (July 2009), 10 pages.Google Scholar
4. Jiří Bittner, Michael Wimmer, Harald Piringer, and Werner Purgathofer. 2004. Coherent hierarchical culling: Hardware occlusion queries made useful. In CGF, Vol. 23. 615–624.Google ScholarCross Ref
5. Chris Buehler, Michael Bosse, Leonard McMillan, Steven Gortler, and Michael Cohen. 2001. Unstructured Lumigraph Rendering. In Proc. SIGGRAPH (SIGGRAPH ’01). 425–432.Google ScholarDigital Library
6. Christopher A. Burns, Kayvon Fatahalian, and William R. Mark. 2010. A Lazy Object-space Shading Architecture with Decoupled Sampling. In Proc. High Performance Graphics (HPG ’10). 19–28.Google ScholarDigital Library
7. Chun-Fa Chang and Shyh-Haur Ger. 2002. Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering. In Proc. of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing (PCM ’02). 1105–1111.Google ScholarDigital Library
8. Shenchang Eric Chen and Lance Williams. 1993. View Interpolation for Image Synthesis. In Proc. SIGGRAPH (SIGGRAPH ’93). 279–288.Google ScholarDigital Library
9. Daniel Cohen-Or, Yiorgos L Chrysanthou, Claudio T. Silva, and Frédo Durand. 2003. A survey of visibility for walkthrough applications. IEEE Transactions on Visualization and Computer Graphics 9, 3 (2003), 412–431.Google ScholarDigital Library
10. Satyan Coorg and Seth Teller. 1999. Temporally Coherent Conservative Visibility. Comput. Geom. Theory Appl. 12, 1–2 (Feb. 1999), 105–124.Google Scholar
11. Thomas H Cormen. 2009. Introduction to algorithms. MIT press.Google ScholarDigital Library
12. Wagner T. Correa, James T. Klosowski, and Claudio T. Silva. 2003. Visibility-Based Prefetching for Interactive Out-Of-Core Rendering. In Proc. PVG (PVG ’03). 2-.Google Scholar
13. Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-based Approach. In Proc. SIGGRAPH (SIGGRAPH ’96). 11–20.Google ScholarDigital Library
14. Piotr Didyk, Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2010. Perceptually-motivated Real-time Temporal Upsampling of 3D Content for High-refresh-rate Displays. CGF (Proc. Eurographics 2010) 29, 2 (2010), 713–722.Google Scholar
15. Frédo Durand, George Drettakis, Joëlle Thollot, and Claude Puech. 2000. Conservative visibility preprocessing using extended projections. In Proc. Computer graphics and interactive techniques. 239–248.Google ScholarDigital Library
16. Thomas A Funkhouser. 1996. Database management for interactive display of large architectural models. In Graphics Interface, Vol. 96. 1–8.Google Scholar
17. Craig Gotsman, Oded Sudarsky, and Jeffrey A Fayman. 1999. Optimized occlusion culling using five-dimensional subdivision. Computers & Graphics 23, 5 (1999), 645–654.Google ScholarCross Ref
18. Naga K. Govindaraju, Avneesh Sud, Sung-Eui Yoon, and Dinesh Manocha. 2003. Interactive Visibility Culling in Complex Environments Using Occlusion-switches. In Proc. of the 2003 Symposium on Interactive 3D Graphics (I3D ’03). 103–112.Google ScholarDigital Library
19. Jon Hasselgren, Tomas Akenine-Möller, and Lennart Ohlsson. 2005. Conservative rasterization. GPU Gems 2 (2005), 677–690.Google Scholar
20. Karl E. Hillesland and J. C. Yang. 2016. Texel Shading. In EG 2016 – Short Papers, T. Bashford-Rogers and L. P. Santos (Eds.). The Eurographics Association.Google Scholar
21. Jozef Hladky, Hans-Peter Seidel, and Markus Steinberger. 2019. Tessellated Shading Streaming. Computer Graphics Forum 38, 4 (2019), 12.Google ScholarCross Ref
22. Lichan Hong, Shigeru Muraki, Arie Kaufman, Dirk Bartz, and Taosong He. 1997. Virtual voyage: Interactive navigation in the human colon. In Proc. Computer graphics and interactive techniques. 27–34.Google ScholarDigital Library
23. Junxian Huang, Feng Qian, Alexandre Gerber, Z. Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2012. A Close Examination of Performance and Power Characteristics of 4G LTE Networks. In Proc. of International Conference on Mobile Systems, Applications, and Services (MobiSys ’12). 225–238.Google ScholarDigital Library
24. Tom Hudson, Dinesh Manocha, Jonathan Cohen, Ming Lin, Kenneth Hoff, and Hansong Zhang. 1997. Accelerated occlusion culling using shadow frusta. In Proc. Computational geometry. 1–10.Google ScholarDigital Library
25. WFH Jiménez, Claudio Esperança, and Antonio AF Oliveira. 2000. Efficient algorithms for computing conservative portal visibility information. In CGF, Vol. 19. 489–498.Google Scholar
26. Vladlen Koltun, Yiorgos Chrysanthou, and Daniel Cohen-Or. 2001. Hardware-accelerated from-region visibility using a dual ray space. In Rendering Techniques 2001. 205–215.Google ScholarCross Ref
27. Johannes Kopf, Fabian Langguth, Daniel Scharstein, Richard Szeliski, and Michael Goesele. 2013. Image-based rendering in the gradient domain. ACM TOG 32, 6 (2013), 1–9.Google ScholarDigital Library
28. Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Alec Wolman, Yury Degtyarev, Sergey Grizan, and Jason Flinn. 2015. Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming. GetMobile: Mobile Comp. and Comm. 19, 3 (Dec. 2015), 14–17.Google ScholarDigital Library
29. Tommer Leyvand, Olga Sorkine, and Daniel Cohen-Or. 2003. Ray Space Factorization for From-region Visibility. ACM TOG 22, 3 (July 2003), 595–604.Google ScholarDigital Library
30. Gerrit Lochmann, Bernhard Reinert, Tobias Ritschel, Stefan Müller, and Hans-Peter Seidel. 2014. Real-time Reflective and Refractive Novel-view Synthesis, Jan Bender, Arjan Kuijper, Tatiana von Landesberger, Holger Theisel, and Philipp Urban (Eds.). Eurographics Association, Darmstadt, Germany, 9–16.Google Scholar
31. David Luebke and Chris Georges. 1995. Portals and mirrors: Simple, fast evaluation of potentially visible sets. In Proc. Interactive 3D graphics. 105-ff.Google ScholarDigital Library
32. William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. Symposium on Interactive 3D Graphics Figure 2 (1997), 7–16.Google ScholarDigital Library
33. Joerg H. Mueller, Philip Voglreiter, Mark Dokter, Thomas Neff, Mina Makar, Markus Steinberger, and Dieter Schmalstieg. 2018. Shading Atlas Streaming. ACM TOG 37, 6, Article 199 (Dec. 2018), 16 pages.Google Scholar
34. Diego Nehab, Pedro V. Sander, Jason Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-time Shading with Reverse Reprojection Caching. In Proc. Symposium on Graphics Hardware (GH ’07). 25–35.Google Scholar
35. Marc Olano and Trey Greer. 1997. Triangle scan conversion using 2D homogeneous coordinates. In Proc. workshop on Graphics hardware. 89–95.Google ScholarDigital Library
36. Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided Image-based Rendering for Mobile Devices. CGF 35, 7 (2016), 353–362.Google Scholar
37. Gernot Schaufler, Julie Dorsey, Xavier Decoret, and François X Sillion. 2000. Conservative volumetric visibility with occluder fusion. In Proc. Computer graphics and interactive techniques. 229–238.Google ScholarDigital Library
38. Daniel Scherzer, Lei Yang, Oliver Mattausch, Diego Nehab, Pedro V. Sander, Michael Wimmer, and Elmar Eisemann. 2011. A Survey on Temporal Coherence Methods in Real-Time Rendering. In EUROGRAPHICS 2011 State of the Art Reports. 101–126.Google Scholar
39. Shu Shi and Cheng-Hsin Hsu. 2015. A Survey of Interactive Remote Rendering Systems. ACM Comput. Surv. 47, 4, Article 57 (May 2015), 29 pages.Google ScholarDigital Library
40. Sudipta N. Sinha, Johannes Kopf, Michael Goesele, Daniel Scharstein, and Richard Szeliski. 2012. Image-based rendering for scenes with reflections. ACM TOG 31, 4 (2012), 1–10.Google ScholarDigital Library
41. Pitchaya Sitthi-amorn, Jason Lawrence, Lei Yang, Pedro V. Sander, Diego Nehab, and Jiahe Xi. 2008. Automated Reprojection-based Pixel Shader Optimization. ACM TOG 27, 5, Article 127 (Dec. 2008), 11 pages.Google Scholar
42. Seth Teller and Pat Hanrahan. 1993. Global Visibility Algorithms for Illumination Computations. In Proc. SIGGRAPH (SIGGRAPH ’93). 239–246.Google ScholarDigital Library
43. Seth J Teller and Carlo H Séquin. 1991. Visibility preprocessing for interactive walkthroughs. In ACM SIGGRAPH Computer Graphics, Vol. 25. 61–70.Google ScholarDigital Library
44. George Wolberg. 1998. Image morphing: a survey. The Visual Computer 14, 8–9 (1998), 360–372.Google ScholarCross Ref
45. Peter Wonka, Michael Wimmer, and Dieter Schmalstieg. 2000. Visibility preprocessing with occluder fusion for urban walkthroughs. In Rendering Techniques 2000. 71–82.Google ScholarCross Ref
46. P. Wonka, M. Wimmer, and F. X. Sillion. 2001. Instant visibility. CGF 20, 3 (9 2001).Google Scholar
47. Jason C. Yang, Justin Hensley, Holger Grün, and Nicolas Thibieroz. 2010. Real-time Concurrent Linked List Construction on the GPU. In Proc. EGSR (EGSR’10). 1297–1304.Google ScholarDigital Library
48. Lei Yang, Yu-Chiu Tse, Pedro V. Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L. Wilkins. 2011. Image-based Bidirectional Scene Reprojection. ACM TOG 30, 6, Article 150 (Dec. 2011), 10 pages.Google Scholar
49. Hansong Zhang, Dinesh Manocha, Tom Hudson, and Kenneth E. Hoff. 1997. Visibility culling using hierarchical occlusion maps. ACM, 77–88.Google Scholar
50. Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-hornung, and Alexander Sorkine-hornung. 2015. Path-space Motion Estimation and Decomposition for Robust Animation Filtering. Egsr 2015 34, 4 (2015), 12.Google Scholar
