“Deep Real-time Volumetric Rendering Using Multi-feature Fusion” by Hu, Yu, Liu, Yan, Wu, et al. …
Conference:
Type:
Title:
- Deep Real-time Volumetric Rendering Using Multi-feature Fusion
Session/Category Title: Real-time Rendering: Gotta Go Fast!
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present Multi-feature Radiance-Predicting Neural Networks (MRPNN), a practical framework with a lightweight feature fusion neural network for rendering high-order scattered radiance of participating media in real time. By reformulating the Radiative Transfer Equation (RTE) through theoretical examination, we propose transmittance fields, generated at a low cost, as auxiliary information to help the network better approximate the RTE, drastically reducing the size of the neural network. The light weight network efficiently estimates the difficult-to-solve in-scattering term and allows for configurable shading parameters while improving prediction accuracy. In addition, we propose a frequency-sensitive stencil design in order to handle non-cloud shapes, resulting in accurate shadow boundaries. Results show that our MRPNN is able to synthesize indistinguishable output compared to the ground truth. Most importantly, MRPNN achieves a speedup of two orders of magnitude compared to the state-of-the-art, and is able to render high-quality participating material in real time.
References:
1. James Arvo. 1993. Transfer equations in global illumination. Global Illumination, SIGGRAPH ‘93 Course Notes 2 (1993).
2. Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph. 36, 4 (2017), 97:1–97:14.
3. George EP Box and David R Cox. 1964. An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological) 26, 2 (1964), 211–243.
4. Xi Deng, Shaojie Jiao, Benedikt Bitterli, and Wojciech Jarosz. 2019. Photon surfaces for robust, unbiased volumetric density estimation. ACM Trans. Graph. 38, 4 (2019), 46:1–46:12.
5. Robert A Drebin, Loren Carpenter, and Pat Hanrahan. 1988. Volume rendering. ACM Siggraph Computer Graphics 22, 4 (1988), 65–74.
6. Sebastian Herholz, Yangyang Zhao, Oskar Elek, Derek Nowrouzezahrai, Hendrik P. A. Lensch, and Jaroslav Krivánek. 2019. Volume path guiding based on zero-variance random walk theory. ACM Trans. Graph. 38, 3 (2019), 25:1–25:19.
7. Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132–7141.
8. Wojciech Jarosz, Derek Nowrouzezahrai, Robert Thomas, Peter-Pike J. Sloan, and Matthias Zwicker. 2011. Progressive photon beams. ACM Trans. Graph. 30, 6 (2011), 181:1–181:12.
9. Wojciech Jarosz, Matthias Zwicker, and Henrik Wann Jensen. 2008. The beam radiance estimate for volumetric photon mapping. In ACM SIGGRAPH 2008 classes. 1–112.
10. Henrik Wann Jensen, Stephen R. Marschner, Marc Levoy, and Pat Hanrahan. 2001. A practical model for subsurface light transport. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2001, Los Angeles, California, USA, August 12-17, 2001. ACM, 511–518.
11. James T. Kajiya and Brian Von Herzen. 1984. Ray tracing volume densities. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1984, Minneapolis, Minnesota, USA, July 23-27, 1984. ACM, 165–174.
12. Simon Kallweit, Thomas Müller, Brian McWilliams, Markus H. Gross, and Jan Novák. 2017. Deep scattering: rendering atmospheric clouds with radiance-predicting neural networks. ACM Trans. Graph. 36, 6 (2017), 231:1–231:11.
13. Anton Kaplanyan and Carsten Dachsbacher. 2010. Cascaded light propagation volumes for real-time indirect illumination. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 99–107.
14. David Koerner, Jamie Portsmouth, Filip Sadlo, Thomas Ertl, and Bernd Eberhardt. 2014. Flux-limited diffusion for multiple scattering in participating media. Computer Graphics Forum 33, 6 (2014), 178–189.
15. Jaroslav Krivánek, Iliyan Georgiev, Toshiya Hachisuka, Petr Vévoda, Martin Sik, Derek Nowrouzezahrai, and Wojciech Jarosz. 2014. Unifying points, beams, and paths in volumetric light transport simulation. ACM Trans. Graph. 33, 4 (2014), 103:1–103:13.
16. Eric P Lafortune and Yves D Willems. 1996. Rendering participating media with bidirectional path tracing. In Eurographics Workshop on Rendering Techniques. Springer, 91–100.
17. Ludwig Leonard, Kevin Höhlein, and Rüdiger Westermann. 2021. Learning multiple‐scattering solutions for sphere‐tracing of volumetric subsurface effects. Computer Graphics Forum 40, 2 (2021), 165–178.
18. Liangchen Luo, Yuanhao Xiong, Yan Liu, and Xu Sun. 2019. Adaptive gradient methods with dynamic bound of learning rate. In Proceedings of the 7th International Conference on Learning Representations. New Orleans, Louisiana.
19. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: representing scenes as neural radiance fields for view synthesis. In Computer Vision – ECCV 2020 – 16th European Conference(Lecture Notes in Computer Science, Vol. 12346). Springer, 405–421.
20. Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-time neural radiance caching for path tracing. ACM Trans. Graph. 40, 4 (2021), 36:1–36:16.
21. Mikhail Panin and Sergey I. Nikolenko. 2019. Faster RPNN: rendering clouds with latent space light probes. In SIGGRAPH Asia 2019 Technical Briefs, SA 2019, Brisbane, QLD, Australia, November 17-20, 2019. ACM, 21–24.
22. Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. In Eurographics Workshop on Rendering Techniques. Springer, 11–22.
23. Jos Stam. 1995. Multiple scattering as a diffusion process. In Eurographics Workshop on Rendering Techniques. Springer, 41–50.
24. Eric Veach. 1998. Robust Monte Carlo Methods for Light Transport Simulation. Ph. D. Dissertation. Stanford University, Stanford, CA, USA. Advisor(s) Guibas, Leonidas J. AAI9837162.
25. D. Vicini, V. Koltun, and W. Jakob. 2019. A learned shape-adaptive subsurface scattering model. ACM Trans. Graph. 38, 4 (2019), 127:1–127:15.
26. E Woodcock, T Murphy, P Hemmings, and S Longworth. 1965. Techniques used in the GEM code for Monte Carlo neutronics calculations in reactors and other systems of complex geometry. In Proc. Conf. Applications of Computing Methods to Reactor Problems, Vol. 557. 557–579.
27. M. Wrenninge, N. B. Zafar, A. Bouthors, J. Tessendorf, and G. Graham. 2011. Production Volume Rendering: Systems. In ACM SIGGRAPH (Courses).
28. In-Kwon Yeo and Richard A Johnson. 2000. A new family of power transformations to improve normality or symmetry. Biometrika 87, 4 (2000), 954–959.
29. Arno Zinke, Cem Yuksel, Andreas Weber, and John Keyser. 2008. Dual scattering approximation for fast multiple scattering in hair. ACM Trans. Graph. 27, 3 (2008), 32:1–32:10.
