“A learned shape-adaptive subsurface scattering model” by Vicini, Koltun and Jakob
Conference:
Type:
Title:
- A learned shape-adaptive subsurface scattering model
Session/Category Title: Machine Learning for Rendering
Presenter(s)/Author(s):
Abstract:
Subsurface scattering, in which light refracts into a translucent material to interact with its interior, is the dominant mode of light transport in many types of organic materials. Accounting for this phenomenon is thus crucial for visual realism, but explicit simulation of the complex internal scattering process is often too costly. BSSRDF models based on analytic transport solutions are significantly more efficient but impose severe assumptions that are almost always violated, e.g. planar geometry, isotropy, low absorption, and spatio-directional separability. The resulting discrepancies between model and usage lead to objectionable errors in renderings, particularly near geometric features that violate planarity.This article introduces a new shape-adaptive BSSRDF model that retains the efficiency of prior analytic methods while greatly improving overall accuracy. Our approach is based on a conditional variational autoencoder, which learns to sample from a reference distribution produced by a brute-force volumetric path tracer. In contrast to the path tracer, our autoencoder directly samples outgoing locations on the object surface, bypassing a potentially lengthy internal scattering process.The distribution is conditional on both material properties and a set of features characterizing geometric variation in a neighborhood of the incident location. We use a low-order polynomial to model the local geometry as an implicitly defined surface, capturing curvature, thickness, corners, as well as cylindrical and toroidal regions. We present several examples of objects with challenging medium parameters and complex geometry and compare to ground truth simulations and prior work.
References:
1. Martin Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Jia Yangqing, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mane, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viegas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/Google Scholar
2. Adam Arbree, Bruce Walter, and Kavita Bala. 2011. Heterogeneous Subsurface Scattering Using the Finite Element Method. IEEE Transactions on Visualization and Computer Graphics 17, 7 (July 2011), 956–969. Google ScholarDigital Library
3. Steve Bako, Thijs Vogels, Brian McWilliams, Mark Meyer, Jan Novák, Alex Harvill, Pradeep Sen, Tony DeRose, and Fabrice Rousselle. 2017. Kernel-predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Trans. Graph. (Proc. SIGGRAPH) 36, 4 (jul 2017), 97:1–97:14. Google ScholarDigital Library
4. Chakravarty R Alla Chaitanya, Anton S Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Trans. Graph. 36, 4 (jul 2017), 98:1–98:12. Google ScholarDigital Library
5. Subrahmanyan Chandrasekhar. 1960. Radiative transfer. Dover publications, New York.Google Scholar
6. Per Christensen, Julian Fong, Jonathan Shade, Wayne Wooten, Brenden Schubert, Andrew Kensler, Stephen Friedman, Charlie Kilpatrick, Cliff Ramshaw, Marc Bannister, Brenton Rayner, Jonathan Brouillat, and Max Liani. 2018. RenderMan: An Advanced Path-Tracing Architecture for Movie Rendering. ACM Trans. Graph. 37, 3, Article 30 (Aug. 2018), 21 pages. Google ScholarDigital Library
7. Eugene d’Eon. 2012. A better dipole. http://www.eugenedeon.com/project/a-better-dipole/Google Scholar
8. Eugene D’Eon and Geoffrey Irving. 2011. A Quantized-diffusion Model for Rendering Translucent Materials. ACM Trans. Graph. (Proc. SIGGRAPH 2011) 30, 4 (jul 2011), 56:1–56:14. Google ScholarDigital Library
9. Eugene d’Eon, David Luebke, and Eric Enderton. 2007. Efficient Rendering of Human Skin. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (EGSR’07). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 147–157. Google ScholarDigital Library
10. Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density estimation using Real NVP. arXiv:cs.LG/1605.08803Google Scholar
11. Carl Doersch. 2016. Tutorial on Variational Autoencoders. stat/1606 (2016), 23. arXiv:1412.6980 https://arxiv.org/abs/1606.05908Google Scholar
12. Craig Donner and Henrik Wann Jensen. 2005. Light Diffusion in Multi-layered Translucent Materials. In ACM Trans. Graph. (Proc. SIGGRAPH) (SIGGRAPH ’05). ACM, New York, NY, USA, 1032–1039. Google ScholarDigital Library
13. Craig Donner, Jason Lawrence, Ravi Ramamoorthi, Toshiya Hachisuka, Henrik Wann Jensen, and Shree Nayar. 2009. An Empirical BSSRDF Model. ACM Trans. Graph. 28, 3, Article 30 (July 2009), 10 pages. Google ScholarDigital Library
14. Luca Fascione, Johannes Hanika, Mark Leone, Marc Droske, Jorge Schwarzhaupt, Tomáš Davidovič, Andrea Weidlich, and Johannes Meng. 2018. Manuka: A Batch-Shading Architecture for Spectral Path Tracing in Movie Production. ACM Trans. Graph. 37, 3, Article 31 (Aug. 2018), 18 pages. Google ScholarDigital Library
15. Roald Frederickx and Philip Dutré. 2017. A Forward Scattering Dipole Model from a Functional Integral Approximation. ACM Trans. Graph. 36, 4, Article 109 (July 2017), 13 pages. Google ScholarDigital Library
16. Jeppe Revall Frisvad, Toshiya Hachisuka, and Thomas Kim Kjeldsen. 2014. Directional Dipole Model for Subsurface Scattering. ACM Trans. Graph. 34, 1, Article 5 (Dec. 2014), 12 pages. Google ScholarDigital Library
17. Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. http://eigen.tuxfamily.org.Google Scholar
18. Ralf Habel, Per H Christensen, and Wojciech Jarosz. 2013. Photon Beam Diffusion: A Hybrid Monte Carlo Method for Subsurface Scattering. Computer Graphics Forum (Proceedings of EGSR) 32, 4 (jun 2013), 27–37. Google ScholarDigital Library
19. Pat Hanrahan and Wolfgang Krueger. 1993. Reflection from Layered Surfaces Due to Subsurface Scattering. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’93). ACM, New York, NY, USA, 165–174. Google ScholarDigital Library
20. Louis G. Henyey and Jesse L. Greenstein. 1941. Diffuse radiation in the galaxy. The Astrophysical Journal 93 (1941), 70–83.Google ScholarCross Ref
21. Pedro Hermosilla, Sebastian Maisch, Tobias Ritschel, and Timo Ropinski. 2018. Deep-learning the Latent Space of Light Transport. arXiv:cs.GR/1811.04756Google Scholar
22. Akira Ishimaru. 1999. Wave propagation and scattering in random media. Vol. 12. John Wiley & Sons.Google Scholar
23. Wenzel Jakob. 2010. Mitsuba renderer.Google Scholar
24. Wenzel Jakob, Adam Arbree, Jonathan T Moon, Kavita Bala, and Steve Marschner. 2010. A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 4 (2010), 53:1–53:13. Google ScholarDigital Library
25. Wenzel Jakob, Marco Tarini, Daniele Panozzo, and Olga Sorkine-Hornung. 2015. Instant Field-Aligned Meshes. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia) 34, 6 (Nov. 2015), 189:1–189:15. Google ScholarDigital Library
26. Henrik Wann Jensen and Juan Buhler. 2002. A Rapid Hierarchical Rendering Technique for Translucent Materials. ACM Trans. Graph. 21, 3 (July 2002), 576–581. Google ScholarDigital Library
27. Henrik Wann Jensen and Per H. Christensen. 1998. Efficient Simulation of Light Transport in Scenes with Participating Media Using Photon Maps. (1998), 311–320. Google ScholarDigital Library
28. Henrik Wann Jensen, Stephen R. Marschner, Marc Levoy, and Pat Hanrahan. 2001. A Practical Model for Subsurface Light Transport. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01). ACM, New York, NY, USA, 511–518. Google ScholarDigital Library
29. Jorge Jimenez, Veronica Sundstedt, and Diego Gutierrez. 2009. Screen-space perceptual rendering of human skin. ACM Transactions on Applied Perception 6, 4, Article 23 (2009), 15 pages. Google ScholarDigital Library
30. James T. Kajiya. 1986. The Rendering Equation. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’86). ACM, New York, NY, USA, 143–150. Google ScholarDigital Library
31. James T. Kajiya and Brian P. Von Herzen. 1984. Ray Tracing Volume Densities. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’84). ACM, New York, NY, USA, 165–174. Google ScholarDigital Library
32. Simon Kallweit, Thomas Müller, Brian McWilliams, Markus Gross, and Jan Novák. 2017. Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 36, 6 (nov 2017), 231:1–231:11. Google ScholarDigital Library
33. Benjamin Keinert, Henry Schäfer, Johann Korndörfer, Urs Ganse, and Marc Stamminger. 2014. Enhanced Sphere Tracing. In Smart Tools and Apps for Graphics – Eurographics Italian Chapter Conference, Andrea Giachetti (Ed.). The Eurographics Association.Google Scholar
34. Alan King, Christopher Kulla, Alejandro Conty, and Marcos Fajardo. 2013. BSSRDF Importance Sampling. In ACM SIGGRAPH 2013 Talks (SIGGRAPH ’13). ACM, New York, NY, USA, Article 48, 1 pages. Google ScholarDigital Library
35. Diederik P Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6 (2014), 15. arXiv:1412.6980 http://arxiv.org/abs/1412.6980Google Scholar
36. Diederik P. Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. CoRR abs/1312.6114 (2013), 14.Google Scholar
37. Konstantin Kolchin. 2010. Surface Curvature Effects on Reflectance from Translucent Materials. CoRR abs/1010.2623 (2010), 4. arXiv:1010.2623 http://arxiv.org/abs/1010.2623Google Scholar
38. Jaroslav Křivánek and Eugene d’Eon. 2014. A Zero-variance-based Sampling Scheme for Monte Carlo Subsurface Scattering. In ACM SIGGRAPH 2014 Talks (SIGGRAPH ’14). ACM, New York, NY, USA, Article 66, 1 pages. Google ScholarDigital Library
39. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (2015), 436–444.Google ScholarCross Ref
40. Johannes Meng, Johannes Hanika, and Carsten Dachsbacher. 2016. Improving the Dwivedi Sampling Scheme. Computer Graphics Forum (Proceedings of Eurographics Symposium on Rendering) 35, 4 (2016), 37–44.Google Scholar
41. Michael I Mishchenko, Larry D Travis, and Andrew A Lacis. 2006. Multiple scattering of light by particles: radiative transfer and coherent backscattering. Cambridge University Press.Google Scholar
42. Ashish Myles, Nico Pietroni, and Denis Zorin. 2014. Robust Field-aligned Global Parametrization. ACM Trans. Graph. 33, 4, Article 135 (July 2014), 14 pages. Google ScholarDigital Library
43. Thomas Müller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. 2018. Neural Importance Sampling. arXiv:cs.LG/1808.03856Google Scholar
44. Oliver Nalbach, Elena Arabadzhiyska, Dushyant Mehta, Hans-Peter Seidel, and Tobias Ritschel. 2017. Deep Shading: Convolutional Neural Networks for Screen-Space Shading. Computer Graphics Forum (Proc. EGSR 2017) 36, 4 (2017), 65–78. Google ScholarDigital Library
45. Srinivasa G. Narasimhan, Mohit Gupta, Craig Donner, Ravi Ramamoorthi, Shree K. Nayar, and Henrik Wann Jensen. 2006. Acquiring scattering properties of participating media by dilution. ACM Trans. Graph. 25, 3 (2006), 1003–1012. Google ScholarDigital Library
46. Jan Novák, Derek Nowrouzezahrai, Carsten Dachsbacher, and Wojciech Jarosz. 2012. Virtual Ray Lights for Rendering Scenes with Participating Media. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 31, 4, Article 60 (jul 2012), 11 pages. Google ScholarDigital Library
47. Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically Based Rendering: From Theory to Implementation (3rd ed.) (3rd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. 1266 pages. Google ScholarDigital Library
48. Simon Premože, Michael Ashikhmin, and Peter Shirley. 2003. Path Integration for Light Transport in Volumes. In Proceedings of the 14th Eurographics Workshop on Rendering (EGRW ’03). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 52–63. http://dl.acm.org/citation.cfm?id=882404.882413 Google ScholarDigital Library
49. Chen Shen, James F. O’Brien, and Jonathan R. Shewchuk. 2004. Interpolating and Approximating Implicit Surfaces from Polygon Soup. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (Aug. 2004), 896–904. Google ScholarDigital Library
50. Jos Stam. 1995. Multiple scattering as a diffusion process. In Rendering Techniques ’95. Springer Vienna, Vienna, 41–50.Google Scholar
51. Jerry Tessendorf. 1987. Radiative transfer as a sum over paths. Physical review A 35, 2 (1987), 872.Google Scholar
52. Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Trans. Graph. 37, 4, Article 124 (July 2018), 15 pages. Google ScholarDigital Library
53. Bruce Walter, Pramook Khungurn, and Kavita Bala. 2012. Bidirectional Lightcuts. ACM Trans. Graph. 31, 4, Article 59 (July 2012), 11 pages. Google ScholarDigital Library
54. Jiaping Wang, Shuang Zhao, Xin Tong, Stephen Lin, Zhouchen Lin, Yue Dong, Baining Guo, and Heung-Yeung Shum. 2008. Modeling and Rendering of Heterogeneous Translucent Materials Using the Diffusion Equation. ACM Trans. Graph. 27, 1, Article 9 (March 2008), 18 pages. Google ScholarDigital Library
55. Douglas R. Wyman, Michael S. Patterson, and Brian C. Wilson. 1989. Similarity relations for the interaction parameters in radiation transport. Appl. Opt. 28, 24 (Dec 1989), 5243–5249.Google ScholarCross Ref
56. Shuang Zhao, Ravi Ramamoorthi, and Kavita Bala. 2014. High-order Similarity Relations in Radiative Transfer. ACM Trans. Graph. 33, 4, Article 104 (July 2014), 12 pages. Google ScholarDigital Library
57. Quan Zheng and Matthias Zwicker. 2018. Learning to Importance Sample in Primary Sample Space. arXiv:cs.LG/1808.07840Google Scholar
