“A differential theory of radiative transfer” by Zhang, Wu, Zheng, Gkioulekas, Ramamoorthi, et al. …
Conference:
Type(s):
Title:
- A differential theory of radiative transfer
Session/Category Title: Differentiable Rendering
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Physics-based differentiable rendering is the task of estimating the derivatives of radiometric measures with respect to scene parameters. The ability to compute these derivatives is necessary for enabling gradient-based optimization in a diverse array of applications: from solving analysis-by-synthesis problems to training machine learning pipelines incorporating forward rendering processes. Unfortunately, physics-based differentiable rendering remains challenging, due to the complex and typically nonlinear relation between pixel intensities and scene parameters.We introduce a differential theory of radiative transfer, which shows how individual components of the radiative transfer equation (RTE) can be differentiated with respect to arbitrary differentiable changes of a scene. Our theory encompasses the same generality as the standard RTE, allowing differentiation while accurately handling a large range of light transport phenomena such as volumetric absorption and scattering, anisotropic phase functions, and heterogeneity. To numerically estimate the derivatives given by our theory, we introduce an unbiased Monte Carlo estimator supporting arbitrary surface and volumetric configurations. Our technique differentiates path contributions symbolically and uses additional boundary integrals to capture geometric discontinuities such as visibility changes.We validate our method by comparing our derivative estimations to those generated using the finite-difference method. Furthermore, we use a few synthetic examples inspired by real-world applications in inverse rendering, non-line-of-sight (NLOS) and biomedical imaging, and design, to demonstrate the practical usefulness of our technique.
References:
1. James Arvo. 1994. The Irradiance Jacobian for partially occluded polyhedral sources. In SIGGRAPH ’94. 343–350.Google ScholarDigital Library
2. Dejan Azinovic, Tzu-Mao Li, Anton Kaplanyan, and Matthias Niessner. 2019. Inverse Path Tracing for Joint Material and Lighting Estimation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
3. Benedikt Bitterli, Srinath Ravichandran, Thomas Müller, Magnus Wrenninge, Jan Novák, Steve Marschner, and Wojciech Jarosz. 2018. A Radiative Transfer Framework for Non-exponential Media. ACM Trans. Graph. 37, 6 (2018), 225:1–225:17.Google ScholarDigital Library
4. James F. Blinn. 1982. Light reflection functions for simulation of clouds and dusty surfaces. In SIGGRAPH ’82. 21–29.Google ScholarDigital Library
5. Kenneth M Case and Paul Frederick Zweifel. 1967. Linear transport theory. (1967).Google Scholar
6. Eva Cerezo, Frederic Pérez, Xavier Pueyo, Francisco J Seron, and François X Sillion. 2005. A survey on participating media rendering techniques. The Visual Computer 21, 5 (2005), 303–328.Google ScholarDigital Library
7. Subrahmanyan Chandrasekhar. 1960. Radiative Transfer. Courier Corporation.Google Scholar
8. Vivide Tuan-Chyan Chang, Peter S Cartwright, Sarah M Bean, Greg M Palmer, Rex C Bentley, and Nirmala Ramanujam. 2009. Quantitative physiology of the precancerous cervix in vivo through optical spectroscopy. Neoplasia 11, 4 (2009), 325–332.Google ScholarCross Ref
9. Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2018. Inverse transport networks. arXiv preprint arXiv:1809.10820 (2018).Google Scholar
10. Min Chen and James Arvo. 2000. Theory and application of specular path perturbation. ACM Trans. Graph. 19, 4 (2000), 246–278.Google ScholarDigital Library
11. Tom Collier, Dizem Arifler, Anais Malpica, Michele Follen, and Rebecca Richards-Kortum. 2003. Determination of epithelial tissue scattering coefficient using confocal microscopy. IEEE Journal of selected topics in quantum electronics 9, 2 (2003), 307–313.Google ScholarCross Ref
12. Harley Flanders. 1973. Differentiation under the integral sign. The American Mathematical Monthly 80, 6 (1973), 615–627.Google ScholarCross Ref
13. Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering. In Computer Vision – ECCV 2016. Springer International Publishing, 685–701.Google ScholarCross Ref
14. Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse volume rendering with material dictionaries. ACM Trans. Graph. 32, 6 (2013), 162:1–162:13.Google ScholarDigital Library
15. Milovš Hašan and Ravi Ramamoorthi. 2013. Interactive albedo editing in path-traced volumetric materials. ACM Trans. Graph. 32, 2 (2013), 11:1–11:11.Google ScholarDigital Library
16. Homan Igehy. 1999. Tracing ray differentials. In SIGGRAPH ’99. 179–186.Google ScholarDigital Library
17. Akira Ishimaru. 1978. Wave propagation and scattering in random media. Academic press New York.Google Scholar
18. Wenzel Jakob, Adam Arbree, Jonathan T. Moon, Kavita Bala, and Steve Marschner. 2010. A radiative transfer framework for rendering materials with anisotropic structure. ACM Trans. Graph. 29, 4 (2010), 53:1–53:13.Google ScholarDigital Library
19. Wenzel Jakob and Steve Marschner. 2012. Manifold exploration: A Markov chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Trans. Graph. 31, 4 (2012), 58:1–58:13.Google ScholarDigital Library
20. James T. Kajiya and Brian P Von Herzen. 1984. Ray tracing volume densities. SIGGRAPH Comput. Graph. 18, 3 (1984), 165–174.Google ScholarDigital Library
21. Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Neural 3D mesh renderer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3907–3916.Google ScholarCross Ref
22. Csaba Kelemen, László Szirmay-Kalos, György Antal, and Ferenc Csonka. 2002. A simple and robust mutation strategy for the metropolis light transport algorithm. In Computer Graphics Forum, Vol. 21. Wiley Online Library, 531–540.Google Scholar
23. Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2015. Matching real fabrics with micro-appearance models. ACM Trans. Graph. 35, 1 (2015), 1:1–1:26.Google ScholarDigital Library
24. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
25. Tejas D. Kulkarni, Pushmeet Kohli, Joshua B. Tenenbaum, and Vikash Mansinghka. 2015. Picture: A Probabilistic Programming Language for Scene Perception. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
26. Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and decomposition tracking for rendering heterogeneous volumes. ACM Trans. Graph. 36, 4 (2017), 111:1–111:16.Google ScholarDigital Library
27. Eric P Lafortune and Yves D Willems. 1996. Rendering participating media with bidirectional path tracing. In Rendering techniques’ 96. Springer, 91–100.Google ScholarDigital Library
28. L. Gary Leal. 2007. Advanced Transport Phenomena: fluid mechanics and convective transport processes. Vol. 7. Cambridge University Press.Google Scholar
29. Tzu-Mao Li. 2019. Differentiable visual computing. MIT PhD Dissertation.Google Scholar
30. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018a. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 222:1–222:11.Google ScholarDigital Library
31. Tzu-Mao Li, Jaakko Lehtinen, Ravi Ramamoorthi, Wenzel Jakob, and Frédo Durand. 2015. Anisotropic Gaussian mutations for Metropolis light transport through Hessian-Hamiltonian dynamics. ACM Trans. Graph. 34, 6 (2015), 209:1–209:13.Google ScholarDigital Library
32. Zhengqin Li, Zexiang Xu, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. 2018b. Learning to reconstruct shape and spatially-varying reflectance from a single image. ACM Trans. Graph. 37, 6 (2018), 269:1–269:11.Google ScholarDigital Library
33. Matthew M Loper and Michael J Black. 2014. OpenDR: An approximate differentiable renderer. In European Conference on Computer Vision. Springer, 154–169.Google ScholarCross Ref
34. Abhimitra Meka, Maxim Maximov, Michael Zollhoefer, Avishek Chatterjee, Hans-Peter Seidel, Christian Richardt, and Christian Theobalt. 2018. Lime: Live intrinsic material estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6315–6324.Google ScholarCross Ref
35. Michael I Mishchenko, Larry D Travis, and Andrew A Lacis. 2006. Multiple scattering of light by particles: radiative transfer and coherent backscattering. Cambridge University Press.Google Scholar
36. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo methods for volumetric light transport simulation. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 551–576.Google Scholar
37. Matthew O’Toole, David B Lindell, and Gordon Wetzstein. 2018. Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555, 7696 (2018), 338.Google Scholar
38. Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. In Rendering Techniques 2000. Springer, 11–22.Google ScholarCross Ref
39. GC Pomraning. 1973. The equations of radiation hydrodynamics. International Series of Monographs in Natural Philosophy, Oxford: Pergamon Press, 1973 (1973).Google Scholar
40. Ravi Ramamoorthi, Dhruv Mahajan, and Peter Belhumeur. 2007. A First-order Analysis of Lighting, Shading, and Shadows. ACM Trans. Graph. 26, 1 (2007), 2:1–2:21.Google ScholarDigital Library
41. Holly E. Rushmeier and Kenneth E. Torrance. 1987. The zonal method for calculating light intensities in the presence of a participating medium. In SIGGRAPH ’87. 293–302.Google Scholar
42. Charles Saunders, John Murray-Bruce, and Vivek K Goyal. 2019. Computational periscopy with an ordinary digital camera. Nature 565, 7740 (2019), 472.Google Scholar
43. Soumyadip Sengupta, Angjoo Kanazawa, Carlos D Castillo, and David W Jacobs. 2018. SfSNet: Learning shape, reflectance and illuminance of faces in the wild’. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6296–6305.Google ScholarCross Ref
44. Jerome Spanier and Ely M Gelbard. 1969. Monte Carlo principles and neutron transport problems. The Addison-Wesley Publishing Company.Google Scholar
45. Denis Sumin, Tobias Rittig, Vahid Babaei, Thomas Nindel, Alexander Wilkie, Piotr Didyk, Bernd Bickel, Jaroslav Křivánek, Karol Myszkowski, and Tim Weyrich. 2019. Geometry-aware Scattering Compensation for 3D Printing. ACM Trans. Graph. 38, 4 (2019), 111:1–111:14.Google ScholarDigital Library
46. Chia-Yin Tsai, Aswin C. Sankaranarayanan, and Ioannis Gkioulekas. 2019. Beyond Volumetric Albedo – A Surface Optimization Framework for Non-Line-Of-Sight Imaging. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
47. Hendrik Christoffel van de Hulst et al. 1957. Light scattering by small particles. (1957).Google Scholar
48. Zdravko Velinov, Marios Papas, Derek Bradley, Paulo Gotardo, Parsa Mirdehghan, Steve Marschner, Jan Novák, and Thabo Beeler. 2018. Appearance Capture and Modeling of Human Teeth. ACM Trans. Graph. 37, 6 (2018), 207:1–207:13.Google ScholarDigital Library
49. Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan, Moungi G Bawendi, and Ramesh Raskar. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature communications 3 (2012), 745.Google Scholar
50. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research 11, Dec (2010), 3371–3408.Google ScholarDigital Library
51. DC Walker, BH Brown, AD Blackett, J Tidy, and RH Smallwood. 2003. A study of the morphological parameters of cervical squamous epithelium. Physiological measurement 24, 1 (2003), 121.Google Scholar
52. Jiajun Wu, Joshua B Tenenbaum, and Pushmeet Kohli. 2017. Neural scene de-rendering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 699–707.Google ScholarCross Ref
53. Shumian Xin, Sotiris Nousias, Kiriakos N Kutulakos, Aswin C Sankaranarayanan, Srinivasa G Narasimhan, and Ioannis Gkioulekas. 2019. A theory of fermat paths for non-line-of-sight shape reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
54. Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling scattering parameters for rendering anisotropic media. ACM Trans. Graph. 35, 6 (2016), 166:1–166:11.Google ScholarDigital Library
