“A non-exponential transmittance model for volumetric scene representations” by Vicini, Jakob and Kaplayan

We introduce a novel transmittance model to improve the volumetric representation of 3D scenes. The model can represent opaque surfaces in the volumetric light transport framework. Volumetric representations are useful for complex scenes, and become increasingly popular for level of detail and scene reconstruction. The traditional exponential transmittance model found in volumetric light transport cannot capture correlations in visibility across volume elements. When representing opaque surfaces as volumetric density, this leads to both bloating of silhouettes and light leaking artifacts. By introducing a parametric non-exponential transmittance model, we are able to approximate these correlation effects and significantly improve the accuracy of volumetric appearance representation of opaque scenes. Our parametric transmittance model can represent a continuum between the linear transmittance that opaque surfaces exhibit and the traditional exponential transmittance encountered in participating media and unstructured geometries. This covers a large part of the spectrum of geometric structures encountered in complex scenes. In order to handle the spatially varying transmittance correlation effects, we further extend the theory of non-exponential participating media to a heterogeneous transmittance model. Our model is compact in storage and computationally efficient both for evaluation and for reverse-mode gradient computation. Applying our model to optimization algorithms yields significant improvements in volumetric scene appearance quality. We further show improvements for relevant applications, such as scene appearance prefiltering, image-based scene reconstruction using differentiable rendering, neural representations, and compare it to a conventional exponential model.

1. Johanna Beyer, Markus Hadwiger, and Hanspeter Pfister. 2015. State-of-the-Art in GPU-Based Large-Scale Volume Visualization. Computer Graphics Forum 34, 8 (Dec. 2015), 13–37.Google ScholarDigital Library
2. Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020a. Neural Reflectance Fields for Appearance Acquisition. arXiv:2008.03824Google Scholar
3. Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020b. Deep Reflectance Volumes: Relightable Reconstructions from Multi-view Photometric Images. In ECCV. 25 pages.Google Scholar
4. Benedikt Bitterli, Srinath Ravichandran, Thomas Müller, Magnus Wrenninge, Jan Novák, Steve Marschner, and Wojciech Jarosz. 2018. A radiative transfer framework for non-exponential media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 225:1–225:17.Google Scholar
5. Emanuele Caglioti and Francois Golse. 2003. On the Distribution of Free Path Lengths for the Periodic Lorentz Gas III. Communications in Mathematical Physics 236, 2 (May 2003), 199–221.Google ScholarCross Ref
6. Thomas Camminady, Martin Frank, and Edward W. Larsen. 2017. Nonclassical particle transport in heterogeneous materials. In M&C 2017 : International Conference on Mathematics & Computational Methods Applied to Nuclear Science & Engineering.Google Scholar
7. Subrahmanyan Chandrasekhar. 1960. Radiative transfer. Dover publications, New York.Google Scholar
8. Cyril Crassin, Fabrice Neyret, Sylvain Lefebvre, and Elmar Eisemann. 2009. GigaVoxels: Ray-Guided Streaming for Efficient and Detailed Voxel Rendering. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (Boston, Massachusetts). Association for Computing Machinery, New York, NY, USA, 15–22.Google ScholarDigital Library
9. Cyril Crassin, Fabrice Neyret, Miguel Sainz, Simon Green, and Elmar Eisemann. 2011. Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum 30, 7 (2011), 1921–1930.Google ScholarCross Ref
10. Anthony Davis and Mark Mineev-Weinstein. 2011. Radiation propagation in random media: From positive to negative correlations in high-frequency fluctuations. Journal of Quantitative Spectroscopy and Radiative Transfer 112 (03 2011), 632–645.Google Scholar
11. Eugene d’Eon. 2019. A Reciprocal Formulation of Nonexponential Radiative Transfer. 3: Binary Mixtures. arXiv:1903.08783Google Scholar
12. Jonathan Dupuy, Eric Heitz, and Eugene d’Eon. 2016. Additional Progress towards the Unification of Microfacet and Microflake Theories. In Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas and Implementations (Dublin, Ireland) (Computer Graphics Forum (Proceedings of EGSR)). Eurographics Association, Goslar, DEU, 55–63.Google Scholar
13. Eugene d’Eon. 2018. A Reciprocal Formulation of Nonexponential Radiative Transfer. 1: Sketch and Motivation. Journal of Computational and Theoretical Transport 47, 1-3 (2018), 84–115.Google Scholar
14. Eugene d’Eon. 2019. A Reciprocal Formulation of Nonexponential Radiative Transfer. 2: Monte Carlo Estimation and Diffusion Approximation. Journal of Computational and Theoretical Transport 48, 6 (2019), 201–262.Google ScholarCross Ref
15. Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Ja roslav Křivánek, and Wojciech Jarosz. 2019. Integral formulations of volumetric transmittance. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (2019), 17 pages.Google Scholar
16. Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering. In ECCV, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, 685–701.Google Scholar
17. Aidan N. Gomez, Mengye Ren, Raquel Urtasun, and Roger B. Grosse. 2017. The Reversible Residual Network: Backpropagation without Storing Activations. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 2211–2221.Google Scholar
18. Jie Guo, Yanjun Chen, Bingyang Hu, Ling-Qi Yan, Yanwen Guo, and Yuntao Liu. 2019. Fractional Gaussian Fields for Modeling and Rendering of Spatially-Correlated Media. ACM Trans. Graph. (Proc. SIGGRAPH) 38, 4, Article 45 (July 2019), 13 pages.Google ScholarDigital Library
19. Markus Hadwiger, Ali K. Al-Awami, Johanna Beyer, Marco Agus, and Hanspeter Pfister. 2018. SparseLeap: Efficient Empty Space Skipping for Large-Scale Volume Rendering. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 974–983.Google ScholarCross Ref
20. Eric Heitz, Jonathan Dupuy, Cyril Crassin, and Carsten Dachsbacher. 2015. The SGGX Microflake Distribution. ACM Trans. Graph. (Proc. SIGGRAPH) 34, 4, Article 48 (July 2015), 11 pages.Google ScholarDigital Library
21. Eric Heitz and Fabrice Neyret. 2012. Representing Appearance and Pre-Filtering Sub-pixel Data in Sparse Voxel Octrees. In Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics (Paris, France) (EGGH-HPG’12). Eurographics Association, Goslar, DEU, 125–134.Google ScholarDigital Library
22. Homan Igehy. 1999. Tracing Ray Differentials. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 99). ACM Press/Addison-Wesley Publishing Co., USA, 179–186.Google ScholarDigital Library
23. Wenzel Jakob. 2019. Enoki: structured vectorization and differentiation on modern processor architectures. https://github.com/mitsuba-renderer/enoki.Google Scholar
24. Wenzel Jakob, Jonathan T. Moon, Adam Arbree, Kavita Bala, and Steve Marschner. 2010. A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 10 (July 2010), 53:1–53:13.Google ScholarDigital Library
25. Adrian Jarabo, Carlos Aliaga, and Diego Gutierrez. 2018. A Radiative Transfer Framework for Spatially-Correlated Materials. ACM Trans. Graph. (Proc. SIGGRAPH) 37, 4, Article 83 (July 2018), 13 pages.Google ScholarDigital Library
26. Anton Kaplanyan, Anton Sochenov, Thomas Leimkuehler, Mikhail Okunev, Todd Goodall, and Rufo Gizem. 2019. DeepFovea: Neural Reconstruction for Foveated Rendering and Video Compression using Learned Statistics of Natural Videos. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 4 (2019), 212:1–212:13.Google Scholar
27. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, Conference Track Proceedings.Google Scholar
28. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1–222:11.Google Scholar
29. Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. 2019. Neural Volumes: Learning Dynamic Renderable Volumes from Images. ACM Trans. Graph. (Proc. SIGGRAPH) 38, 4, Article 65 (July 2019), 14 pages.Google ScholarDigital Library
30. Guillaume Loubet and Fabrice Neyret. 2017. Hybrid Mesh-Volume LoDs for All-Scale Pre-Filtering of Complex 3D Assets. Computer Graphics Forum 36, 2 (2017), 431–442.Google ScholarDigital Library
31. David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. 2002. Level of Detail for 3D Graphics. Morgan Kaufmann Publishers Inc.Google ScholarDigital Library
32. Johannes Meng, Marios Papas, Ralf Habel, Carsten Dachsbacher, Steve Marschner, Markus Gross, and Wojciech Jarosz. 2015. Multi-scale Modeling and Rendering of Granular Materials. ACM Trans. Graph. (Proc. SIGGRAPH) 34, 4, Article 49 (July 2015), 13 pages.Google ScholarDigital Library
33. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV. 25 pages.Google Scholar
34. Patrick Min. 2004 – 2021. binvox. http://www.patrickmin.com/binvox. Accessed: 2021-04-20.Google Scholar
35. Jonathan T. Moon, Bruce Walter, and Stephen R. Marschner. 2007. Rendering Discrete Random Media Using Precomputed Scattering Solutions. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (Grenoble, France) (Computer Graphics Forum (Proceedings of EGSR)). Eurographics Association, Goslar, DEU, 231–242.Google Scholar
36. Nate Morrical, Will Usher, Ingo Wald, and Valerio Pascucci. 2019. Efficient Space Skipping and Adaptive Sampling of Unstructured Volumes Using Hardware Accelerated Ray Tracing. In 2019 IEEE Visualization Conference (VIS). 256–260.Google Scholar
37. Thomas Müller, Marios Papas, Markus Gross, Wojciech Jarosz, and Jan Novák. 2016. Efficient Rendering of Heterogeneous Polydisperse Granular Media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35, 6 (Dec. 2016), 168:1–168:14.Google Scholar
38. Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 39, 4, Article 146 (July 2020), 15 pages.Google ScholarDigital Library
39. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A Retargetable Forward and Inverse Renderer. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (Nov. 2019), 17 pages.Google ScholarDigital Library
40. Fakir S. Nooruddin and Greg Turk. 2003. Simplification and Repair of Polygonal Models Using Volumetric Techniques. IEEE Transactions on Visualization and Computer Graphics 9, 2 (2003), 191–205.Google ScholarDigital Library
41. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo Methods for Volumetric Light Transport Simulation. Computer Graphics Forum (Proceedings of Eurographics – State of the Art Reports) 37, 2 (May 2018), 26 pages.Google ScholarCross Ref
42. Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, and Martin Stich. 2010. OptiX: A General Purpose Ray Tracing Engine. ACM Trans. Graph. (Proc. SIGGRAPH), Article 66 (2010), 13 pages.Google Scholar
43. Alexander Schwank, Callum James James, and Tony Micilotta. 2016. The Trees of The Jungle Book. In ACM SIGGRAPH Talks (Anaheim, California). Association for Computing Machinery, New York, NY, USA, Article 21, 2 pages.Google Scholar
44. Jos Stam. 2020. Computing Light Transport Gradients using the Adjoint Method. CoRR (2020), 23 pages. arXiv:2006.15059Google Scholar
45. Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014. Embree: a kernel framework for efficient CPU ray tracing. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–8.Google ScholarDigital Library
46. Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala. 2012. Structure-aware synthesis for predictive woven fabric appearance. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (2012), 75.Google ScholarDigital Library
47. Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling Scattering Parameters for Rendering Anisotropic Media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35, 6, Article 166 (Nov. 2016), 11 pages.Google Scholar
48. Quan Zheng, Vahid Babaei, Gordon Wetzstein, Hans-Peter Seidel, Matthias Zwicker, and Gurprit Singh. 2020. Neural Light Field 3D Printing. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 39, 6, Article 207 (2020), 12 pages.Google Scholar