“Efficient reflectance capture using an autoencoder” by Kang, Chen, Wang, Zhou and Wu
Conference:
Type:
Entry Number: 127
Title:
- Efficient reflectance capture using an autoencoder
Session/Category Title: Learning for Rendering and Material Acquisition
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We propose a novel framework that automatically learns the lighting patterns for efficient reflectance acquisition, as well as how to faithfully reconstruct spatially varying anisotropic BRDFs and local frames from measurements under such patterns. The core of our framework is an asymmetric deep autoencoder, consisting of a nonnegative, linear encoder which directly corresponds to the lighting patterns used in physical acquisition, and a stacked, nonlinear decoder which computationally recovers the BRDF information from captured photographs. The autoencoder is trained with a large amount of synthetic reflectance data, and can adapt to various factors, including the geometry of the setup and the properties of appearance. We demonstrate the effectiveness of our framework on a wide range of physical materials, using as few as 16 ~ 32 lighting patterns, which correspond to 12 ~ 25 seconds of acquisition time. We also validate our results with the ground truth data and captured photographs. Our framework is useful for increasing the efficiency in both novel and existing acquisition setups.
References:
1. Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance Modeling by Neural Texture Synthesis. ACM Trans. Graph. 35, 4, Article 65 (July 2016), 13 pages. Google ScholarDigital Library
2. Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2013. Practical SVBRDF Capture in the Frequency Domain. ACM Trans. Graph. 32, 4, Article 110 (July 2013), 12 pages. Google ScholarDigital Library
3. Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2015. Two-shot SVBRDF Capture for Stationary Materials. ACM Trans. Graph. 34, 4, Article 110 (July 2015), 13 pages. Google ScholarDigital Library
4. Guojun Chen, Yue Dong, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Reflectance Scanning: Estimating Shading Frame and BRDF with Generalized Linear Light Sources. ACM Trans. Graph. 33, 4, Article 117 (July 2014), 11 pages. Google ScholarDigital Library
5. Kristin J. Dana, Bram van Ginneken, Shree K. Nayar, and Jan J. Koenderink. 1999. Reflectance and Texture of Real-world Surfaces. ACM Trans. Graph. 18, 1 (Jan. 1999), 1–34. Google ScholarDigital Library
6. Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Yanxiang Lan, Moshe Ben-Ezra, and Baining Guo. 2010. Manifold Bootstrapping for SVBRDF Capture. ACM Trans. Graph. 29, 4, Article 98 (July 2010), 10 pages. Google ScholarDigital Library
7. Andrew Gardner, Chris Tchou, Tim Hawkins, and Paul Debevec. 2003. Linear light source reflectomerry. ACM Trans. Graph. 22, 3 (2003), 749–758. Google ScholarDigital Library
8. Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, and Paul Debevec. 2009. Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination. Computer Graphics Forum 28, 4 (2009), 1161–1170. Google ScholarDigital Library
9. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org. Google ScholarDigital Library
10. G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the Dimensionality of Data with Neural Networks. Science 313, 5786 (2006), 504–507.Google Scholar
11. Eric P. F. Lafortune, Sing-Choong Foo, Kenneth E. Torrance, and Donald P. Greenberg. 1997. Non-linear Approximation of Reflectance Functions. In Proc. SIGGRAPH. 117–126. Google ScholarDigital Library
12. Jason Lawrence, Aner Ben-Artzi, Christopher DeCoro, Wojciech Matusik, Hanspeter Pfister, Ravi Ramamoorthi, and Szymon Rusinkiewicz. 2006. Inverse Shade Trees for Non-parametric Material Representation and Editing. ACM Trans. Graph. 25, 3 (July 2006), 735–745. Google ScholarDigital Library
13. Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich, and Hans-Peter Seidel. 2003. Image-based Reconstruction of Spatial Appearance and Geometric Detail. ACM Trans. Graph. 22, 2 (April 2003), 234–257. Google ScholarDigital Library
14. Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling Surface Appearance from a Single Photograph Using Self-augmented Convolutional Neural Networks. ACM Trans. Graph. 36, 4, Article 45 (July 2017), 11 pages. Google ScholarDigital Library
15. Stephen R. Marschner, Stephen H. Westin, Eric P. F. Lafortune, Kenneth E. Torrance, and Donald P. Greenberg. 1999. Image-based BRDF Measurement Including Human Skin. In Proc. EGWR. 131–144. Google ScholarDigital Library
16. Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003a. A Data-driven Reflectance Model. ACM Trans. Graph. 22, 3 (July 2003), 759–769. Google ScholarDigital Library
17. Wojciech Matusik, Hanspeter Pfister, Matthew Brand, and Leonard McMillan. 2003b. Efficient Isotropic BRDF Measurement. In Proc. EGWR. 241–247. Google ScholarDigital Library
18. Stephen McAuley, Stephen Hill, Naty Hoffman, Yoshiharu Gotanda, Brian Smits, Brent Burley, and Adam Martinez. 2012. Practical Physically-based Shading in Film and Game Production. In ACM SIGGRAPH 2012 Courses. Article 10, 7 pages. Google ScholarDigital Library
19. Giljoo Nam, Joo Ho Lee, Hongzhi Wu, Diego Gutierrez, and Min H. Kim. 2016. Simultaneous Acquisition of Microscale Reflectance and Normals. ACM Trans. Graph. 35, 6, Article 185 (Nov. 2016), 11 pages. Google ScholarDigital Library
20. Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. 2015. On Optimal, Minimal BRDF Sampling for Reflectance Acquisition. ACM Trans. Graph. 34, 6, Article 186 (Oct. 2015), 11 pages. Google ScholarDigital Library
21. Pieter Peers, Dhruv K. Mahajan, Bruce Lamond, Abhijeet Ghosh, Wojciech Matusik, Ravi Ramamoorthi, and Paul Debevec. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 1, Article 3 (Feb. 2009), 18 pages. Google ScholarDigital Library
22. Peiran Ren, Jiaping Wang, John Snyder, Xin Tong, and Baining Guo. 2011. Pocket reflectometry. ACM Trans. Graph. 30, 4 (2011), 1–10. Google ScholarDigital Library
23. Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-RMSProp: Divide the Gradient by a Running Average of Its Recent Magnitude. Neural Networks for Machine Learning 4 (2012), 26–31.Google Scholar
24. Borom Tunwattanapong, Graham Fyffe, Paul Graham, Jay Busch, Xueming Yu, Abhijeet Ghosh, and Paul Debevec. 2013. Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination. ACM Trans. Graph. 32, 4, Article 109 (July 2013), 12 pages. Google ScholarDigital Library
25. Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction through Rough Surfaces. In Rendering Techniques (Proc. EGWR). Google ScholarDigital Library
26. Jiaping Wang, Shuang Zhao, Xin Tong, John Snyder, and Baining Guo. 2008. Modeling Anisotropic Surface Reflectance with Example-based Microfacet Synthesis. ACM Trans. Graph. 27, 3, Article 41 (Aug. 2008), 9 pages. Google ScholarDigital Library
27. Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition. In SIGGRAPH Asia Courses. Article 1, 71 pages. Google ScholarDigital Library
28. Tim Weyrich, Jason Lawrence, Hendrik P. A. Lensch, Szymon Rusinkiewicz, and Todd Zickler. 2009. Principles of Appearance Acquisition and Representation. Found. Trends. Comput. Graph. Vis. 4, 2 (2009), 75–191. Google ScholarDigital Library
29. Zexiang Xu, Jannik Boll Nielsen, Jiyang Yu, Henrik Wann Jensen, and Ravi Ramamoorthi. 2016. Minimal BRDF Sampling for Two-shot Near-field Reflectance Acquisition. ACM Trans. Graph. 35, 6, Article 188 (Nov. 2016), 12 pages. Google ScholarDigital Library
30. Todd Zickler, Sebastian Enrique, Ravi Ramamoorthi, and Peter Belhumeur. 2005. Reflectance Sharing: Image-based Rendering from a Sparse Set of Images. In Proc. EGSR. 253–264. Google ScholarDigital Library
