“Simultaneous acquisition of polarimetric SVBRDF and normals”
Conference:
Type(s):
Title:
- Simultaneous acquisition of polarimetric SVBRDF and normals
Session/Category Title: Advanced SVBRDF
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Capturing appearance often requires dense sampling in light-view space, which is often achieved in specialized, expensive hardware setups. With the aim of realizing a compact acquisition setup without multiple angular samples of light and view, we sought to leverage an alternative optical property of light, polarization. To this end, we capture a set of polarimetric images with linear polarizers in front of a single projector and camera to obtain the appearance and normals of real-world objects. We encountered two technical challenges: First, no complete polarimetric BRDF model is available for modeling mixed polarization of both specular and diffuse reflection. Second, existing polarization-based inverse rendering methods are not applicable to a single local illumination setup since they are formulated with the assumption of spherical illumination. To this end, we first present a complete polarimetric BRDF (pBRDF) model that can define mixed polarization of both specular and diffuse reflection. Second, by leveraging our pBRDF model, we propose a novel inverse-rendering method with joint optimization of pBRDF and normals to capture spatially-varying material appearance: per-material specular properties (including the refractive index, specular roughness and specular coefficient), per-pixel diffuse albedo and normals. Our method can solve the severely ill-posed inverse-rendering problem by carefully accounting for the physical relationship between polarimetric appearance and geometric properties. We demonstrate how our method overcomes limited sampling in light-view space for inverse rendering by means of polarization.
References:
1. Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2015. Two-shot SVBRDF capture for stationary materials. ACM Trans. Graph. 34, 4 (2015), 110:1–13. Google ScholarDigital Library
2. Gary A Atkinson and Edwin R Hancock. 2006. Recovery of surface orientation from diffuse polarization. IEEE Transactions on image Processing 15, 6 (2006), 1653–1664. Google ScholarDigital Library
3. Seung-Hwan Baek, Diego Gutierrez, and Min H Kim. 2016. Birefractive stereo imaging for single-shot depth acquisition. ACM Transactions on Graphics 35, 6 (2016), 194. Google ScholarDigital Library
4. Guojun Chen, Yue Dong, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Reflectance scanning: Estimating shading frame and BRDF with generalized linear light sources. ACM Transactions on Graphics (TOG) 33, 4 (2014), 117. Google ScholarDigital Library
5. Edward Collett. 2005. Field guide to polarization. SPIE Bellingham, WA.Google Scholar
6. Robert L. Cook and Kenneth E. Torrance. 1982. A Reflectance Model for Computer Graphics. ACM Transactions on Graphics (TOG) 1, 1 (1982), 7–24. Google ScholarDigital Library
7. Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, and Jan Kautz. 2017. Polarimetric Multi-View Stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1558–1567.Google ScholarCross Ref
8. Paul E. Debevec and Jitendra Malik. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proc. ACM SIGGRAPH ’97. 369–378. Google ScholarDigital Library
9. Kenneth K Ellis. 1996. Polarimetric bidirectional reflectance distribution function of glossy coatings. JOSA A 13, 8 (1996), 1758–1762.Google ScholarCross Ref
10. Graham Fyffe, Paul Graham, Borom Tunwattanapong, Abhijeet Ghosh, and P Debevec. 2016. Near-Instant Capture of High-Resolution Facial Geometry and Reflectance. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 353–363.Google Scholar
11. Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A Wilson, and Paul Debevec. 2010. Circularly polarized spherical illumination reflectometry. In ACM Transactions on Graphics (TOG), Vol. 29. ACM, 162. Google ScholarDigital Library
12. Abhijeet Ghosh, Graham Fyffe, Borom Tunwattanapong, Jay Busch, Xueming Yu, and Paul Debevec. 2011. Multiview face capture using polarized spherical gradient illumination. ACM Transactions on Graphics (TOG) 30, 6 (2011), 129. Google ScholarDigital Library
13. Abhijeet Ghosh, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance. ACM Trans. Graph. 27, 5 (2008), 139. Google ScholarDigital Library
14. Darya Guarnera, Giuseppe Guarnera, Abhijeet Ghosh, Cornelia Denk, and Mashhuda Glencross. 2016. BRDF Representation and Acquisition. Computer Graphics Forum 35 (2016), 625–650.Google ScholarCross Ref
15. Giuseppe Claudio Guarnera, Pieter Peers, Paul Debevec, and Abhijeet Ghosh. 2012. Estimating surface normals from spherical stokes reflectance fields. In European Conference on Computer Vision. Springer, 340–349. Google ScholarDigital Library
16. Eric Heitz. 2014. Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs. J. Comput. Graph. Techn. 3, 2 (2014), 48–107.Google Scholar
17. Michael Holroyd, Jason Lawrence, and Todd Zickler. 2010. A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans. Graph. 29, 4 (2010), 99. Google ScholarDigital Library
18. Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. Vol. 26. ACM. Google ScholarDigital Library
19. MW Hyde IV, JD Schmidt, and MJ Havrilla. 2009. A geometrical optics polarimetric bidirectional reflectance distribution function for dielectric and metallic surfaces. Optics express 17, 24 (2009), 22138–22153.Google Scholar
20. Adrian Jarabo and Victor Arellano. 2017. Bidirectional Rendering of Vector Light Transport. Computer Graphics Forum (12 2017).Google Scholar
21. Achuta Kadambi, Vage Taamazyan, Boxin Shi, and Ramesh Raskar. 2015. Polarized 3D: High-Quality Depth Sensing with Polarization Cues. In Proc. ICCV. IEEE Computer Society, 3370–3378. Google ScholarDigital Library
22. Stefka Nikolova Kasarova, Nina Georgieva Sultanova, Christo Dimitrov Ivanov, and Ivan Dechev Nikolov. 2007. Analysis of the dispersion of optical plastic materials. Optical Materials 29, 11 (2007), 1481–1490.Google ScholarCross Ref
23. Jaewon Kim, Shahram Izadi, and Abhijeet Ghosh. 2016. Single-shot Layered Reflectance Separation Using a Polarized Light Field Camera. In Proc. Eurographics Symposium on Rendering. Google ScholarDigital Library
24. Jan Korger, Tobias Kolb, Peter Banzer, Andrea Aiello, Christoffer Wittmann, Christoph Marquardt, and Gerd Leuchs. 2013. The polarization properties of a tilted polarizer. Optics express 21, 22 (2013), 27032–27042.Google Scholar
25. Joo Ho Lee, Adrian Jarabo, Daniel S. Jeon, Diego Gutierrez, and Min H. Kim. 2018. Practical Multiple Scattering for Rough Surfaces. ACM Transactions on Graphics (TOG) 37, 6 (2018).Google ScholarDigital Library
26. Anat Levin, Rob Fergus, Frédo Durand, and William T Freeman. 2007. Image and depth from a conventional camera with a coded aperture. In ACM Transactions on Graphics, Vol. 26. ACM, 70. Google ScholarDigital Library
27. Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, and Paul Debevec. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Proc. Eurographics conference on Rendering Techniques. 183–194. Google ScholarDigital Library
28. JR Maxwell, J Beard, S Weiner, Da Ladd, and S Ladd. 1973. Bidirectional Reflectance Model Validation and Utilization. Technical Report. Environmental Research Inst Of Michiganann Arbor Infrared And Optics Div.Google Scholar
29. Daisuke Miyazaki, Robby T Tan, Kenji Hara, and Katsushi Ikeuchi. 2003. Polarization-based inverse rendering from a single view. In null. IEEE, 982. Google ScholarDigital Library
30. Michal Mojzik, Tomas Skrivan, Alexander Wilkie, and Jaroslav Krivanek. 2016. BiDirectional Polarised Light Transport. In Eurographics Symposium on Rendering. Google ScholarDigital Library
31. Daniel Moreno and Gabriel Taubin. 2012. Simple, accurate, and robust projector-camera calibration. In 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 2012 Second International Conference on. IEEE, 464–471. Google ScholarDigital Library
32. Koki Nagano, Graham Fyffe, Oleg Alexander, Jernej Barbic, Hao Li, Abhijeet Ghosh, and Paul E Debevec. 2015. Skin microstructure deformation with displacement map convolution. ACM Trans. Graph. 34, 4 (2015), 109–1. Google ScholarDigital Library
33. Giljoo Nam and Min H. Kim. 2014. Multispectral Photometric Stereo for Acquiring High-Fidelity Surface Normals. IEEE Computer Graphics and Applications 34, 6 (2014), 57–68.Google ScholarCross Ref
34. Giljoo Nam, Joo Ho Lee, Diego Gutierrez, and Min H. Kim. 2018. Practical SVBRDF Acquisition of 3D Objects with Unstructured Flash Photography. ACM Transactions on Graphics (TOG) 37, 6 (2018). Google ScholarDigital Library
35. Giljoo Nam, Joo Ho Lee, Hongzhi Wu, Diego Gutierrez, and Min H. Kim. 2016. Simultaneous Acquisition of Microscale Reflectance and Normals. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2016) 35, 6 (2016). Google ScholarDigital Library
36. Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. 2015. On optimal, minimal BRDF sampling for reflectance acquisition. ACM Transactions on Graphics 34, 6 (2015), 186. Google ScholarDigital Library
37. Richard G Priest and Thomas A Gerner. 2000. Polarimetric BRDF in the microfacet model: Theory and measurements. Technical Report. NAVAL RESEARCH LAB WASHINGTON DC.Google Scholar
38. Jérémy Riviere, Ilya Reshetouski, Luka Filipi, and Abhijeet Ghosh. 2017. Polarization Imaging Reflectometry in the Wild. ACM Trans. on Graphics 36, 6 (2017), 206:1–206:14. Google ScholarDigital Library
39. Szymon Rusinkiewicz. 1998. A New Change of Variables for Efficient BRDF Representation. In Proc. Eurographics Rendering Workshop. 11–22.Google ScholarCross Ref
40. Yinlong Sun. 2007. Statistical ray method for deriving reflection models of rough surfaces. JOSA A 24, 3 (2007), 724–744.Google ScholarCross Ref
41. Kenneth E Torrance and Ephraim M Sparrow. 1967. Theory for off-specular reflection from roughened surfaces. JOSA 57, 9 (1967), 1105–1112.Google ScholarCross Ref
42. Silvia Tozza, William AP Smith, Dizhong Zhu, Ravi Ramamoorthi, and Edwin R Hancock. 2017. Linear Differential Constraints for Photo-polarimetric Height Estimation. arXiv preprint arXiv:1708.07718 (2017).Google Scholar
43. Borom Tunwattanapong, Graham Fyffe, Paul Graham, Jay Busch, Xueming Yu, Abhijeet Ghosh, and Paul Debevec. 2013. Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination. ACM Trans. Graph. 32, 4 (2013), 109:1–12. Google ScholarDigital Library
44. Bruce Walter, Stephen R Marschner, Hongsong Li, and Kenneth E Torrance. 2007. Microfacet models for refraction through rough surfaces. In Eurographics conference on Rendering Techniques. Eurographics Association, 195–206. Google ScholarDigital Library
45. Richard A Waltz, José Luis Morales, Jorge Nocedal, and Dominique Orban. 2006. An interior algorithm for nonlinear optimization that combines line search and trust region steps. Mathematical programming 107, 3 (2006), 391–408.Google Scholar
46. Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition (Course Notes). In SIGGRAPH Asia 2015 Courses (SA ’15). ACM, New York, NY, USA, Article 1, 71 pages. Google ScholarDigital Library
47. Tim Weyrich, Jason Lawrence, Hendrik P.A. Lensch, Szymon Rusinkiewicz, and Todd Zickler. 2009. Principles of Appearance Acquisition and Representation. Foundations and Trends in Computer Graphics and Vision 4, 2 (Oct. 2009), 75–191. Google ScholarDigital Library
48. Alexander Wilkie and Andrea Weidlich. 2012. Polarised light in computer graphics. In SIGGRAPH Asia 2012 Courses. ACM, 8. Google ScholarDigital Library
49. Zexiang Xu, Jannik Boll Nielsen, Jiyang Yu, Henrik Wann Jensen, and Ravi Ramamoorthi. 2016. Minimal BRDF sampling for two-shot near-field reflectance acquisition. ACM Transactions on Graphics 35, 6 (2016), 188. Google ScholarDigital Library
50. Zhengyou Zhang. 2000. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence 22, 11 (2000), 1330–1334. Google ScholarDigital Library
51. Zhiming Zhou, Guojun Chen, Yue Dong, David Wipf, Yong Yu, John Snyder, and Xin Tong. 2016. Sparse-as-possible SVBRDF acquisition. ACM Transactions on Graphics (TOG) 35, 6 (2016), 189. Google ScholarDigital Library
