“Polarization imaging reflectometry in the wild”
Conference:
Type(s):
Title:
- Polarization imaging reflectometry in the wild
Session/Category Title: Reflectance and Scattering
Presenter(s)/Author(s):
Abstract:
We present a novel approach for on-site acquisition of surface reflectance for planar, spatially varying, isotropic samples in uncontrolled outdoor environments. Our method exploits the naturally occurring linear polarization of incident and reflected illumination for this purpose. By rotating a linear polarizing filter in front of a camera at three different orientations, we measure the polarization reflected off the sample and combine this information with multi-view analysis and inverse rendering in order to recover per-pixel, high resolution reflectance and surface normal maps. Specifically, we employ polarization imaging from two near orthogonal views close to the Brewster angle of incidence in order to maximize polarization cues for surface reflectance estimation. To the best of our knowledge, our method is the first to successfully extract a complete set of reflectance parameters with passive capture in completely uncontrolled outdoor settings. To this end, we analyze our approach under the general, but previously unstudied, case of incident partial linear polarization (due to the sky) in order to identify the strengths and weaknesses of the method under various outdoor conditions. We provide practical guidelines for on-site acquisition based on our analysis, and demonstrate high quality results with an entry level DSLR as well as a mobile phone.
References:
1. Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance Modeling by Neural Texture Synthesis. ACM Trans. Graph. 35, 4, Article 65 (July 2016), 13 pages.
2. Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2013. Practical SVBRDF Capture In The Frequency Domain. ACM Trans. on Graphics (Proc. SIGGRAPH) 32, 4 (2013).
3. Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2015. Two-shot svbrdf capture for stationary materials. ACM Transactions on Graphics 34, 4 (2015), 110.
4. Gary Atkinson and Edwin R Hancock. 2006. Recovery of surface orientation from diffuse polarization. Image Processing, IEEE Transactions on 15, 6 (2006), 1653–1664.
5. Guojun Chen, Yue Dong, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Reflectance Scanning: Estimating Shading Frame and BRDF with Generalized Linear Light Sources. ACM Trans. Graph. 33, 4, Article 117 (July 2014), 11 pages.
6. Hua Chen and Lawrence B. Wolff. 1998. Polarization Phase-Based Method For Material Classification In Computer Vision. IJCV 28, 1 (1998), 73–83.
7. E. Collett. 2005. Field Guide to Polarization, SPIE Field Guides vol. FG05. SPIE.
8. R. L. Cook and K. E. Torrance. 1982. A Reflectance Model for Computer Graphics. ACM TOG 1, 1 (1982), 7–24.
9. Paul Debevec. 1998. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography. In Proceedings of ACM SIGGRAPH 98.
10. Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. 2000. Acquiring the Reflectance Field of a Human Face. In Proceedings of ACM SIGGRAPH 2000. 145–156.
11. Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Appearance-from-motion: recovering spatially varying surface reflectance under unknown lighting. ACM Transactions on Graphics (TOG) 33, 6 (2014), 193.
12. Yannick Francken, Tom Cuypers, Tom Mertens, and Philippe Bekaert. 2009. Gloss and Normal Map Acquisition of Mesostructures Using Gray Codes. In Proceedings of the 5th International Symposium on Advances in Visual Computing: Part II (ISVC ’09). Springer-Verlag, Berlin, Heidelberg, 788–798.
13. Andrew Gardner, Chris Tchou, Tim Hawkins, and Paul Debevec. 2003. Linear light source reflectometry. ACM Trans. Graph. (Proc. SIGGRAPH) 22, 3 (2003), 749–758.
14. Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, and Paul Debevec. 2010. Circularly polarized spherical illumination reflectometry. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 29 (December 2010), 162:1–162:12. Issue 6.
15. Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, and Paul E. Debevec. 2009. Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination. Comput. Graph. Forum 28, 4 (2009), 1161–1170.
16. Abhijeet Ghosh, Graham Fyffe, Borom Tunwattanapong, Jay Busch, Xueming Yu, and Paul Debevec. 2011. Multiview face capture using polarized spherical gradient illumination. ACM Trans. Graphics (Proc. SIGGRAPH Asia), Article 129 (2011), 10 pages.
17. Abhijeet Ghosh, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. Practical Modeling and Acquisition of Layered Facial Reflectance. ACM Transactions on Graphics 27, 5 (Dec. 2008), 139:1–139:10.
18. Mashhuda Glencross, Gregory J. Ward, Francho Melendez, Caroline Jay, Jun Liu, and Roger Hubbold. 2008. A Perceptually Validated Model for Surface Depth Hallucination. ACM Trans. Graph. 27, 3, Article 59 (Aug. 2008), 8 pages.
19. D. Guarnera, G.C. Guarnera, A. Ghosh, C. Denk, and M. Glencross. 2016. BRDF Representation and Acquisition. Computer Graphics Forum 35, 2 (2016), 625–650. Cross Ref
20. Giuseppe Claudio Guarnera, Pieter Peers, Paul E. Debevec, and Abhijeet Ghosh. 2012. Estimating Surface Normals from Spherical Stokes Reflectance Fields. In ECCV Workshop on Color and Photometry in Computer Vision. 340–349.
21. Daniel Hauagge, Scott Wehrwein, Paul Upchurch, Kavita Bala, and Noah Snavely. 2014. Reasoning about photo collections using models of outdoor illumination. In Proceedings of the British Machine Vision Conference. BMVA Press. Cross Ref
22. Cong Phuoc Huynh, Antonio Robles-Kelly, and Edwin Hancock. 2010. Shape and refractive index recovery from single-view polarisation images. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 1229–1236. Cross Ref
23. Achuta Kadambi, Vage Taamazyan, Boxin Shi, and Ramesh Raskar. 2015. Polarized 3D: High-Quality Depth Sensing with Polarization Cues. In Proceedings of the IEEE International Conference on Computer Vision. 3370–3378.
24. Davis E. King. 2009. Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research 10 (2009), 1755–1758.
25. G.P. Können. 1985. Polarized Light in Nature. Cambridge University Press.
26. Kazutada Koshikawa. 1992. A polarimetric approach to shape understanding of glossy objects. (1992), 190–192.
27. Stephen Lombardi and Ko Nishino. 2012. Reflectance and Natural Illumination from a Single Image. In Proc. European Conference on Computer Vision (ECCV’12). 582–595.
28. Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, and Paul Debevec. 2007. Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination. In Rendering Techniques. 183–194.
29. Rafal Mantiuk, Grzegorz Krawczyk, Radoslaw Mantiuk, and Hans-Peter Seidel. 2007. High dynamic range imaging pipeline: Perception-motivated representation of visual content. In Electronic Imaging 2007. International Society for Optics and Photonics, 649212-649212.
30. Daisuke Miyazaki, Masataka Kagesawa, and Katsushi Ikeuchi. 2003. Polarization-based Transparent Surface Modeling from Two Views. In ICCV. 1381–1386.
31. Daisuke Miyazaki, Takuya Shigetomi, Masashi Baba, Ryo Furukawa, Shinsaku Hiura, and Naoki Asada. 2012. Polarization-based surface normal estimation of black specular objects from multiple viewpoints. In 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 2012 Second International Conference on. IEEE, 104–111.
32. Daisuke Miyazaki, Robby T. Tan, Kenji Hara, and Katsushi Ikeuchi. 2003. Polarization-based Inverse Rendering from a Single View. In ICCV. 982–987.
33. Volker Müller. 1995. Polarization-Based Separation of Diffuse and Specular Surface-Reflection. In Mustererkennung 1995. Springer, 202–209.
34. Shree K. Nayar, Xi-Sheng Fang, and Terrance Boult. 1997. Separation of Reflection Components Using Color and Polarization. IJCV 21, 3 (1997).
35. Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. 2015. On Optimal, Minimal BRDF Sampling for Relectance Acquisition. ACM Trans. Graph. 34, 6, Article 186 (Oct. 2015), 11 pages.
36. Geoffrey Oxholm and Ko Nishino. 2012. Shape and Reflectance from Natural Illumination. In Proc. European Conference on Computer Vision (ECCV’12). 528–541.
37. KE Peiponen, V Kontturi, I Niskanen, M Juuti, J Räty, H Koivula, and M Toivakka. 2008. On estimation of complex refractive index and colour of dry black and cyan offset inks by a multi-function spectrophotometer. Measurement Science and Technology 19, 11 (2008), 115601. Cross Ref
38. Stefan Rahmann and Nikos Canterakis. 2001. Reconstruction of Specular Surfaces using Polarization Imaging. CVRP 1 (2001), 149.
39. Peiran Ren, Jiaping Wang, John Snyder, Xin Tong, and Baining Guo. 2011. Pocket reflectometry. ACM Trans. Graph. 30, 4 (July 2011), 45:1–45:10.
40. J. Riviere, P. Peers, and A. Ghosh. 2016. Mobile Surface Relectometry. Computer Graphics Forum 35, 1 (2016), 191–202.
41. Fabiano Romeiro, Yuriy Vasilyev, and Todd Zickler. 2008. Passive relectometry. In Computer Vision-ECCV 2008. Springer, 859–872.
42. Fabiano Romeiro and Todd Zickler. 2010. Blind reflectometry. In Proc. of ECCV. 45–58.
43. Firooz Sadjadiz and Farzad Sadjadi. 2007. Extraction of surface normal and index of refraction using a pair of passive infrared polarimetric sensors. In IEEE Conference on Computer Vision and Pattern Recognition. 1–5. Cross Ref
44. Megumi Saito, Yoichi Sato, Katsushi Ikeuchi, and Hiroshi Kashiwagi. 1999. Measurement of surface orientations of transparent objects by use of polarization in highlight. J. Opt. Soc. Am. A 16, 9 (1999), 2286–2293. Cross Ref
45. C. Schlick. 1994. An inexpensive BRDF model for physically-based rendering. Computer Graphics Forum 13, 3 (1994), 233–246. Cross Ref
46. W.A.P. Smith, R. Ramamoorthi, and S. Tozza. 2016. Linear Depth Estimation from an Uncalibrated, Monocular Polarisation Image. In Proc. of European Conference on Computer Vision (ECCV). 517–526.
47. John W Strutt. 1871. XV. On the light from the sky, its polarization and colour. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 41, 271 (1871), 107–120. Cross Ref
48. Vimal Thilak, David G. Voelz, and Charles D. Creusere. 2007. Polarization-based index of refraction and reflection angle estimation for remote sensing applications. Appl. Opt. 46, 30 (2007), 7527–7536. Cross Ref
49. Shinji Umeyama and Guy Godin. 2004. Separation of Diffuse and Specular Components of Surface Reflection by Use of Polarization and Statistical Analysis of Images. PAMI 26, 5 (2004).
50. Bruce Walter, Stephen R Marschner, Hongsong Li, and Kenneth E Torrance. 2007. Microfacet models for refraction through rough surfaces. In Proceedings of the 18th Eurographics conference on Rendering Techniques. Eurographics Association, 195–206.
51. Chun-Po Wang, Noah Snavely, and Steve Marschner. 2011. Estimating Dual-scale Properties of Glossy Surfaces from Step-edge Lighting. ACM Trans. Graph. 30, 6, Article 172 (Dec. 2011), 12 pages.
52. Tim Weyrich, Jason Lawrence, Hendrik P. A. Lensch, Szymon Rusinkiewicz, and Todd Zickler. 2009. Principles of Appearance Acquisition and Representation. Found. Trends. Comput. Graph. Vis. 4, 2 (Feb. 2009), 75–191.
53. L. B. Wolff. 1989. Surface orientation from two camera stereo with polarizers. In Proc. SPIE Conf. Optics, Illumination and Image Sensing for Machine Vision IV, Vol. 1194. 287–297.
54. Lawrence B. Wolff and Terrance E. Boult. 1991. Constraining Object Features Using a Polarization Relectance Model. PAMI 13, 7 (1991), 635–657.
55. Changchang Wu. 2011. VisualSFM: A visual structure from motion system. (2011).
56. Hongzhi Wu and Kun Zhou. 2015. AppFusion: Interactive Appearance Acquisition Using a Kinect Sensor. Computer Graphics Forum 34, 6 (2015), 289–298.
57. Rui Xia, Yue Dong, Pieter Peers, and Xin Tong. 2016. Recovering Shape and Spatially-varying Surface Reflectance Under Unknown Illumination. ACM Trans. Graph. 35, 6, Article 187 (Nov. 2016), 12 pages.
