“Two-shot SVBRDF capture for stationary materials” by Aittala, Weyrich and Lehtinen

Material appearance acquisition usually makes a trade-off between acquisition effort and richness of reflectance representation. In this paper, we instead aim for both a light-weight acquisition procedure and a rich reflectance representation simultaneously, by restricting ourselves to one, but very important, class of appearance phenomena: texture-like materials. While such materials’ reflectance is generally spatially varying, they exhibit self-similarity in the sense that for any point on the texture there exist many others with similar reflectance properties. We show that the texturedness assumption allows reflectance capture using only two images of a planar sample, taken with and without a headlight flash. Our reconstruction pipeline starts with redistributing reflectance observations across the image, followed by a regularized texture statistics transfer and a non-linear optimization to fit a spatially-varying BRDF (SVBRDF) to the resulting data. The final result describes the material as spatially-varying, diffuse and specular, anisotropic reflectance over a detailed normal map. We validate the method by side-by-side and novel-view comparisons to photographs, comparing normal map resolution to sub-micron ground truth scans, as well as simulated results. Our method is robust enough to use handheld, JPEG-compressed photographs taken with a mobile phone camera and built-in flash.

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3 (Aug.), 294–302. Google ScholarDigital Library
2. Aittala, M., Weyrich, T., and Lehtinen, J. 2013. Practical SVBRDF capture in the frequency domain. ACM Transactions on Graphics (Proc. SIGGRAPH) 32, 4 (July), 110:1–110:12. Google ScholarDigital Library
3. Barron, J., and Malik, J. 2015. Shape, illumination, and reflectance from shading. IEEE Transactions on Pattern Analysis and Machine Intelligence (to appear), 1–19.Google ScholarCross Ref
4. Barrow, H. G. B., and Tenenbaum, J. M. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems (Apr.), 3–26.Google Scholar
5. Brady, A., Lawrence, J., Peers, P., and Weimer, W. 2014. genBRDF: Discovering new analytic BRDFs with genetic programming. ACM Transactions on Graphics (Proc. SIGGRAPH) 33, 4 (July), 114:1–114:11. Google ScholarDigital Library
6. Calonder, M., Lepetit, V., Strecha, C., and Fua, P. 2010. BRIEF: binary robust independent elementary features. In Proc. European Conference on Computer Vision, Springer-Verlag, Berlin, Heidelberg, 778–792. Google ScholarDigital Library
7. Clark, R., 2010. Crazybump. http://Www.Crazybump.Com, Last Access: 7 May 2015.Google Scholar
8. Dong, Y., Wang, J., Tong, X., Snyder, J., Lan, Y., Ben-Ezra, M., and Guo, B. 2010. Manifold bootstrapping for SVBRDF capture. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 4 (July), 98:1–98:10. Google ScholarDigital Library
9. Dong, Y., Tong, X., Pellacini, F., and Guo, B. 2011. Appgen: interactive material modeling from a single image. ACM Transactions on Graphics (Proc. SIGGRAPH ASIA) 30, 6 (Dec.), 146:1–146:10. Google ScholarDigital Library
10. DPReview, 2012. Quick Review: Apple Iphone 5. Www.Dpreview.Com/Articles/6867454450/Quick-review-Apple-Iphone-5-Camera, Last Access: 7 May 2015.Google Scholar
11. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In Proc. SIGGRAPH, ACM, New York, NY, USA, 341–346. Google ScholarDigital Library
12. Glencross, M., Ward, G., Jay, C., Liu, J., Melendez, F., and Hubbold, R. 2008. A perceptually validated model for surface depth hallucination. ACM Transactions on Graphics (Proc. SIGGRAPH) 27, 3 (Aug.), 59:1–59:8. Google ScholarDigital Library
13. Goldman, D., Curless, B., Hertzmann, A., and Seitz, S. 2005. Shape and spatially-varying BRDFs from photometric stereo. In Proc. IEEE International Conference on Computer Vision, vol. 1, 341–348. Google ScholarDigital Library
14. Haindl, M., and Filip, J. 2013. Visual Texture. Advances in Computer Vision and Pattern Recognition. Springer.Google Scholar
15. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In Proc. SIGGRAPH, ACM, New York, NY, USA, 229–238. Google ScholarDigital Library
16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proc. SIGGRAPH, 327–340. Google ScholarDigital Library
17. Johnson, K., Cole, F., Raj, A., and Adelson, E. 2011. Microgeometry capture using an elastomeric sensor. ACM Transactions on Graphics (Proc. SIGGRAPH) 30, 4 (July), 46:1–46:8. Google ScholarDigital Library
18. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Transactions on Graphics 22, 2 (Apr.), 234–257. Google ScholarDigital Library
19. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2006. Image-based reconstruction of spatially varying materials. In Proc. Eurographics Symposium on Rendering, Eurographics Association, Aire-la-Ville, Switzerland, 31–40. Google ScholarDigital Library
20. Marschner, S. 1998. Inverse Rendering for Computer Graphics. PhD thesis, Cornell University. Google ScholarDigital Library
21. Ngan, A., and Durand, F. 2006. Statistical acquisition of texture appearance. In Proc. Eurographics Symposium on Rendering, Eurographics Association, Aire-la-Ville, Switzerland, 31–40. Google ScholarDigital Library
22. Rabin, J., Peyré, G., Delon, J., and Bernot, M. 2011. Wasserstein barycenter and its application to texture mixing. In Proc. Intl. Conference on Scale Space and Variational Methods in Computer Vision (SSVM), Springer, vol. 6667 of Lecture Notes in Computer Science, 435–446. Google ScholarDigital Library
23. Schröder, K., Zinke, A., and Klein, R. 2015. Image-based reverse engineering and visual prototyping of woven cloth. IEEE Transactions on Visualization and Computer Graphics 21, 2 (Feb.), 188–200.Google ScholarCross Ref
24. Wang, J., Zhao, S., Tong, X., Snyder, J., and Guo, B. 2008. Modeling anisotropic surface reflectance with example-based microfacet synthesis. ACM Transactions on Graphics (Proc. SIGGRAPH) 27, 3 (Aug.), 41:1–41:9. Google ScholarDigital Library
25. Wang, C.-P., Snavely, N., and Marschner, S. 2011. Estimating dual-scale properties of glossy surfaces from step-edge lighting. ACM Transactions on Graphics (Proc. SIGGRAPH ASIA) 30, 6 (Dec.), 172:1–172:12. Google ScholarDigital Library
26. Weyrich, T., Lawrence, J., Lensch, H., Rusinkiewicz, S., and Zickler, T. 2009. Principles of appearance acquisition and representation. Foundations and Trends in Computer Graphics and Vision 4, 2, 75–191. Google ScholarDigital Library
27. Zickler, T., Ramamoorthi, R., Enrique, S., and Belhumeur, P. N. 2006. Reflectance sharing: predicting appearance from a sparse set of images of a known shape. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 8, 1287–1302. Google ScholarDigital Library