“Inverse shade trees for non-parametric material representation and editing” by Lawrence, Ben-Artzi, DeCoro, Matusik, Pfister, et al. …

Recent progress in the measurement of surface reflectance has created a demand for non-parametric appearance representations that are accurate, compact, and easy to use for rendering. Another crucial goal, which has so far received little attention, is editability: for practical use, we must be able to change both the directional and spatial behavior of surface reflectance (e.g., making one material shinier, another more anisotropic, and changing the spatial “texture maps” indicating where each material appears). We introduce an Inverse Shade Tree framework that provides a general approach to estimating the “leaves” of a user-specified shade tree from high-dimensional measured datasets of appearance. These leaves are sampled 1- and 2-dimensional functions that capture both the directional behavior of individual materials and their spatial mixing patterns. In order to compute these shade trees automatically, we map the problem to matrix factorization and introduce a flexible new algorithm that allows for constraints such as non-negativity, sparsity, and energy conservation. Although we cannot infer every type of shade tree, we demonstrate the ability to reduce multi-gigabyte measured datasets of the Spatially-Varying Bidirectional Reflectance Distribution Function (SVBRDF) into a compact representation that may be edited in real time.

1. Ashikhmin, M., Premože, S., and Shirley, P. 2000. A microfacet-based BRDF generator. In Proceedings of ACM SIGGRAPH 2000. Google ScholarDigital Library
2. Chen, W.-C., Bouguet, J.-Y., Chu, M. H., and Grzeszczuk, R. 2002. Light field mapping: efficient representation and hardware rendering of surface light fields. In ACM Transactions on Graphics (SIGGRAPH 2002). Google ScholarDigital Library
3. Cook, R. L. 1984. Shade trees. In Computer Graphics (Proceedings of ACM SIGGRAPH 1984), 223–231. Google ScholarDigital Library
4. Dana, K., van Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics 18, 1, 1–34. Google ScholarDigital Library
5. Furukawa, R., Kawasaki, H., Ikeuchi, K., and Sakauchi, M. 2002. Appearance based object modeling using texture database: acquisition, compression and rendering. In Eurographics Workshop on Rendering, 257–266. Google ScholarDigital Library
6. Gardner, A., Tchou, C., Hawkins, T., and Debevec, P. 2003. Linear light source reflectometry. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 749–758. Google ScholarDigital Library
7. Gill, P., Murray, W., Saunders, M., and Wright, M. 1984. Procedures for optimization problems with a mixture of bounds and general linear constraints. In ACM Transactions on Mathematical Software. Google ScholarDigital Library
8. Goldman, D. B., Curless, B., Hertzmann, A., and Seitz, S. M. 2005. Shape and spatially-varying BRDFs from photometric stereo. In IEEE International Conference on Computer Vision. Google ScholarDigital Library
9. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In Proceedings of ACM SIGGRAPH 1996. Google ScholarDigital Library
10. Han, J. Y., and Perlin, K. 2003. Measuring bidirectional texture reflectance with a kaleidoscope. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 741–748. Google ScholarDigital Library
11. Hartigan, J. A., and Wong, M. A. 1979. A k-means clustering algorithm. Applied Statistics 28, 100–108.Google ScholarDigital Library
12. Heidrich, W., and Seidel, H.-P. 1999. Realistic, hardware-accelerated shading and lighting. In Proceedings of ACM SIGGRAPH 1999. Google ScholarDigital Library
13. Hofmann, T. 1999. Probabilistic latent semantic analysis. In Proceedings of Uncertainty in Artificial Intelligence. Google ScholarDigital Library
14. Hoyer, P. O. 2002. Non-negative sparse coding. In IEEE Workshop on Neural Networks for Signal Processing, 557–565.Google ScholarCross Ref
15. Jaroszkiewicz, R., and McCool, M. D. 2003. Fast extraction of BRDFs and material maps from images. In Graphcs Interface.Google Scholar
16. Kautz, J., and McCool, M. 1999. Interactive rendering with arbitrary BRDFs using separable approximations. In Eurographics Workshop on Rendering, 247–260. Google ScholarDigital Library
17. Lafortune, E. P. F., Foo, S.-C., Torrance, K. E., and Green-Berg, D. P. 1997. Non-linear approximation of reflectance functions. In Proceedings of ACM SIGGRAPH 1997, 117–126. Google ScholarDigital Library
18. Lawrence, J., Rusinkiewicz, S., and Ramamoorthi, R. 2004. Efficient BRDF importance sampling using a factored representation. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3. Google ScholarDigital Library
19. Lee, D., and Seung, H. S. 2000. Algorithms for non-negative matrix factorization. In Proceedings of Neural Information Processing Systems, 556–562.Google Scholar
20. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Transactions on Graphics 22, 2. Google ScholarDigital Library
21. Leung, T., and Malik, J. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. International Journal of Computer Vision 43, 1, 29–44. Google ScholarDigital Library
22. Marschner, S., Westin, S., Lafortune, E., Torrance, K., and Greenberg, D. 1999. Image-Based BRDF measurement including human skin. In Eurographics Workshop on Rendering, 139–152. Google ScholarDigital Library
23. Marschner, S. R., Westin, S. H., Arbree, A., and Moon, J. T. 2005. Measuring and modeling the appearance of finished wood. ACM Transactions on Graphics (SIGGRAPH 2005) 24, 3, 727–734. Google ScholarDigital Library
24. Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 759–769. Google ScholarDigital Library
25. McAllister, D. 2002. A Generalized Surface Appearance Representation for Computer Graphics. PhD thesis, UNC. Google ScholarDigital Library
26. McCool, M. D., Ang, J., and Ahmad, A. 2001. Homomorphic factorization of BRDFs for high-performance rendering. In Proceedings of ACM SIGGRAPH 2001, 185–194. Google ScholarDigital Library
27. Nag, 2005. Numerical Algorithms Group C Library.Google Scholar
28. Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of BRDF models. In Proceedings of the Eurographics Symposium on Rendering, 117–226. Google ScholarDigital Library
29. Olshausen, B. A., and Field, D. J. 2002. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609.Google ScholarCross Ref
30. Peers, P., Vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., and Dutré, P. 2006. A compact factored representation of heterogeneous subsurface scattering. ACM Transactions on Graphics (SIGGRAPH 2006) 25, 3. Google ScholarDigital Library
31. Rusinkiewicz, S. 1998. A new change of variables for efficient BRDF representation. In Eurographics Workshop on Rendering, 11–22.Google ScholarCross Ref
32. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM Transactions on Graphics (SIGGRAPH 2003) 22, 3, 770–779. Google ScholarDigital Library
33. Vasilescu, M. A., and Terzopoulos, D. 2004. TensorTextures: Multilinear image-based rendering. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3. Google ScholarDigital Library
34. Ward, G. J. 1992. Measuring and modeling anisotropic reflection. In Computer Graphics (Proceedings of ACM SIGGRAPH 1992), 265–272. Google ScholarDigital Library