“TensorTextures: multilinear image-based rendering” by Vasilescu and Terzopoulos

This paper introduces a tensor framework for image-based rendering. In particular, we develop an algorithm called TensorTextures that learns a parsimonious model of the bidirectional texture function (BTF) from observational data. Given an ensemble of images of a textured surface, our nonlinear, generative model explicitly represents the multifactor interaction implicit in the detailed appearance of the surface under varying photometric angles, including local (per-texel) reflectance, complex mesostructural self-occlusion, interreflection and self-shadowing, and other BTF-relevant phenomena. Mathematically, TensorTextures is based on multilinear algebra, the algebra of higher-order tensors, hence its name. It is computed through a decomposition known as the N-mode SVD, an extension to tensors of the conventional matrix singular value decomposition (SVD). We demonstrate the application of TensorTextures to the image-based rendering of natural and synthetic textured surfaces under continuously varying viewpoint and illumination conditions.

1. DANA, K. J., VAN GINNEKEN, B., NAYAR, S. K., AND KOENDERINK, J. J. 1999. Reflectance and texture of real-world surfaces. ACM Trans. Graphics 18, 1, 1–34. Google ScholarDigital Library
2. FURUKAWA, R., KAWASAKI, H., IKEUCHI, K., AND SAKAUCHI, M. 2002. Appearance based object modelling using texture database: Acquisition, compression and rendering. In 8th Eurographics Workshop on Virtual Environments, 257–266. Google ScholarDigital Library
3. GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. 1996. The lumigraph. Computer Graphics (Proc. SIGGRAPH) 30, 43–45. Google ScholarDigital Library
4. HAN, J. Y. AND PERLIN, K. 2003. Measuring bidirectional texture reflectance with a kaleidoscope. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 741–748. Google ScholarDigital Library
5. KOENDERINK, J. J., AND VAN DOORN, A. J. 1996. Illuminance texture due to surface mesostructure. Journal of the Optical Society of America 13, 3, 452–463.Google ScholarCross Ref
6. KOLDA, T. G. 2001. Orthogonal tensor decompositions. SIAM J. Matrix Analysis and Applications 23, 1, 243–255. Google ScholarDigital Library
7. KOUDELKA, M. L., MAGDA, S., BELHUMEUR, P. N., AND KRIEGMAN, D. J. 2003. Acquisition, compression, and synthesis of bidirectional texture functions. In Proc. 3rd Int. Workshop on Texture Analysis and Synthesis (Oct), Nice, France, 59–64.Google Scholar
8. KROONENBERG, P., ANDDE LEEUW, J. 1980. Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45, 69–97.Google ScholarCross Ref
9. LATHAUWER, L. DE, MOOR, B. DE, AND VANDEWALLE, J. 2000. On the best rank-1 and rank-(R1, R2, …, Rn) approximation of higher-order tensors. SIAM J. Matrix Analysis and Applications 21, 4, 1324–1342. Google ScholarDigital Library
10. LEVOY, M., AND HANRAHAN, R. 1996. Light field rendering. Computer Graphics (Proc. SIGGRAPH) 30, 31–42. Google ScholarDigital Library
11. LIU, X., YU, Y., AND SHUM, H.-Y. 2001. Synthesizing bidirectional texture functions for real-world surfaces. ACM Trans. Graphics (Proc. SIGGRAPH) 20, 97–106. Google ScholarDigital Library
12. MALZBENDER, T., GELB, D., AND WOLTERS, H. 2001. Polynomial texture maps. Computer Graphics (SIGGRAPH 01) (Aug), 519–528. Google ScholarDigital Library
13. MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 2003. A data driven reflectance model. Computer Graphics (SIGGRAPH 03) 22, 3 (July), 759–769. Google ScholarDigital Library
14. MESETH, J., MULLER, G., SATTLER, M., AND KLEIN, R. 2003. BTF rendering for virtual environments. In Proc. Virtual Concepts 2003 (Nov), 356–363.Google Scholar
15. SATTLER, M., SARLETTE, R., AND KLEIN, R. 2003. Efficient and realistic visualization of cloth. In Proc. Eurographics Symposium on Rendering, 167–177. Google ScholarDigital Library
16. SUYKENS, F., BERGE, K. VON, LAGAE, A., AND KLEIN, R. 2003. Interactive rendering with bidirectional texture functions. Comp. Graphics Forum (Proc. Eurographics) 22, 3, 463–472.Google ScholarCross Ref
17. TEO, P. AND HEEGER, P. 1994. Perceptual image distortion. In Proc. IEEE Conf. Image Processing (Nov), 982–986.Google ScholarCross Ref
18. TONG, X., ZHANG, J., LIU, L., WANG, X., GUO, B., AND SHUM, H.-Y. 2002. Synthesis of bidirectional texture functions on arbitrary surfaces. ACM Trans. Graphics (Proc. SIGGRAPH) 21, 3, 665–672. Google ScholarDigital Library
19. TUCKER, L. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279–311.Google ScholarCross Ref
20. VASILESCU, M. A. O., AND TERZOPOULOS, D. 2002. Multilinear analysis of image ensembles: TensorFaces. In Proc. European Conf. Computer Vision, 447–460. Google ScholarDigital Library
21. WANG, L., WANG, X., TONG, X., LIN, S., HU, S., GUO, B., AND SHUM, H. Y. 2003. View displacement mapping. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 3, 334–339. Google ScholarDigital Library