“Synthesizing bidirectional texture functions for real-world surfaces” by Liu, Yu and Shum

In this paper, we present a novel approach to synthetically generating bidirectional texture functions (BTFs) of real-world surfaces. Unlike a conventional two-dimensional texture, a BTF is a six-dimensional function that describes the appearance of texture as a function of illumination and viewing directions. The BTF captures the appearance change caused by visible small-scale geometric details on surfaces. From a sparse set of images under different viewing/lighting settings, our approach generates BTFs in three steps. First, it recovers approximate 3D geometry of surface details using a shape-from-shading method. Then, it generates a novel version of the geometric details that has the same statistical properties as the sample surface with a non-parametric sampling method. Finally, it employs an appearance preserving procedure to synthesize novel images for the recovered or generated geometric details under various viewing/lighting settings, which then define a BTF. Our experimental results demonstrate the effectiveness of our approach.

1. E.H. Adelson and J.R. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, Mass., 1991.
2. J.F. Blinn. Models of light reflection for computer synthesized pictures. In Computer Graphics, SIGGRAPH’77, Vol.11, pages 192-198, 1977.
3. J.F. Blinn. Simulation of wrinkled surfaces. In SIGGRAPH’78, pages 286-292, 1978.
4. J. De Bonet. Multiresolution sampling procedure for analysis and synthesis of texture images. In Proc. of Siggraph, pages 361-368, 1997.
5. M.J. Brooks and B.K.P. Horn. Shape and source from shading. In Proc. Intern. Joint Conf. Art. Int., pages 932-936, 1988.
6. E. Catmull. A Subdivision Algorithm for Computer Display of Curved Surfaces. PhD thesis, Univ. of Utah, 1974. Report UTEC-CSc-74-133.
7. K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink. Reflectance and texture of real world surfaces. ACM Transactions on Graphics, 18(1):1-34, 1999.
8. K.J. Dana and S.K. Nayar. Histogram model for 3d textures. In Proc. of IEEE Conf. on Comp. Vision and Patt. Recog., 1998.
9. K.J. Dana and S.K. Nayar. Correlation model for 3d textures. In International Conference Computer Vision, 1999.
10. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar. Acquiring the reflectance field of a human face. In Proceedings of SIGGRAPH, pages 145-156, 2000.
11. P.E. Debevec, C.J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIG-GRAPH ’96, pages 11-20, 1996.
12. A. Efros and T. Leung. Texture synthesis by non-parametric sampling. In International Conference Computer Vision, 1999.
13. O. Faugeras. Three-Dimensional Computer Vision. The MIT Press, Cambridge, Massachusetts, 1993.
14. S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, pages 43-54, 1996.
15. F.R. Hampel, P.J. Rousseeuw, E.M. Ronchetti, and W.A. Stahel. Robust Statistics. John Wiley & Sons, New York, 1986.
16. C. Harris and M.J. Stephens. A combined corner and edge detector. In Alvey Vision Conference, pages 147-152, 1988.
17. D.J. Heeger and J.R. Bergen. Pyramid-based texture analysis/synthesis. In Proc. of SIGGRAPH, pages 229-238, 1995.
18. W. Heidrich, K. Daubert, J. Kautz, and H.-P. Seidel. Illuminating micro geometry based on precomputed visibility. In SIGGRAPH’2000, pages 455-464, 2000.
19. B.K.P. Horn and M.J. Brooks. The variational approach to shape from shading. Computer Vision, Graphics & Image Processing, 33:174-208, 1986.
20. J.J. Koenderink and A.J. van Doorn. Illuminance texture due to surface mesostructure. J. Opt. Soc. Am.A, 13(3):452-463, 1996.
21. Y.G. Leclerc and A.F. Bobick. The direct computation of height from shading. In Proc. of IEEE Conf. on Comp. Vision and Patt. Recog., pages 552-558, 1991.
22. T. Leung and J. Malik. Recognizing surfaces using three dimensional textons. In International Conference Computer Vision, 1999.
23. M. Levoy and P. Hanrahan. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, pages 31-42, 1996.
24. S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and D. P. Greenberg. Image-based brdf measurement including human skin. In 10th Eurographics Workshop on Rendering, pages 139-152, 1999.
25. L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. In Computer Graphics Proceedings, Annual Conference Series, pages 39-46, 1995.
26. D.M. Mount. ANN Programming Manual. Dept. Comput. Sci., Univ. of Maryland, College Park, Maryland, 1998.
27. S.K. Nayar, K. Ikeuchi, and T. Kanade. Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Trans. Robotics and Automa-tion, 6(4):418-431, 1990.
28. K. Nishino, Y. Sato, and K. Ikeuchi. Eigen-texture method: appearance compression based on 3d model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’99), pages 618-624, 1999.
29. E. Praun, A. Finkelstein, and H. Hoppe. Lapped textures. In Siggraph’00, pages 465-470, 2000.
30. W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling. Numerical Recipes in C. Cambridge Univ. Press, New York, 1988.
31. H. Rushmeier, G. Taubin, and A. Gue’ziec. Applying shape from lighting variation to bump map capture. In Proceedings of the Eighth Eurographics Workshop on Rendering, pages 35-44, 1997.
32. Y. Sato, M.D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In Computer Graphics Proceedings, Annual Conference Series, pages 379-388, 1997.
33. S.M. Seitz and C.R. Dyer. Photorealistic scene reconstruction by voxel coloring. In Proc. of IEEE Conf. on Comp. Vision and Patt. Recog., pages 1067- 1073, 1997.
34. J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Proc. of SIGGRAPH, pages 231-242, 1998.
35. H.-Y. Shum and L.-W. He. Rendering with concentric mosaics. In Proc. of SIGGRAPH, pages 299-306, 1999.
36. E. Simoncelli and J. Portilla. Texture characterization via joint statistics of wavelet coefficient magnitudes. In Fifth International Conference on Image Processing, Vol.1, pages 62-66, 1998.
37. R. Szeliski and H. Shum. Creating full view panoramic mosaics and environment maps. In Computer Graphics Proceedings, Annual Conference Series, pages 251-258, 1997.
38. L.-Y. Wei and M. Levoy. Fast texture synthesis using tree-structured vector quantization. In Proceedings of Siggraph, pages 479-488, 2000.
39. D.N. Wood, D.I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle. Surface light fields for 3d photography. In SIGGRAPH’00, pages pp.287-296, 2000.
40. R.J. Woodham. Photometric method for determining surface orientation from multiple images. In B.K.P. Horn and M.J. Brooks, editors, Shape from Shading, pages 513-532. MIT Press, 1989.
41. Y. Xu, B. Guo, and H.-Y. Shum. Chaos mosaic: Fast and memory efficient texture synthesis. Technical Report MSR-TR-2000-32, Microsoft Research, 2000.
42. Y. Yu. Modeling and Editing Real Scenes with Image-Based Techniques. PhD thesis, Computer Science Division, UC Berkeley, 2000.
43. Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In Proc. of SIG- GRAPH, pages 215-224, 1999.
44. Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. In Proceedings of SIGGRAPH, pages 207-217, 1998.
45. S. Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy (frame)-towards a unified theory for texture modeling. International Journal of Computer Vision, 27(2):107-126, 1998.