“Light field mapping: efficient representation and hardware rendering of surface light fields” by Chen, Bouguet, Chu and Grzeszczuk

A light field parameterized on the surface offers a natural and intuitive description of the view-dependent appearance of scenes with complex reflectance properties. To enable the use of surface light fields in real-time rendering we develop a compact representation suitable for an accelerated graphics pipeline. We propose to approximate the light field data by partitioning it over elementary surface primitives and factorizing each part into a small set of lower-dimensional functions. We show that our representation can be further compressed using standard image compression techniques leading to extremely compact data sets that are up to four orders of magnitude smaller than the input data. Finally, we develop an image-based rendering method, light field mapping, that can visualize surface light fields directly from this compact representation at interactive frame rates on a personal computer. We also implement a new method of approximating the light field data that produces positive only factors allowing for faster rendering using simpler graphics hardware than earlier methods. We demonstrate the results for a variety of non-trivial synthetic scenes and physical objects scanned through 3D photography.

1. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, 1995. Google Scholar
2. S. Boivin and A. Gagalowicz. Image-Based Rendering of Diffuse, Specular and Glossy Surfaces From a Single Image. Proceedings of SIGGRAPH 2001, pages 107-116, August 2001. Google Scholar
3. J-Y. Bouguet and P. Perona. 3D Photography Using Shadows in Dual-Space Geometry. International Journal of Computer Vision, 35(2):129-149, December 1999. Google Scholar
4. C. Buehler, M. Bosse, L. McMillan, S. J. Gortler, and M. F. Cohen. Unstructured Lumigraph Rendering. Proceedings of SIGGRAPH 2001, pages 425-432, August 2001. Google Scholar
5. J-X. Chai, X. Tong, S-C. Chan, and H-Y. Shum. Plenoptic Sampling. Proceedings of SIGGRAPH 2000, pages 307-318, July 2000. Google Scholar
6. W-C. Chen, R. Grzeszczuk, and J-Y. Bouguet. Light Field Mapping: Hardware-accelerated Visualization of Surface Light Fields. Published as part of “Acquisition and Visualization of Surface Light Fields,” SIGGRAPH 2001 Course Notes for Course 46, pages 410-416, August 2001.Google Scholar
7. R. L. Cook and K. E. Torrance. A Reflectance Model for Computer Graphics. ACM Transactions on Graphics, 1(1):7-24, January 1982. Google Scholar
8. B. Curless and M. Levoy. Better Optical Triangulation through Spacetime Analysis. Proc. 5th Int. Conf. Computer Vision, Boston, USA, pages 987-993, 1995. Google Scholar
9. P. Debevec, T. Hawkins, C. Tchou, H-P. Duiker, W. Sarokin, and M. Sagar. Acquiring the Reflectance Field of a Human Face. Proceedings of SIGGRAPH 2000, pages 145-156, July 2000. Google Scholar
10. P. E. Debevec, C. J. Taylor, and J. Malik. Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach. Proceedings of SIGGRAPH 96, pages 11-20, August 1996. Google Scholar
11. P. E. Debevec, Y. Yu, and G. D. Borshukov. Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping. Eurographics Rendering Workshop 1998, pages 105-116, June 1998. Google Scholar
12. A. Fournier. Separating Reflection Functions for Linear Radiosity. Eurographics Rendering Workshop 1995, pages 296-305, June 1995. Google Scholar
13. A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Kluwer Academic Publishers, 1992. Google Scholar
14. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. The Lumigraph. Proceedings of SIGGRAPH 96, pages 43-54, August 1996. Google Scholar
15. W. Heidrich and H-P. Seidel. Realistic, Hardware-Accelerated Shading and Lighting. Proceedings of SIGGRAPH 99, pages 171-178, August 1999. Google Scholar
16. J. Kautz and M. D. McCool. Interactive Rendering with Arbitrary BRDFs using Separable Approximations. Eurographics Rendering Workshop 1999, June 1999. Google Scholar
17. J. Kautz and H-P. Seidel. Towards Interactive Bump Mapping with Anisotropic Shift-Variant BRDFs. 2000 SIGGRAPH / Eurographics Workshop on Graphics Hardware, pages 51-58, August 2000. Google Scholar
18. J. J. Koenderink and A. J. van Doorn. Bidirectional Reflection Distribution Function Expressed in Terms of Surface Scattering Modes. In European Conference on Computer Vision, pages II:28-39, 1996. Google Scholar
19. E. P. F. Lafortune, S-C. Foo, K. E. Torrance, and D. P. Greenberg. Non-Linear Approximation of Reflectance Functions. Proceedings of SIGGRAPH 97, pages 117-126, August 1997. Google Scholar
20. D. D. Lee and H. S. Seung. Learning the Parts of Objects by Non-Negative Matrix Factorization. Nature, 401:788-791, 1999.Google Scholar
21. Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich, and Hans-Peter Seidel. Image-Based Reconstruction of Spatially Varying Materials. In Twelveth Eurographics Rendering Workshop 2001, pages 104-115. Eurographics, June 2001. Google Scholar
22. M. Levoy and P. Hanrahan. Light Field Rendering. Proceedings of SIGGRAPH 96, pages 31-42, August 1996. Google Scholar
23. M. Magnor and B. Girod. Data Compression for Light Field Rendering. IEEE Trans. Circuits and Systems for Video Technology, 10(3):338-343, April 2000. Google Scholar
24. T. Malzbender, D. Gelb, and H. Wolters. Polynomial Texture Maps. Proceedings of SIGGRAPH 2001, pages 519-528, August 2001. Google Scholar
25. Michael D. McCool, Jason Ang, and Anis Ahmad. Homomorphic Factorization of BRDFs for High-Performance Rendering. Proceedings of SIGGRAPH 2001, pages 171-178, August 2001. Google Scholar
26. L. McMillan and G. Bishop. Plenoptic Modeling: An Image-Based Rendering System. Proceedings of SIGGRAPH 95, pages 39-46, August 1995. Google Scholar
27. G. S. P. Miller, S. Rubin, and D. Ponceleon. Lazy Decompression of Surface Light Fields for Precomputed Global Illumination. Eurographics Rendering Workshop 1998, pages 281-292, June 1998.Google Scholar
28. K. Nishino, Y. Sato, and K. Ikeuchi. Eigen-Texture Method: Appearance Compression Based on 3D Model. In Proceedings of the IEEE Computer Science Conference on Computer Vision and Pattern Recognition (CVPR-99), pages 618-624, 1999.Google Scholar
29. P. Poulin and A. Fournier. A Model for Anisotropic Reflection. Proceedings of SIGGRAPH 90, 24(4):273-282, August 1990. Google Scholar
30. K. Pulli, M. Cohen, T. Duchamp, H. Hoppe, L. Shapiro, and W. Stuetzle. View-based Rendering: Visualizing Real Objects from Scanned Range and Color Data. Eurographics Rendering Workshop 1997, pages 23-34, June 1997. Google Scholar
31. R. Ramamoorthi and P. Hanrahan. A Signal-Processing Framework for Inverse Rendering. Proceedings of SIGGRAPH 2001, pages 117-128, August 2001. Google Scholar
32. Sam Roweis. EM algorithms for PCA and SPCA. In Advances in Neural Information Processing Systems, volume 10. The MIT Press, 1998. Google Scholar
33. Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object Shape and Reflectance Modeling from Observation. Proceedings of SIGGRAPH 97, pages 379-388, August 1997. Google Scholar
34. P. Schröder and W. Sweldens. Spherical Wavelets: Efficiently Representing Functions on the Sphere. Proceedings of SIGGRAPH 95, pages 161-172, August 1995. Google Scholar
35. J. Spitzer. Texture Compositing With Register Combiners. Game Developers Conference, April 2000.Google Scholar
36. G. Taubin and J. Rossignac. Geometric Compression Through Topological Surgery. ACM Transactions on Graphics, 17(2):84-115, April 1998. Google Scholar
37. K. E. Torrance and E. M. Sparrow. Polarization, Directional Distribution, and Off-Specular Peak Phenomena in Light Reflected from Roughened Surfaces. Journal of Optical Society of America, 56(7), 1966.Google Scholar
38. G. J. Ward. Measuring and Modeling Anisotropic Reflection. Proceedings of SIGGRAPH 92, 26(2):265-272, July 1992. Google Scholar
39. D. N. Wood, D. I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. H. Salesin, and W. Stuetzle. Surface Light Fields for 3D Photography. Proceedings of SIGGRAPH 2000, pages 287-296, July 2000. Google Scholar
40. Y. Yu, P. E. Debevec, J. Malik, and T. Hawkins. Inverse Global Illumination: Recovering Reflectance Models of Real Scenes From Photographs. Proceedings of SIGGRAPH 99, pages 215-224, August 1999. Google Scholar