“Inverse global illumination: recovering reflectance models of real scenes from photographs” by Yu, Debevec, Malik and Hawkins

In this paper we present a method for recovering the reflectance properties of all surfaces in a real scene from a sparse set of photographs, taking into account both direct and indirect illumination. The result is a lighting-independent model of the scene’s geometry and reflectance properties, which can be rendered with arbitrary modifications to structure and lighting via traditional rendering methods. Our technique models reflectance with a low-parameter reflectance model, and allows diffuse albedo to vary arbitrarily over surfaces while assuming that non-diffuse characteristics remain constant across particular regions. The method’s input is a geometric model of the scene and a set of calibrated high dynamic range photographs taken with known direct illumination. The algorithm hierarchically partitions the scene into a polygonal mesh, and uses image-based rendering to construct estimates of both the radiance and irradiance of each patch from the photographic data. The algorithm computes the expected location of specular highlights, and then analyzes the highlight areas in the images by running a novel iterative optimization procedure to recover the diffuse and specular reflectance parameters for each region. Lastly, these parameters are used in constructing high-resolution diffuse albedo maps for each surface. The algorithm has been applied to both real and synthetic data, including a synthetic cubical room and a real meeting room. Re-renderings are produced using a global illumination system under both original and novel lighting, and with the addition of synthetic objects. Side-by-side comparisons show success at predicting the appearance of the scene under novel lighting conditions.

1. AUPPERLE, L., AND HANRAHAN, P. A hierarchical illumination algorithm for surfaces with glossy reflection. In SIGGRAPH 93 (August 1993), pp. 155-164.
2. BAUM, D. R., RUSHMEIER, H. E., AND WINGET, J. M. Improving radiosity solutions through the use of analytically determined form factors. In SIGGRAPH 89 (1989), pp. 325-334.
3. CHEN, E. QuickTime VR – an image-based approach to virtual environment navigation. In SIGGRAPH ’95 (1995).
4. CURLESS, B., AND LEVOY, M. A volumetric method for building complex models from range images. In SIGGRAPH ’96 (1996), pp. 303-312.
5. DANA, K. J., GINNEKEN, B., NAYAR, S. K., AND KOENDERINK, J. J. Reflectance and texture of real-world surfaces. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog. (1997), pp. 151-157.
6. DEBEVEC, P., YU, Y., AND BORSHUKOV, G. Efficient View-Dependent Image- Based Rendering with Projective Texture-Mapping. In 9th Eurographics Workshop on Rendering, (1998), pp. 105-116.
7. DEBEVEC, P. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH 98 (July 1998).
8. DEBEVEC, P. E., AND MALIK, J. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH 97 (August 1997), pp. 369-378.
9. DEBEVEC, P. E., TAYLOR, C. J., AND MALIK, J. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIGGRAPH ’96 (August 1996), pp. 11-20.
10. DRETTAKIS, G., ROBERT, L., AND BOUGNOUX, S. Interactive common illumination for computer augmented reality. In 8th Eurographics workshop on Rendering, St. Etienne, France (May 1997), J. Dorsey and R Slusallek, Eds., pp. 45-57.
11. GORAL, C. M., TORRANCE, K. E., GREENBERG, D. P., AND BATTAILE, B. Modeling the interaction of light between diftuse surfaces. In SIGGRAPH ’84 (1984), pp. 213-222.
12. GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. The Lumigraph. In SIGGRAPH ’96 (1996), pp. 43-54.
13. HANRAHAN, P., SALZMAN, P., AND AUPPERLE, L. A rapid hierarchical radiosity algorithm. In SIGGRAPH 91 (1991), pp. 197-206.
14. HE, X. D., TORRANCE, K. E., SILLION, F., AND GREENBERG, D. P. A comprehensive physical model for light reflection. In SIGGRAPH 91, (August 1991).
15. KAJIYA, J. The rendering equation. In SIGGRAPH ’86 (1986), pp. 143-150.
16. KARNER, K. F., MAYER, H., AND GERVAUTZ, M. An image based measurement system for anisotropic reflection. In EUROGRAPHICSAnnual Conference P1vceedings (1996).
17. LAFORTUNE, E.P.F., FOO, S., TORRANCE,K.E., AND GREENBERG, D.P. Non-Linear Approximation of Reflectance Functions. In SIGGRAPH 97, (1997), pp.117-126.
18. LAVEAU, S., AND FAUGERAS, O. 3-D scene representation as a collection of images. In Proceedings of l2th International Conference on Pattern Recognition (1994), vol. 1, pp. 689-691.
19. LEVOY, M., AND HANRAHAN, P. Light field rendering. In SIGGRAPH ’96 (1996), pp. 31-42.
20. LOSCOS, C., FRASSON, M.-C., DRETTAKIS, G., WALTER, B., GRANIER, X., AND POULIN, P. Interactive Virtual Relighting and Remodeling of Real Scenes. Technical Report, iMAGIS-GRAVIR/IMAG-INRIA, (May 1999), http://wwwimagis.imag.fr/Membres/Celine.Loscos/relight.html.
21. MARSHNER, S. Inverse Rendering for Computer Graphics. PhD thesis, Cornell University, August 1998.
22. MCMILLAN, L., AND BISHOP, G. Plenoptic Modeling: An image-based rendering system. In SIGGRAPH ’95 (1995).
23. NAYAR, S. K., IKEUCHI, K., AND KANADE, T. Shape from interreflections. International Journal of Computer Vision 6, 3 (1991), 173-195.
24. NIMEROFF, J., SIMONCELLI, E., AND DORSEY, J. Efficient re-rendering of naturally illuminated environments. In 5th Ewvgraphics Workshop on Rendering (1994).
25. Oren, M., and Nayar, S.K., “Generalization of Lambert’s Reflectance Model”, Computer Graphics P1vceedings, Annual Conference Series, pp.239-246 (1994).
26. PRESS, W., FLANNERY, B., TEUKOLSKY, S., AND VETTERLING, W. Numerical Recipes in C. Cambridge Univ. Press, New York, 1988.
27. SATO, Y., WHEELER, M. D., AND IKEUCHI, K. Object shape and reflectance modeling from observation. In SIGGRAPH ’97 (1997), pp. 379-387.
28. SILLION, F. X., AND PUECH, C. Radiosity and Global Illumination. Morgan Kaufmann Publishers, San Francisco, 1994.
29. SZELISKI, R., AND SHUM, H.-Y. Creating full view panoramic image mosaics and environment maps. In SIGGRAPH 97 (1997), pp. 251-258.
30. TURK, G., AND LEVOY, M. Zippered polygon meshes from range images. In SIGGRAPH ’94 (1994), pp. 311-318.
31. VEACH, E., AND GUIBAS, L. J. Metropolis light transport. In SIGGRAPH 97 (August 1997), pp. 65-76.
32. WARD, G. J. Measuring and modeling anisotropic reflection. In SIGGRAPH ’92 (July 1992), pp. 265-272.
33. WARD, G. J. The RADIANCE lighting simulation and rendering system. In SIGGRAPH ’94 (July 1994), pp. 459-472.
34. Y.CHEN, AND MEDIONI, G. Object modeling from multiple range images. Image and Vision Computing 10, 3 (April 1992), pp.145-155.
35. WONG T.-T., HENG P.-A., OR S.-H. AND NG W.-Y. Image-based Rendering with Controllable Illumination. In 8th Eurographics Workshop on Rendering, (June 1997), pp.13-22.
36. Yu, Y., AND MALIK, J. Recovering photometric properties of architectural scenes from photographs. In SIGGRAPH 98 (July 1998), pp. 207-217.
37. Yu, Y., AND WU, H. A Rendering Equation for Specular Transfers and its Integration into Global Illumination. Eurographics’97, In J. Computer Graphics Forum, 16(3), (1997), pp. 283-292.