“Image-based rendering of diffuse, specular and glossy surfaces from a single image” by Boivin and Gagalowicz

In this paper, we present a new method to recover an approximation of the bidirectional reflectance distribution function (BRDF) of the surfaces present in a real scene. This is done from a single photograph and a 3D geometric model of the scene. The result is a full model of the reflectance properties of all surfaces, which can be rendered under novel illumination conditions with, for example, viewpoint modification and the addition of new synthetic objects. Our technique produces a reflectance model using a small number of parameters. These parameters nevertheless approximate the BRDF and allow the recovery of the photometric properties of diffuse, specular, isotropic or anisotropic textured objects. The input data are a geometric model of the scene including the light source positions and the camera properties, and a single image captured using this camera. Our algorithm generates a new synthetic image using classic rendering techniques, and a lambertian hypothesis about the reflectance model of the surfaces. Then, it iteratively compares the original image to the new one, and chooses a more complex reflectance model if the difference between the two images is greater than a user-defined threshold.
We present several synthetic images that are compared to the original ones, and some possible applications in augmented reality.

1. Arthur Appel. Some techniques for shading machine renderings of solids. AFIPS 1968 Spring Joint Computer Conf., 32:37-45, 1968.
2. R. Baribeau, M. Rioux, and G. Godin. Color reflectance modeling using a polychromatic laser range sensor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):263-269, February 1992.
3. Loren Carpenter. The a-buffer, an antialiased hidden surface method. Computer Graphics (Proceedings of SIGGRAPH 84), 18(3):103-108, July 1984. Held in Minneapolis, Minnesota.
4. Michael F. Cohen, Shenchang Eric Chen, John R. Wallace, and Donald P. Greenberg. A progressive refinement approach to fast radiosity image generation. In John Dill, editor, Computer Graphics (Proceedings of SIGGRAPH 88), volume 22, pages 75-84, August 1988.
5. Michael F. Cohen and Donald P. Greenberg. The Hemi-Cube: A radiosity solution for complex environments. In B. A. Barsky, editor, Computer Graphics (Proceedings of SIGGRAPH 85), volume 19, pages 31-40, August 1985.
6. Kristin J. Dana, Bram van Ginneken, Shree K. Nayar, and Jan J. Koenderink. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics, 18(1):1-34, January 1999. ISSN 0730-0301.
7. Paul Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Michael Cohen, editor, Computer Graphics (Proceedings of SIGGRAPH 98), Annual Conference Series, pages 189-198. Addison Wesley, July 1998.
8. Paul E. Debevec and Jitendra Malik. Recovering high dynamic range radiance maps from photographs. In Turner Whitted, editor, Computer Graphics (Proceedings of SIGGRAPH 97), Annual Conference Series, pages 369-378. Addison Wesley, August 1997.
9. Paul Ernest Debevec. Modeling and Rendering Architecture from Photographs. PhD thesis, University of California, Berkeley, 1996.
10. D.F DeMenthon and L. Davis. Model-based object pose in 25 lines of code. In Second European Conference on Computer Vision(ECCV), pages 335-343. Springer-Verlag, May 1992.
11. George Drettakis, Luc Robert, and Sylvain Bougnoux. Interactive common illumination for computer augmented reality. In Julie Dorsey and Philipp Slusallek, editors, Eurographics Rendering Workshop 1997, pages 45-56. Springer Wien, June 1997.
12. Fredo Durand. Visibilite tridimensionnelle: etude analytique et applications. Ph.D. thesis, Universite Joseph Fourier, Grenoble, France, July 1999.
13. E. Fiume, A. Fournier, and L. Rudolph. A parallel scan conversion algorithm with anti-aliasing for a general purpose ultracomputer. Computer Graphics (Proceedings of SIGGRAPH 83), 17(3):141-150, July 1983. Held in Detroit, Michigan.
14. Alain Fournier, Atjeng S. Gunawan, and Chris Romanzin. Common illumination between real and computer generated scenes. In Graphics Interface ’93, pages 254-262. Canadian Information Processing Society, May 1993. Held in Toronto, Ontario, Canada.
15. Andrew Glassner. An Introduction to Ray Tracing. Academic Press, New York, NY, 1989.
16. Cindy M. Goral, Kenneth E. Torrance, Donald P. Greenberg, and Bennett Battaile. Modeling the interaction of light between diffuse surfaces. In Hank Christiansen, editor, Computer Graphics (Proceedings of SIGGRAPH 84), volume 18, pages 213-222, July 1984.
17. Press W. H., Teukolsky S.A., Vetterling W.T., and Flannery B.P. Numerical Recipes in C, The Art of Scientific Computing, chapter 10.4 Downhill Simplex Method in Multidimensions, pages 305-309. Cambridge University Press, Cambridge, 1992.
18. Pat Hanrahan, David Salzman, and Larry Aupperle. A rapid hierarchical radiosity algorithm. Computer Graphics (Proceedings of SIGGRAPH 91), 25(4):197-206, July 1991.
19. Nelder J.A. and Mead R. A simplex method for function minimization. Computer Journal, 7:308-313, 1965.
20. Konrad F. Karner, Heinz Mayer, and Michael Gervautz. An image based measurement system for anisotropic reflection. Computer Graphics Forum, 15(3):119-128, August 1996.
21. G. Kay and T. Caelli. Inverting an illumination model from range and intensity maps. CGVIP: Image Understanding, 59:183-201, 1994.
22. Eric P. Lafortune, Sing-Choong Foo, Kenneth E. Torrance, and Donald P. Greenberg. Non-linear approximation of reflectance functions. In Computer Graphics (Proceedings of SIGGRAPH 97), volume 31, pages 117-126, 1997.
23. C. Loscos, M. C. Frasson, G. Drettakis, B. Walter, X. Grainer, and P. Poulin. Interactive virtual relighting and remodeling of real scenes. Available from www.imagis.imag.fr/Publications RT-0230, Institut National de Recherche en Informatique en Automatique (INRIA), Grenoble, France, April 1999.
24. Celine Loscos, George Drettakis, and Luc Robert. Interactive virtual relighting of real scenes. IEEE Transactions on Visualization and Computer Graphics, 6(3):289-305, 2000.
25. J. Lu and J. Little. Reflectance function estimation and shape recovery from image sequence of rotating object. In International Conference on Computer Vision, pages 80-86, June 1995.
26. Stephen R. Marschner and Donald P. Greenberg. Inverse lighting for photography. In Proceedings of the Fifth Color Imaging Conference. Society for Imaging Science and Technology, November 1997.
27. Stephen R. Marschner, Stephen H. Westin, Eric P. F. Lafortune, Kenneth E. Torrance, and Donald P.Greenberg. Image-based brdf measurement including human skin. In Dani Lischinski and Greg Ward Larson, editors, Eurographics Rendering Workshop 1999. Eurographics, June 1999.
28. A. Rosenblum. Data Visualization, chapter Modeling Complex indoor scenes using an analysis/synthesis framework (Andre Gagalowicz). Academic Press, 1994.
29. Imari Sato, Yoichi Sato, and Katsushi Ikeuchi. Illumination distribution from brightness in shadows: Adaptive extimation of illumination distribution with unknown reflectance properties in shadow regions. In Proceedings of IEEE ICCC’99, pages 875-882, September 1999.
30. Kosuke Sato and Katsushi Ikeuchi. Determining reflectance properties of an object using range and brightness images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(11):1139-1153, 1991.
31. Yoichi Sato and Katsushi Ikeuchi. Temporal-color space analysis of reflection. Journal of Optical Society of America, 11(11):2990-3002, November 1994.
32. Yoichi Sato and Katsushi Ikeuchi. Reflectance analysis for 3d computer graphics model generation. Graphical Models and Image Processing, 58(5):437-451, 1996.
33. Yoichi Sato, Mark D. Wheeler, and Katsushi Ikeuchi. Object shape and reflectance modeling from observation. In Turner Whitted, editor, Computer Graphics (Proceedings of SIGGRAPH 97), pages 379-388. Addison Wesley, August 1997.
34. Siu-Hang Or Tien-Tsin Wong, Pheng-Ann Heng and Wai-Yin Ng. Image-based rendering with controllable illumination. In Julie Dorsey and Phillip Slusallek, editors, Rendering Techniques ’97 (Proceedings of the Eighth Eurographics Workshop on Rendering), pages 13-22, New York, NY, 1997. Springer Wien. ISBN 3-211-83001-4.
35. K.E. Torrance, E.M. Sparrow, and R.C. Birkebak. Polarization, directional distribution, and off-specular peak phenomena in light reflected from roughened surfaces. Journal of Optical Society of America, 56(7):916-925, 1966.
36. Jack Tumblin and Holly Rushmeier. Tone reproduction for realistic images. IEEE Computer Graphics and Applications, 13(6):42-48, November 1993.
37. Gregory J. Ward. Measuring and modeling anisotropic reflection. In Edwin E. Catmull, editor, Computer Graphics (Proceedings of SIGGRAPH 92), volume 26, pages 265-272. ACM Press, July 1992.
38. Gregory J. Ward. The radiance lighting simulation and rendering system. In Andrew Glassner, editor, Computer Graphics (Proceedings of SIGGRAPH 94), Annual Conference Series, pages 459-472. ACM Press, July 1994. ISBN 0-89791-667-0. Held in Orlando, Florida.
39. Hank Weghorst, Gary Hooper, and Donald P. Greenberg. Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1):52-69, January 1984.
40. George Wolberg. Digital Image Warping. IEEE Computer Society Press, Los Alamitos, 1990.
41. Y. Yu, P. Debevec, J. Malik, and T. Hawkins. Inverse global illumination : Recovering reflectance models of real scenes from photographs. In A. Rockwood, editor, Computer Graphics (Proceedings of SIGGRAPH 99), volume 19, pages 215-224. Addison Wesley Longman, August 1999.