“Relighting with 4D incident light fields” by Masselus, Peers, Dutré and Willems

We present an image-based technique to relight real objects illuminated by a 4D incident light field, representing the illumination of an environment. By exploiting the richness in angular and spatial variation of the light field, objects can be relit with a high degree of realism.We record photographs of an object, illuminated from various positions and directions, using a projector mounted on a gantry as a moving light source. The resulting basis images are used to create a subset of the full reflectance field of the object. Using this reflectance field, we can create an image of the object, relit with any incident light field and observed from a flxed camera position.To maintain acceptable recording times and reduce the amount of data, we propose an efficient data acquisition method.Since the object can be relit with a 4D incident light field, illumination effects encoded in the light field, such as shafts of shadow or spot light effects, can be realized.

1. ADELSON, E. H., AND BERGEN, J. R. 1991. The plenoptic function and the elements of early vision. In Computational models of visual processing, Landy and Movshon, Eds., MIT press.Google Scholar
2. ALIAGA, D. G., AND CARLBOM, I. 2001. Plenoptic stitching: A scalable method for reconstructing 3D interactive walkthroughs. In SIGGRAPH 2001, Computer Graphics Proceedings, ACM Press / ACM SIGGRAPH, E. Fiume, Ed., Annual Conference Series, SIGGRAPH, 443–450. Google Scholar
3. CHEN, W.-C., BOUGUET, J.-Y., CHU, M. H., AND GRZESZCZUK, R. 2002. Light field mapping: Efficient representation and hardware rendering of surface light fields. In SIGGRAPH 2002 Conference Proceedings, ACM Press / ACM SIGGRAPH, J. Hughes, Ed., Annual Conference Series, SIGGRAPH, 447–456. Google Scholar
4. DEBEVEC, P., AND MALIK, J. 1997. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH 97 Conference Graphics Proceedings, Addison Wesley, T. Whitted, Ed., Annual Conference Series, ACM SIGGRAPH, 369–378. Google Scholar
5. DEBEVEC, P., HAWKINS, T., TCHOU, C., DUIKER, H.-P., SAROKIN, W., AND SAGAR, M. 2000. Acquiring the reflectance field of a human face. In SIGGRAPH 2000, Computer Graphics Proceedings, Addison Wesley, K. Akeley, Ed., Annual Conference Series, ACM SIGGRAPH, 145–156. Google Scholar
6. DEBEVEC, P., WENGER, A., TCHOU, C., GARDNER, A., WAESE, J., AND HAWKINS, T. 2002. A lighting reproduction approach to live-action compositing. In SIGGRAPH 2002 Conference Proceedings, ACM Press/ACM SIGGRAPH, J. Hughes, Ed., Annual Conference Series, SIGGRAPH, 547–556. Google Scholar
7. DEBEVEC, P. 1998. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH 98 Conference Proceedings, Addison Wesley, M. Cohen, Ed., Annual Conference Series, ACM SIGGRAPH, 189–198. ISBN 0-89791-999-8. Google Scholar
8. GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. 1996. The lumigraph. In SIGGRAPH 96, Computer Graphics Proceedings, Addison Wesley, H. Rushmeier, Ed., Annual Conference Series, ACM SIGGRAPH, 43–54. Google Scholar
9. HAWKINS, T., COHEN, J., AND DEBEVEC, P. 2001. A photometric approach to digitizing cultural artifacts. In In 2nd International Symposium on Virtual Reality, Archaeology, and Cultural Heritage, Glyfada, Greece, November 2001. Google ScholarDigital Library
10. IHM, I., PARK, S., AND LEE, R. K. 1997. Rendering of spherical light fields. In 5th Pacific Conference on Computer Graphics and Applications (Pacific Graphics ’97), Y. G. Shin and J. K. Hahn, Eds., Annual Conference Series, IEEE Computer Society, 59–69. Google Scholar
11. J NIMEROFF, E. S., AND DORSEY, J. 1994. Efficient re-rendering of naturally illuminated environments. In Eurographics Rendering Workshop 1994, Springer-Verlag, Darmstadt, Germany, EG.Google Scholar
12. LEVOY, M., AND HANRAHAN, P. 1996. Light field rendering. In SIGGRAPH 96 Conference Proceedings, Addison Wesley, H. Rushmeier, Ed., Annual Conference Series, ACM SIGGRAPH, 31–42. Google Scholar
13. LIN, Z., WONG, T.-T., AND SHUM, H.-Y. 2001. Relighting with the reflected irradiance field: Representation, sampling and reconstruction. IEEE Computer Vision and Pattern Recognition 1 (Dec.), 561–567. Google Scholar
14. MASSELUS, V., DUTRÉ, P., AND ANRYS, F. 2002. The free-form light stage. In Rendering Techniques EG 2002, Annual Conference Series, EG, 247–255. Google Scholar
15. MATUSIK, W., PFISTER, H., NGAN, A., BEARDSLEY, P., ZIEGLER, R., AND MCMILLAN, L. 2002. Image-based 3D photography using opacity hulls. In SIGGRAPH 2002 Conference Proceedings, ACM Press/ACM SIGGRAPH, J. Hughes, Ed., Annual Conference Series, SIGGRAPH, 427–437. Google Scholar
16. MATUSIK, W., PFISTER, H., ZIEGLER, R., NGAN, A., AND MCMILLAN, L. 2002. Acquisition and rendering of transparant and refractive objects. In Rendering Techniques EG 2002, Annual Conference Series, EG, 267–277. Google Scholar
17. MELISSA L KOUDELKA, PETER N BELHUMEUR, S. M., AND KRIEGMAN, D. J. 2001. Image-based modeling and rendering of surfaces with arbitrary brdfs. In Computer Vision and Pattern Recognition (CVPR’01), Computer Graphics Proceedings, Annual Conference Series.Google Scholar
18. WONG, T.-T., HENG, P.-A., OR, S.-H., AND NG, W.-Y. 1997. Image-based rendering with controllable illumination. In Eurographics Rendering Workshop 1997, Springer Wien, J. Dorsey and P. Slusallek, Eds., Eurographics, 13–22. Google ScholarDigital Library
19. WONG, T.-T., HENG, P.-A., AND FU, C.-W. 2001. Interactive relighting of panoramas. IEEE Computer Graphics and Applications 21, 2 (Mar./ Apr.), 32–41. Google Scholar
20. WOOD, D. N., AZUMA, D. I., ALDINGER, K., CURLESS, B., DUCHAMP, T., SALESIN, D. H., AND STUETZLE, W. 2000. Surface light fields for 3D photography. In SIGGRAPH 2000, Computer Graphics Proceedings, ACM Press / ACM SIGGRAPH, K. Akeley, Ed., Annual Conference Series, SIGGRAPH, 287–296. Google ScholarDigital Library