“Fast separation of direct and global components of a scene using high frequency illumination” by Nayar, Krishnan, Grossberg and Raskar

We present fast methods for separating the direct and global illumination components of a scene measured by a camera and illuminated by a light source. In theory, the separation can be done with just two images taken with a high frequency binary illumination pattern and its complement. In practice, a larger number of images are used to overcome the optical and resolution limitations of the camera and the source. The approach does not require the material properties of objects and media in the scene to be known. However, we require that the illumination frequency is high enough to adequately sample the global components received by scene points. We present separation results for scenes that include complex interreflections, subsurface scattering and volumetric scattering. Several variants of the separation approach are also described. When a sinusoidal illumination pattern is used with different phase shifts, the separation can be done using just three images. When the computed images are of lower resolution than the source and the camera, smoothness constraints are used to perform the separation using a single image. Finally, in the case of a static scene that is lit by a simple point source, such as the sun, a moving occluder and a video camera can be used to do the separation. We also show several simple examples of how novel images of a scene can be computed from the separation results.

1. Arikan, O., Forsyth, D., and O’Brien, J. 2005. Fast and detailed approximate global illumination by irradiance decomposition. ACM Trans. on Graph. 24, 3, 1108–1114. Google ScholarDigital Library
2. Bouguet, J., and Perona, P. 1999. 3d photography using shadows in dual-space geometry. IJCV35, 2 (November), 129–149. Google ScholarDigital Library
3. Chandrasekhar, S. 1950. Radiative Transfer. Clarendon Press, Oxford, UK. reprinted by Dover Publications, 1960.Google Scholar
4. Chuang, Y., Zongker, D., Hindorff, J., Curless, B., Salesin, D., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proc. of SIGGRAPH, ACM Press, 121–130. Google ScholarDigital Library
5. Debevec, P., Hawkins, T., C., T., Duiker, H., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proc. of SIGGRAPH, ACM Press, 145–156. Google ScholarDigital Library
6. Forsyth, D., and Zisserman, A. 1991. Reflections on shading. IEEE Trans. on PAMI 13, 7, 671–679. Google ScholarDigital Library
7. Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The lumigraph. In Proc. of SIGGRAPH, ACM Press, 43–54. Google ScholarDigital Library
8. Immel, D., Cohen, M., and Greenberg, D. 1986. A radiosity method for non-diffuse environments. In Proc. of SIGGRAPH, ACM Press, 133–142. Google ScholarDigital Library
9. Kajiya, J. T. 1986. The Rendering Equation. In Proc. of SIGGRAPH, ACM Press, New York, NY, USA, 143–150. Google ScholarDigital Library
10. Koenderink, J., and Van Doorn, A. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. JOSA 73, 6 (June), 843–850.Google ScholarCross Ref
11. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proc. of SIGGRAPH, ACM Press, 31–42. Google ScholarDigital Library
12. Lin, Z., Wong, T., and Shum, H. 2002. Relighting with the reflected irradiance field: Representation, sampling and reconstruction. IJCV 49, 2-3 (September), 229–246. Google ScholarDigital Library
13. Mallick, S. P., Zickler, T., Belhumeur, P. N., and Kriegman, D. J. 2006. Specularity removal in images and videos: A PDE approach. In Proc. of ECCV.Google Scholar
14. Nayar, S., Ikeuchi, K., and Kanade, T. 1991. Shape from interreflections. IJCV 6, 3, 173–195. Google ScholarDigital Library
15. Peers, P., and Dutré, P. 2003. Wavelet environment matting. In Eurographics Symposium on Rendering, ACM Press, 157–166. Google ScholarDigital Library
16. Seitz, S., Matsushita, Y., and Kutulakos, K. 2005. A theory of inverse light transport. In Proc. of ICCV, II: 1440–1447. Google ScholarDigital Library
17. Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., and Lensch, H. 2005. Dual photography. ACM Trans. on Graph. 24, 3, 745–755. Google ScholarDigital Library
18. Shim, H., and Chen, T. 2005. A statistical framework for image-based relighting. In Proc. of ICASSP.Google Scholar
19. Sloan, P., Kautz, J., and Snyder, J. 2002. Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. In Proc. of SIGGRAPH, ACM Press, 527–536. Google ScholarDigital Library
20. Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. In Proc. of SIGGRAPH, ACM Press, 770–779. Google ScholarDigital Library
21. Woodham, R. 1980. Photometric Method for Determining Surface Orientation from Multiple Images. Optical Engineering 19, 1 (January), 139–144.Google ScholarCross Ref
22. Wust, C., and Capson, D. W. 1991. Surface profile measurement using color fringe projection. Machine Vision and Applications 4, 193–203. Google ScholarDigital Library
23. Zhu, J., and Yang, Y. 2004. Frequency-based environment matting. In Pacific Conf. on Comp. Graph. and Apps., 402–410. Google ScholarDigital Library
24. Zongker, D., Werner, D., Curless, B., and Salesin, D. 1999. Environment matting and compositing. In Proc. of SIGGRAPH, ACM Press, 205–214. Google ScholarDigital Library