“Dual photography” by Sen, Chen, Garg, Marschner, Horowitz, et al. …

We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, inter-reflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.

1. Debevec, P., and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In SIGGRAPH ’97, 369–378. Google ScholarDigital Library
2. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the Reflectance Field of a Human Face. In SIGGRAPH ’00, 145–156. Google ScholarDigital Library
3. Goesele, M., Lensch, H. P. A., Lang, J., Fuchs, C., and Seidel, H.-P. 2004. DISCO: Acquisition of Translucent Objects. In SIGGRAPH ’04, 835–844. Google ScholarDigital Library
4. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. In SIGGRAPH ’96, 43–54. Google ScholarDigital Library
5. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In SIGGRAPH ’96, 31–42. Google ScholarDigital Library
6. Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M. 2004. Synthetic Aperture Confocal Imaging. In SIGGRAPH ’04, 825–834. Google ScholarDigital Library
7. Masselus, V., Peers, P., Dutré, P., and Willems, Y. D. 2003. Relighting with 4D Incident Light Fields. In SIGGRAPH ’03, 613–620. Google ScholarDigital Library
8. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., and McMillan, L. 2002. Image-Based 3D Photography Using Opacity Hulls. In SIGGRAPH ’02, 427–437. Google ScholarDigital Library
9. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. 2002. Acquisition and Rendering of Transparent and Refractive Objects. In Eurographics Symposium on Rendering, 277–288. Google ScholarDigital Library
10. Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-Refined Reflectance Functions for Natural Illumination. In Eurographics Symposium on Rendering, 299–308. Google ScholarDigital Library
11. Peers, P., and Dutré, P. 2003. Wavelet Environment Matting. In Eurographics Symposium on Rendering. 157–166. Google ScholarDigital Library
12. Rayleigh, J. W. S. B. 1900. On the Law of Reciprocity in Diffuse Reflexion. Philosophical Magazine 49, 324–325.Google Scholar
13. Robertson, M. A., S., and Stevenson, R. L. 1999. Dynamic Range Improvement through Multiple Exposures. In Proceedings of the IEEE Intl. Conference on Image Processing (ICIP’99), 159–163.Google Scholar
14. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D Model Acquisition. In SIGGRAPH ’02, 438–446. Google ScholarDigital Library
15. Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2003. A Theory of Multiplexed Illumination. In Proceedings of the IEEE Intl. Conference on Computer Vision (ICCV ’03), 808–815. Google ScholarDigital Library
16. Tu, P., Mendonca, P., Ross, J., and Miller, J. 2003. Surface Registration with a Helmholtz Reciprocity Image Pair. In IEEE Workshop on Color and Photometric Methods in Computer Vision.Google Scholar
17. Veach, E. 1997. Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University. Google ScholarDigital Library
18. Von Helmholtz, H. 1856. Treatise on Physiological Optics (1925). The Optical Society of America. Electronic edition (2001): University of Pennsylvania http://psych.upenn.edu/backuslab/helmholtz.Google Scholar
19. Whitted, T. 1980. An Improved Illumination Model for Shaded Display. Commun. ACM 23, 6, 343–349. Google ScholarDigital Library
20. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Levoy, M., and Horowitz, M. 2005. High Performance Imaging using Large Camera Arrays. In SIGGRAPH ’05. Google ScholarDigital Library
21. Zickler, T., Belhumeur, P., and Kriegman, D. 2002. Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction. Intl. Journal of Computer Vision 49, 2–3, 215–227. Google ScholarDigital Library
22. Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment Matting and Compositing. In SIGGRAPH ’99, 205–214. Google ScholarDigital Library
23. Zotkin, D., Duraiswami, R., Grassi, E., and Gumerov, N. 2004. Fast Head Related Transfer Function Measurement via Reciprocity. Technical Report No. CS-4620, University of Maryland.Google Scholar