“Dual photography” by Sen, Chen, Garg, Marschner, Horowitz, et al. …

  • ©Pradeep Sen, Billy Chen, Gaurav Garg, Stephen Marschner, Mark Horowitz, Marc Levoy, and Hendrik P. A. Lensch


Abstract:


    We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, inter-reflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.

References:


    1. Debevec, P., and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In SIGGRAPH ’97, 369–378. Google ScholarDigital Library
    2. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the Reflectance Field of a Human Face. In SIGGRAPH ’00, 145–156. Google ScholarDigital Library
    3. Goesele, M., Lensch, H. P. A., Lang, J., Fuchs, C., and Seidel, H.-P. 2004. DISCO: Acquisition of Translucent Objects. In SIGGRAPH ’04, 835–844. Google ScholarDigital Library
    4. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. In SIGGRAPH ’96, 43–54. Google ScholarDigital Library
    5. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In SIGGRAPH ’96, 31–42. Google ScholarDigital Library
    6. Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M. 2004. Synthetic Aperture Confocal Imaging. In SIGGRAPH ’04, 825–834. Google ScholarDigital Library
    7. Masselus, V., Peers, P., Dutré, P., and Willems, Y. D. 2003. Relighting with 4D Incident Light Fields. In SIGGRAPH ’03, 613–620. Google ScholarDigital Library
    8. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., and McMillan, L. 2002. Image-Based 3D Photography Using Opacity Hulls. In SIGGRAPH ’02, 427–437. Google ScholarDigital Library
    9. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. 2002. Acquisition and Rendering of Transparent and Refractive Objects. In Eurographics Symposium on Rendering, 277–288. Google ScholarDigital Library
    10. Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-Refined Reflectance Functions for Natural Illumination. In Eurographics Symposium on Rendering, 299–308. Google ScholarDigital Library
    11. Peers, P., and Dutré, P. 2003. Wavelet Environment Matting. In Eurographics Symposium on Rendering. 157–166. Google ScholarDigital Library
    12. Rayleigh, J. W. S. B. 1900. On the Law of Reciprocity in Diffuse Reflexion. Philosophical Magazine 49, 324–325.Google Scholar
    13. Robertson, M. A., S., and Stevenson, R. L. 1999. Dynamic Range Improvement through Multiple Exposures. In Proceedings of the IEEE Intl. Conference on Image Processing (ICIP’99), 159–163.Google Scholar
    14. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D Model Acquisition. In SIGGRAPH ’02, 438–446. Google ScholarDigital Library
    15. Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2003. A Theory of Multiplexed Illumination. In Proceedings of the IEEE Intl. Conference on Computer Vision (ICCV ’03), 808–815. Google ScholarDigital Library
    16. Tu, P., Mendonca, P., Ross, J., and Miller, J. 2003. Surface Registration with a Helmholtz Reciprocity Image Pair. In IEEE Workshop on Color and Photometric Methods in Computer Vision.Google Scholar
    17. Veach, E. 1997. Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford University. Google ScholarDigital Library
    18. Von Helmholtz, H. 1856. Treatise on Physiological Optics (1925). The Optical Society of America. Electronic edition (2001): University of Pennsylvania http://psych.upenn.edu/backuslab/helmholtz.Google Scholar
    19. Whitted, T. 1980. An Improved Illumination Model for Shaded Display. Commun. ACM 23, 6, 343–349. Google ScholarDigital Library
    20. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Levoy, M., and Horowitz, M. 2005. High Performance Imaging using Large Camera Arrays. In SIGGRAPH ’05. Google ScholarDigital Library
    21. Zickler, T., Belhumeur, P., and Kriegman, D. 2002. Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction. Intl. Journal of Computer Vision 49, 2–3, 215–227. Google ScholarDigital Library
    22. Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment Matting and Compositing. In SIGGRAPH ’99, 205–214. Google ScholarDigital Library
    23. Zotkin, D., Duraiswami, R., Grassi, E., and Gumerov, N. 2004. Fast Head Related Transfer Function Measurement via Reciprocity. Technical Report No. CS-4620, University of Maryland.Google Scholar


ACM Digital Library Publication: