“Synthetic aperture confocal imaging” by Levoy, Chen, Vaish, Horowitz, Bolas, et al. …

Confocal microscopy is a family of imaging techniques that employ focused patterned illumination and synchronized imaging to create cross-sectional views of 3D biological specimens. In this paper, we adapt confocal imaging to large-scale scenes by replacing the optical apertures used in microscopy with arrays of real or virtual video projectors and cameras. Our prototype implementation uses a video projector, a camera, and an array of mirrors. Using this implementation, we explore confocal imaging of partially occluded environments, such as foliage, and weakly scattering environments, such as murky water. We demonstrate the ability to selectively image any plane in a partially occluded environment, and to see further through murky water than is otherwise possible. By thresholding the confocal images, we extract mattes that can be used to selectively illuminate any plane in the scene.

1. CHAI, J.-X., TONG, X., CHAN, S.-C. SHUM, H.-Y. “Plenoptic Sampling.” Proc. SIGGRAPH 2000. Google ScholarDigital Library
2. CHAM, T.-J., REHG, J. M., SUKTHANKAR, R., SUKTHANKAR, G., “Shadow Elimination and Occluder Light Suppression for Multi-Projector Displays,” Proc. PROCAMS 2003.Google Scholar
3. CHUANG, Y.-Y., CURLESS, B., SALESIN, D. H., SZELISKI, R., “A Bayesian Approach to Digital Matting,” Proc. CVPR 2001.Google Scholar
4. COORG, S., TELLER, S., “Extracting Textured Vertical Facades from Controlled Close-Range Imagery,” Proc. CVPR 1999.Google Scholar
5. CORLE, T. R., KINO, G. S., Confocal Scanning Optical Microscopy and Related Imaging Systems, Academic Press, 1996.Google Scholar
6. DEBEVEC, P., “Image-Based Lighting.” IEEE Computer Graphics and Applications, Vol. 22, No. 2, March/April, 2002. Google ScholarDigital Library
7. HAN, J. Y., PERLIN, K., “Measuring Bidirectional Texture Reflectance With a Kaleidoscope,” Proc. SIGGRAPH 2003. Google ScholarDigital Library
8. HARWIT, M., SLOANE, N. J. A., “Hadamard Transform Optics,” Academic Press, 1979.Google Scholar
9. ISAKSEN, A., MCMILLAN, L., GORTLER, S. J., “Dynamically Reparameterized Light Fields,” Proc. SIGGRAPH 2000. Google ScholarDigital Library
10. JAFFE, J. S., “Computer Modeling and the Design of Optimal Underwater Imaging Systems,” J. Oceanic Eng., Vol. 15, No. 2, 1990.Google ScholarCross Ref
11. JAFFE, J. S., MCLEAN, J., STRAND, M. P., MOORE, K. D., “Underwater Optical Imaging: Status and Prospects,” Oceanography. Vol. 14, No. 3, 2001, pp. 66–76.Google ScholarCross Ref
12. LEVOY, M., HANRAHAN, P., “Light Field Rendering,” Proc. SIGGRAPH 1996. Google ScholarDigital Library
13. MALZBENDER, T., GELB, D., WOLTERS, H., “Polynomial Texture Maps,” Proc. SIGGRAPH 2001. Google ScholarDigital Library
14. MASSELUS, V., PEERS, P., DUTRE, P., WILLEMS, Y. D., “Relighting with 4D incident light fields,” Proc. SIGGRAPH 2003. Google ScholarDigital Library
15. MIDDLETON, W., Vision Through the Atmosphere, University of Toronto Press, 1952.Google Scholar
16. MOBLEY, C., Light and Water: Radiative Transfer in Natural Waters, Academic Press, 1994.Google Scholar
17. OKASHI, T., Three-Dimensional Imaging Techniques, Academic Press, 1976.Google Scholar
18. RUSINKIEWICZ, S., HALL-HOLT, O., LEVOY, M., “Real-Time 3D Model Acquisition,” Proc. SIGGRAPH 2002. Google ScholarDigital Library
19. SCHECHNER, Y. Y., KARPEL, N., “Clear Underwater Vision,” Proc. CVPR 2004.Google Scholar
20. SCHECHNER, Y., NAYAR., S., BELHUMEUR, P., “A Theory of Multiplexed Illumination.” Proc. ICCV 2003. Google ScholarDigital Library
21. SMITH, A., BLINN, J., “Blue Screen Matting,” Proc. SIGGRAPH 1996. Google ScholarDigital Library
22. VAISH, V., WILBURN, B., JOSHI, N., LEVOY, M., “Using Plane + Parallax for Calibrating Dense Camera Arrays.” Proc. CVPR 2004.Google Scholar
23. WILBURN, B., SMULSKI, M., LEE, K., HOROWITZ, M. A., “The Light Field Video Camera,” Proc. SPIE Electronic Imaging 2002.Google Scholar
24. WILSON, T., JUSKAITIS, R., NEIL, M., KOZUBEK, M., “Confocal Microscopy by Aperture Correlation,” Optics Letters, 21(3), 1996.Google ScholarCross Ref
25. ZAND, J., “Coded aperture imaging in high energy astronomy,” NASA Laboratory for High Energy Astrophysics (LHEA) at NASA’s GSFC, 1996. URL: http://theawww.gsfc.nasa.gov/docs/cai/.Google Scholar