“Light field microscopy” by Levoy, Ng, Adams, Footer and Horowitz

  • ©Marc Levoy, Ren Ng, Andrew Adams, Matthew Footer, and Mark Horowitz




    Light field microscopy



    By inserting a microlens array into the optical train of a conventional microscope, one can capture light fields of biological specimens in a single photograph. Although diffraction places a limit on the product of spatial and angular resolution in these light fields, we can nevertheless produce useful perspective views and focal stacks from them. Since microscopes are inherently orthographic devices, perspective views represent a new way to look at microscopic specimens. The ability to create focal stacks from a single photograph allows moving or light-sensitive specimens to be recorded. Applying 3D deconvolution to these focal stacks, we can produce a set of cross sections, which can be visualized using volume rendering. In this paper, we demonstrate a prototype light field microscope (LFM), analyze its optical performance, and show perspective views, focal stacks, and reconstructed volumes for a variety of biological specimens. We also show that synthetic focusing followed by 3D deconvolution is equivalent to applying limited-angle tomography directly to the 4D light field.


    1. Adelson, T., Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 2, 99–106. Google ScholarDigital Library
    2. Agard, D. A. 1984. Optical sectioning microscopy: Cellular architecture in three dimensions. Ann. Rev. Biophys. Bioeng 13, 191–219.Google ScholarCross Ref
    3. Andersen, A. H., Kak, A. C., 1984. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrasonic Imaging 6, 81–94.Google ScholarCross Ref
    4. Arridge, S. R. 2001. Methods for the inverse problem in optical tomography. Proc. Waves and Imaging Through Complex Media. Kluwer, 307–329.Google ScholarCross Ref
    5. Castleman, K. R. 1979. Digital Image Processing. Prentice Hall. Google ScholarDigital Library
    6. Chamgoulov, R. O., Lane, P. M., Macaulay, C. E. 2004. Optical computed-tomography microscope using digital spatial light modulation. Proc. SPIE 5324, 182–190.Google Scholar
    7. Colsher, J. G. 1980. Fully three-dimensional positron emission tomography. Phys. Med. Biol. 25, 1, 103–115.Google ScholarCross Ref
    8. Corle, T. R., Kino, G. S. 1996. Confocal Scanning Optical Microscopy and Related Imaging Systems. Academic Press.Google Scholar
    9. Ellis, G. W. 1966. Holomicrography: transformation of image during reconstruction a posteriori. Science 143, 1195–1196.Google ScholarCross Ref
    10. Goldberg, N. 1992. Camera technology: the dark side of the lens. Academic Press.Google Scholar
    11. Goodman, J. 1996. Introduction to Fourier optics. 2nd edition, McGraw-Hill.Google Scholar
    12. Gustafsson, M. G. L. 2005. Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proc. National Academy of Sciences 102, 37.Google ScholarCross Ref
    13. Holmes, T. J., Bhattacharyya, S., et al. 1995. Light microscopic images reconstructed by maximum likelihood deconvolution. In Handbook of Biological Confocal Microscopy, ed. J. B. Pawley, Plenum Press, 389–402.Google Scholar
    14. Inoue, S., Oldenbourg, R. 1995. Microscopes. In Handbook of Optics, 2nd edition, McGraw-Hill.Google Scholar
    15. Inoue, S. and Spring, K. R. 1997. Video Microscopy. 2nd edition, Plenum Press.Google Scholar
    16. Gabor, D. 1948. A new microscopic principle. Nature 161, 777–778.Google ScholarCross Ref
    17. Isaksen, A., Mcmillan, L., Gortler, S. J. 2000. Dynamically reparameterized light fields. Proc. SIGGRAPH 2000. Google ScholarDigital Library
    18. Javidi, B., Okano, F., eds. 2002. Three-Dimensional Television, Video and Display Technologies. Springer-Verlag. Google ScholarDigital Library
    19. Kak, A. C., Slaney, M. 1988. Principles of Computerized Tomographic Imaging. IEEE Press.Google Scholar
    20. Kawata, S., Nakamura, 0., Minami, S. 1987. Optical microscope tomography. I. Support constraint. J. Opt. Soc. Am. A 4, 1, 292–297.Google ScholarCross Ref
    21. Kingslake, R. 1983. Optical system design. Academic Press.Google Scholar
    22. Levoy, M., Hanrahan, P. 1996. Light field rendering. Proc. SIGGRAPH 1996. Google ScholarDigital Library
    23. Levoy, M., Chen, B., Vaish, V., Horowitz, M., Mcdowall, I., Bolas, M. 2004. Synthetic aperture confocal imaging. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3, 825–834. Google ScholarDigital Library
    24. Markham, J., Conchello, J.-A. 2001. Artefacts in restored images due to intensity loss in three-dimensional fluorescence microscopy. J. Microscopy 204, 2, 93–98.Google ScholarCross Ref
    25. Mcnally, J. G., Preza, C., Conchello, J. A., Thomas, L. J. Jr. 1994. Artifacts in computational optical-sectioning microscopy. J. Opt. Soc. Am. A 11, 3, 1056–67.Google ScholarCross Ref
    26. Nayar, S. K., Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. Proc. International Conference on Robotics and Automation (ICRA), Vol. 2, 218–225.Google ScholarCross Ref
    27. Nayar, S. K., Narasimhan, S. G. 2002. Assorted pixels: Multi-sampled imaging with structural models. Proc. ECCV. Google ScholarDigital Library
    28. Ng, R. 2005. Fourier slice photography. ACM Transactions on Graphics (Proc. SIGGRAPH) 24, 3, 735–744. Google ScholarDigital Library
    29. Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Stanford Tech Report CTSR 2005-02.Google Scholar
    30. Ng, R. 2006. Digital Light Field Photography. PhD dissertation, Stanford University. Google ScholarDigital Library
    31. Noguchi, M., Nayar, S. 1994. Microscopic shape from focus using active illumination. Proc. IAPR International Conference on Pattern Recognition (ICPR), Vol. A, 147–152.Google ScholarCross Ref
    32. Okoshi, T. 1976. Three-Dimensional Imaging Techniques. Academic Press.Google Scholar
    33. Piller, H. 1977. Microscope Photometry. Springer-Verlag.Google Scholar
    34. Pluta, M. 1988. Advanced Light Microscopy (in 3 volumes). Elsevier.Google Scholar
    35. Schechner, Y., Kiryati, N. 1999. The optimal axial interval for estimating depth from defocus. Proc. ICCV. Google ScholarDigital Library
    36. Schechner, Y., Kiryati, N. 2000. Depth from defocus vs. stereo: How different really are they? IJCV 39, 2, 141–162. Google ScholarDigital Library
    37. Schechner, Y. Y., Kiryati, N., Basri, R. 2000. Separation of transparent layers using focus. IJCV 39, 1, 25–39. Google ScholarDigital Library
    38. Schechner, Y., Nayar, S. 2001. Generalized Mosaicing. Proc. ICCV.Google Scholar
    39. Shah, U. B., Nayar, S. K. 1992. Extracting 3-D structure and focused images using an optical microscope. Proc. IEEE Symposium on Computer-Based Medical Systems.Google ScholarCross Ref
    40. Streibl, N. 1984. Depth transfer by an imaging system. Optica Acta 31, 11, 1233–1241.Google ScholarCross Ref
    41. Streibl, N. 1985. Three-dimensional imaging by a microscope. J. Opt. Soc. Am. A 2, 2, 121–127.Google ScholarCross Ref
    42. Swedlow, J. R., Sedat, J. W., Agard, D. A. 1997. Deconvolution in optical microscopy. In Deconvolution of Images and Spectra, ed. P. A. Jansson, Academic Press, 284–309. Google ScholarDigital Library
    43. Vaish, V., Garg, G., Talvala, E., Antunez, E., Wilburn, B., Horowitz, M., Levoy, M., Synthetic aperture focusing using a shear-warp factorization of the viewing transform. Proc. Workshop on Advanced 3D Imaging for Safety and Security, in conjunction with CVPR 2005. Google ScholarDigital Library
    44. Weinstein, R. S., Descour, M. R., et al. 2004. An array microscope for ultrarapid virtual slide processing and telepathology. Design, fabrication, and validation study. Human Pathology 35, 11, 1303–1314.Google ScholarCross Ref

ACM Digital Library Publication: