“Improving light field camera sample design with irregularity and aberration” by Wei, Liang, Myhre, Pitts and Akeley

  • ©Li-Yi Wei, Chia-Kai Liang, Graham Myhre, Colvin Pitts, and Kurt Akeley




    Improving light field camera sample design with irregularity and aberration



    Conventional camera designs usually shun sample irregularities and lens aberrations. We demonstrate that such irregularities and aberrations, when properly applied, can improve the quality and usability of light field cameras. Examples include spherical aberrations for the mainlens, and misaligned sampling patterns for the microlens and photosensor elements. These observations are a natural consequence of a key difference between conventional and light field cameras: optimizing for a single captured 2D image versus a range of reprojected 2D images from a captured 4D light field. We propose designs in mainlens aberrations and microlens/photosensor sample patterns, and evaluate them through simulated measurements and captured results with our hardware prototype.


    1. Ben-Ezra, M., Lin, Z., and Wilburn, B. 2007. Penrose pixels super-resolution in the detector layout domain. In Proc. ICCV, 1–8.Google Scholar
    2. Bishop, T. E., and Favaro, P. 2012. The light field camera: Extended depth of field, aliasing, and superresolution. IEEE Trans. PAMI 34, 5, 972–986. Google ScholarDigital Library
    3. Bok, Y., Jeon, H.-G., and Kweon, I. S. 2014. Geometric calibration of micro-lens-based light-field cameras using line features. In Proc. ECCV. 47–61.Google Scholar
    4. Broxton, M., Grosenick, L., Yang, S., Cohen, N., Andalman, A., Deisseroth, K., and Levoy, M. 2013. Wave optics theory and 3-D deconvolution for the light field microscope. Opt. Express 21, 21, 25418–25439.Google ScholarCross Ref
    5. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In SIGGRAPH ’00, 307–318. Google ScholarDigital Library
    6. Cho, D., Lee, M., Kim, S., and Tai, Y.-W. 2013. Modeling the calibration pipeline of the Lytro camera for high quality light-field image reconstruction. In Proc. ICCV, 3280–3287. Google ScholarDigital Library
    7. Cohen, N., Yang, S., Andalman, A., Broxton, M., Grosenick, L., Deisseroth, K., Horowitz, M., and Levoy, M. 2014. Enhancing the performance of the light field microscope using wavefront coding. Opt. Express 22, 20, 24817–24839.Google ScholarCross Ref
    8. Cook, R. L. 1986. Stochastic sampling in computer graphics. ACM Trans. Graph. 5, 1, 51–72. Google ScholarDigital Library
    9. Dansereau, D. G., Pizarro, O., and Williams, S. B. 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In Proc. CVPR, 1027–1034. Google ScholarDigital Library
    10. Dowski, Edward R, J., and Johnson, G. E. 1999. Wavefront coding: a modern method of achieving high-performance and/or low-cost imaging systems. In Proc. SPIE 3779, 137–145.Google Scholar
    11. Ebeida, M. S., Awad, M. A., Ge, X., Mahmoud, A. H., Mitchell, S. A., Knupp, P. M., and Wei, L.-Y. 2014. Improving spatial coverage while preserving blue noise of point sets. Computer-Aided Design 46, 0, 25–36. Google ScholarDigital Library
    12. Fiss, J., Curless, B., and Szeliski, R. 2014. Refocusing plenoptic images using depth-adaptive splatting. In Proc. ICCP, 1–9.Google Scholar
    13. Georgiev, T., Chunev, G., and Lumsdaine, A. 2011. Superresolution with the focused plenoptic camera. In Proc. SPIE, vol. 7873, 1105–1117.Google Scholar
    14. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In SIGGRAPH ’96, 43–54. Google ScholarDigital Library
    15. Heck, D., Schlömer, T., and Deussen, O. 2013. Blue noise sampling with controlled aliasing. ACM Trans. Graph. 32, 3, 25:1–25:12. Google ScholarDigital Library
    16. Heide, F., Rouf, M., Hullin, M. B., Labitzke, B., Heidrich, W., and Kolb, A. 2013. High-quality computational imaging through simple lenses. ACM Trans. Graph. 32, 5, 149:1–149:14. Google ScholarDigital Library
    17. Heide, F., Lanman, D., Reddy, D., Kautz, J., Pulli, K., and Luebke, D. 2014. Cascaded displays: Spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. 33, 4, 60:1–60:11. Google ScholarDigital Library
    18. Hersch, R. D., and Chosson, S. 2004. Band moiré images. ACM Trans. Graph. 23, 3, 239–247. Google ScholarDigital Library
    19. Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field display. ACM Trans. Graph. 33, 4, 59:1–59:12. Google ScholarDigital Library
    20. Imatest, 2014. Sharpness: What is it and how is it measured? http://www.imatest.com/docs/sharpness/.Google Scholar
    21. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In SIGGRAPH ’00, 297–306. Google ScholarDigital Library
    22. Johnson, R. 1992. A historical perspective on understanding optical aberrations. In Lens design, W. J. Smith, Ed.Google Scholar
    23. Keller, A., Premoze, S., and Raab, M. 2012. Advanced (quasi) Monte Carlo methods for image synthesis. In SIGGRAPH 2012 Courses. Google ScholarDigital Library
    24. Kingslake, R., and Johnson, R. B. 2009. Lens Design Fundamentals. Academic Press.Google Scholar
    25. Kitamura, Y., Shogenji, R., Yamada, K., Miyatake, S., Miyamoto, M., Morimoto, T., Masaki, Y., Kondou, N., Miyazaki, D., Tanida, J., et al. 2004. Reconstruction of a high-resolution image on a compound-eye image-capturing system. Applied Optics 43, 8, 1719–1727.Google ScholarCross Ref
    26. Lagae, A., and Dutré, P. 2008. A comparison of methods for generating Poisson disk distributions. CGF 27, 1, 114–129.Google ScholarCross Ref
    27. Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal light field reconstruction for rendering distribution effects. In ACM Trans. Graph., vol. 30, 55:1–55:12. Google ScholarDigital Library
    28. Lessig, C., Desbrun, M., and Fiume, E. 2014. A constructive theory of sampling for image synthesis using reproducing kernel bases. ACM Trans. Graph. 33, 4, 55:1–55:14. Google ScholarDigital Library
    29. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH ’96, 31–42. Google ScholarDigital Library
    30. Liang, C.-K., and Ramamoorthi, R. 2015. A light transport framework for lenslet light field cameras. ACM Trans. Graph. 34, 2, 16:1–16:19. Google ScholarDigital Library
    31. Lytro INC., 2011. First generation Lytro camera. https://store.lytro.com/collections/the-first-generation-product-list.Google Scholar
    32. McGuire, M., 2011. Computer graphics archive. http://graphics.cs.williams.edu/data/.Google Scholar
    33. Ng, R. 2005. Fourier slice photography. In ACM Trans. Graph., vol. 24, 735–744. Google ScholarDigital Library
    34. Ng, R. 2006. Digital light field photography. PhD thesis, Stanford University. Google ScholarDigital Library
    35. Oberdoerster, A., Favaro, P., and Lensch, H. 2014. Anamorphic pixels for multi-channel superresolution. In Proc. ICCP, 1–10.Google Scholar
    36. öztireli, A. C., Alexa, M., and Gross, M. 2010. Spectral sampling of manifolds. ACM Trans. Graph. 29, 6, 168:1–168:8. Google ScholarDigital Library
    37. Perwass, C., and Wietzke, L. 2012. Single lens 3D-camera with extended depth-of-field. In Proc. SPIE, vol. 8291, 829108.Google Scholar
    38. Pharr, M., and Humphreys, G. 2004. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarDigital Library
    39. Sasao, T., Hiura, S., and Sato, K. 2013. Super-resolution with randomly shaped pixels and sparse regularization. In Proc. ICCP, 1–11.Google Scholar
    40. Sasián, J. 2012. Introduction to Aberrations in Optical Imaging Systems. Cambridge University Press.Google Scholar
    41. Schechter, H., and Bridson, R. 2012. Ghost SPH for animating water. ACM Trans. Graph. 31, 4, 61:1–61:8. Google ScholarDigital Library
    42. Schlömer, T., and Deussen, O. 2011. Accurate spectral analysis of two-dimensional point sets. J. Graphics, GPU, and Game Tools 15, 3, 152–160.Google Scholar
    43. Schoberl, M., Seller, J., Foessel, S., and Kaup, A. 2011. Increasing imaging resolution by covering your sensor. In Proc. IEEE ICIP, 1897–1900.Google Scholar
    44. Schuler, C. J., Hirsch, M., Harmeling, S., and Scholkopf, B. 2011. Non-stationary correction of optical aberrations. In Proc. ICCV, 659–666. Google ScholarDigital Library
    45. Shih, Y., Guenter, B., and Joshi, N. 2012. Image enhancement using calibrated lens simulations. In Proc. ECCV. 42–56. Google ScholarDigital Library
    46. Shirley, P. 1991. Discrepancy as a quality measure for sample distributions. In Eurographics ’91, 183–194.Google Scholar
    47. Venkataraman, K., Lelescu, D., Duparré, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., and Nayar, S. 2013. Picam: an ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32, 6, 166:1–166:13. Google ScholarDigital Library
    48. Wei, L.-Y., and Wang, R. 2011. Differential domain analysis for non-uniform sampling. ACM Trans. Graph. 30, 4, 50:1–50:10. Google ScholarDigital Library
    49. Wei, L.-Y. 2010. Multi-class blue noise sampling. ACM Trans. Graph. 29, 4, 79:1–79:8. Google ScholarDigital Library
    50. Yu, Z., Yu, J., Lumsdaine, A., and Georgiev, T. 2012. An analysis of color demosaicing in plenoptic cameras. In Proc. CVPR, 901–908. Google ScholarDigital Library
    51. 2014. Zemax. http://www.zemax.com/.Google Scholar

ACM Digital Library Publication: