“Improving light field camera sample design with irregularity and aberration” by Wei, Liang, Myhre, Pitts and Akeley

Conventional camera designs usually shun sample irregularities and lens aberrations. We demonstrate that such irregularities and aberrations, when properly applied, can improve the quality and usability of light field cameras. Examples include spherical aberrations for the mainlens, and misaligned sampling patterns for the microlens and photosensor elements. These observations are a natural consequence of a key difference between conventional and light field cameras: optimizing for a single captured 2D image versus a range of reprojected 2D images from a captured 4D light field. We propose designs in mainlens aberrations and microlens/photosensor sample patterns, and evaluate them through simulated measurements and captured results with our hardware prototype.

1. Ben-Ezra, M., Lin, Z., and Wilburn, B. 2007. Penrose pixels super-resolution in the detector layout domain. In Proc. ICCV, 1–8.Google Scholar
2. Bishop, T. E., and Favaro, P. 2012. The light field camera: Extended depth of field, aliasing, and superresolution. IEEE Trans. PAMI 34, 5, 972–986. Google ScholarDigital Library
3. Bok, Y., Jeon, H.-G., and Kweon, I. S. 2014. Geometric calibration of micro-lens-based light-field cameras using line features. In Proc. ECCV. 47–61.Google Scholar
4. Broxton, M., Grosenick, L., Yang, S., Cohen, N., Andalman, A., Deisseroth, K., and Levoy, M. 2013. Wave optics theory and 3-D deconvolution for the light field microscope. Opt. Express 21, 21, 25418–25439.Google ScholarCross Ref
5. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In SIGGRAPH ’00, 307–318. Google ScholarDigital Library
6. Cho, D., Lee, M., Kim, S., and Tai, Y.-W. 2013. Modeling the calibration pipeline of the Lytro camera for high quality light-field image reconstruction. In Proc. ICCV, 3280–3287. Google ScholarDigital Library
7. Cohen, N., Yang, S., Andalman, A., Broxton, M., Grosenick, L., Deisseroth, K., Horowitz, M., and Levoy, M. 2014. Enhancing the performance of the light field microscope using wavefront coding. Opt. Express 22, 20, 24817–24839.Google ScholarCross Ref
8. Cook, R. L. 1986. Stochastic sampling in computer graphics. ACM Trans. Graph. 5, 1, 51–72. Google ScholarDigital Library
9. Dansereau, D. G., Pizarro, O., and Williams, S. B. 2013. Decoding, calibration and rectification for lenselet-based plenoptic cameras. In Proc. CVPR, 1027–1034. Google ScholarDigital Library
10. Dowski, Edward R, J., and Johnson, G. E. 1999. Wavefront coding: a modern method of achieving high-performance and/or low-cost imaging systems. In Proc. SPIE 3779, 137–145.Google Scholar
11. Ebeida, M. S., Awad, M. A., Ge, X., Mahmoud, A. H., Mitchell, S. A., Knupp, P. M., and Wei, L.-Y. 2014. Improving spatial coverage while preserving blue noise of point sets. Computer-Aided Design 46, 0, 25–36. Google ScholarDigital Library
12. Fiss, J., Curless, B., and Szeliski, R. 2014. Refocusing plenoptic images using depth-adaptive splatting. In Proc. ICCP, 1–9.Google Scholar
13. Georgiev, T., Chunev, G., and Lumsdaine, A. 2011. Superresolution with the focused plenoptic camera. In Proc. SPIE, vol. 7873, 1105–1117.Google Scholar
14. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In SIGGRAPH ’96, 43–54. Google ScholarDigital Library
15. Heck, D., Schlömer, T., and Deussen, O. 2013. Blue noise sampling with controlled aliasing. ACM Trans. Graph. 32, 3, 25:1–25:12. Google ScholarDigital Library
16. Heide, F., Rouf, M., Hullin, M. B., Labitzke, B., Heidrich, W., and Kolb, A. 2013. High-quality computational imaging through simple lenses. ACM Trans. Graph. 32, 5, 149:1–149:14. Google ScholarDigital Library
17. Heide, F., Lanman, D., Reddy, D., Kautz, J., Pulli, K., and Luebke, D. 2014. Cascaded displays: Spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. 33, 4, 60:1–60:11. Google ScholarDigital Library
18. Hersch, R. D., and Chosson, S. 2004. Band moiré images. ACM Trans. Graph. 23, 3, 239–247. Google ScholarDigital Library
19. Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field display. ACM Trans. Graph. 33, 4, 59:1–59:12. Google ScholarDigital Library
20. Imatest, 2014. Sharpness: What is it and how is it measured? http://www.imatest.com/docs/sharpness/.Google Scholar
21. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In SIGGRAPH ’00, 297–306. Google ScholarDigital Library
22. Johnson, R. 1992. A historical perspective on understanding optical aberrations. In Lens design, W. J. Smith, Ed.Google Scholar
23. Keller, A., Premoze, S., and Raab, M. 2012. Advanced (quasi) Monte Carlo methods for image synthesis. In SIGGRAPH 2012 Courses. Google ScholarDigital Library
24. Kingslake, R., and Johnson, R. B. 2009. Lens Design Fundamentals. Academic Press.Google Scholar
25. Kitamura, Y., Shogenji, R., Yamada, K., Miyatake, S., Miyamoto, M., Morimoto, T., Masaki, Y., Kondou, N., Miyazaki, D., Tanida, J., et al. 2004. Reconstruction of a high-resolution image on a compound-eye image-capturing system. Applied Optics 43, 8, 1719–1727.Google ScholarCross Ref
26. Lagae, A., and Dutré, P. 2008. A comparison of methods for generating Poisson disk distributions. CGF 27, 1, 114–129.Google ScholarCross Ref
27. Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal light field reconstruction for rendering distribution effects. In ACM Trans. Graph., vol. 30, 55:1–55:12. Google ScholarDigital Library
28. Lessig, C., Desbrun, M., and Fiume, E. 2014. A constructive theory of sampling for image synthesis using reproducing kernel bases. ACM Trans. Graph. 33, 4, 55:1–55:14. Google ScholarDigital Library
29. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In SIGGRAPH ’96, 31–42. Google ScholarDigital Library
30. Liang, C.-K., and Ramamoorthi, R. 2015. A light transport framework for lenslet light field cameras. ACM Trans. Graph. 34, 2, 16:1–16:19. Google ScholarDigital Library
31. Lytro INC., 2011. First generation Lytro camera. https://store.lytro.com/collections/the-first-generation-product-list.Google Scholar
32. McGuire, M., 2011. Computer graphics archive. http://graphics.cs.williams.edu/data/.Google Scholar
33. Ng, R. 2005. Fourier slice photography. In ACM Trans. Graph., vol. 24, 735–744. Google ScholarDigital Library
34. Ng, R. 2006. Digital light field photography. PhD thesis, Stanford University. Google ScholarDigital Library
35. Oberdoerster, A., Favaro, P., and Lensch, H. 2014. Anamorphic pixels for multi-channel superresolution. In Proc. ICCP, 1–10.Google Scholar
36. öztireli, A. C., Alexa, M., and Gross, M. 2010. Spectral sampling of manifolds. ACM Trans. Graph. 29, 6, 168:1–168:8. Google ScholarDigital Library
37. Perwass, C., and Wietzke, L. 2012. Single lens 3D-camera with extended depth-of-field. In Proc. SPIE, vol. 8291, 829108.Google Scholar
38. Pharr, M., and Humphreys, G. 2004. Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarDigital Library
39. Sasao, T., Hiura, S., and Sato, K. 2013. Super-resolution with randomly shaped pixels and sparse regularization. In Proc. ICCP, 1–11.Google Scholar
40. Sasián, J. 2012. Introduction to Aberrations in Optical Imaging Systems. Cambridge University Press.Google Scholar
41. Schechter, H., and Bridson, R. 2012. Ghost SPH for animating water. ACM Trans. Graph. 31, 4, 61:1–61:8. Google ScholarDigital Library
42. Schlömer, T., and Deussen, O. 2011. Accurate spectral analysis of two-dimensional point sets. J. Graphics, GPU, and Game Tools 15, 3, 152–160.Google Scholar
43. Schoberl, M., Seller, J., Foessel, S., and Kaup, A. 2011. Increasing imaging resolution by covering your sensor. In Proc. IEEE ICIP, 1897–1900.Google Scholar
44. Schuler, C. J., Hirsch, M., Harmeling, S., and Scholkopf, B. 2011. Non-stationary correction of optical aberrations. In Proc. ICCV, 659–666. Google ScholarDigital Library
45. Shih, Y., Guenter, B., and Joshi, N. 2012. Image enhancement using calibrated lens simulations. In Proc. ECCV. 42–56. Google ScholarDigital Library
46. Shirley, P. 1991. Discrepancy as a quality measure for sample distributions. In Eurographics ’91, 183–194.Google Scholar
47. Venkataraman, K., Lelescu, D., Duparré, J., McMahon, A., Molina, G., Chatterjee, P., Mullis, R., and Nayar, S. 2013. Picam: an ultra-thin high performance monolithic camera array. ACM Trans. Graph. 32, 6, 166:1–166:13. Google ScholarDigital Library
48. Wei, L.-Y., and Wang, R. 2011. Differential domain analysis for non-uniform sampling. ACM Trans. Graph. 30, 4, 50:1–50:10. Google ScholarDigital Library
49. Wei, L.-Y. 2010. Multi-class blue noise sampling. ACM Trans. Graph. 29, 4, 79:1–79:8. Google ScholarDigital Library
50. Yu, Z., Yu, J., Lumsdaine, A., and Georgiev, T. 2012. An analysis of color demosaicing in plenoptic cameras. In Proc. CVPR, 901–908. Google ScholarDigital Library
51. 2014. Zemax. http://www.zemax.com/.Google Scholar