“Compressive light field photography using overcomplete dictionaries and optimized projections” by Marwah, Wetzstein, Bando and Raskar

  • ©Kshitij Marwah, Gordon Wetzstein, Yosuke Bando, and Ramesh Raskar

Conference:


Type:


Title:

    Compressive light field photography using overcomplete dictionaries and optimized projections

Presenter(s)/Author(s):


Session Title: Computational Light Capture

Moderator(s):


Abstract:


    Light field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into a single 2D sensor image or require multiple photographs to be taken for acquiring a high-resolution light field. We propose a compressive light field camera architecture that allows for higher-resolution light fields to be recovered than previously possible from a single image. The proposed architecture comprises three key components: light field atoms as a sparse representation of natural light fields, an optical design that allows for capturing optimized 2D light field projections, and robust sparse reconstruction methods to recover a 4D light field from a single coded 2D projection. In addition, we demonstrate a variety of other applications for light field atoms and sparse coding, including 4D light field compression and denoising.

References:


    1. Adelson, E., and Wang, J. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Trans. PAMI 14, 2, 99–106. Google ScholarDigital Library
    2. Ashok, A., and Neifeld, M. A. 2010. Compressive Light Field Imaging. In Proc. SPIE 7690, 76900Q.Google Scholar
    3. Babacan, S., Ansorge, R., Luessi, M., Mataran, P., Molina, R., and Katsaggelos, A. 2012. Compressive Light Field Sensing. IEEE Trans. Im. Proc. 21, 12, 4746–4757.Google ScholarDigital Library
    4. Bishop, T., Zanetti, S., and Favaro, P. 2009. Light-Field Superresolution. In Proc. ICCP, 1–9.Google Scholar
    5. Candès, E., and Wakin, M. B. 2008. An Introduction to Compressive Sampling. IEEE Signal Processing 25, 2, 21–30.Google ScholarCross Ref
    6. Candès, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed Sensing with Coherent and Redundant Dictionaries. Appl. and Comp. Harmonic Analysis 31, 1, 59–73.Google ScholarCross Ref
    7. Chen, S. S., Donoho, D. L., Michael, and Saunders, A. 1998. Atomic Decomposition by Basis Pursuit. SIAM J. on Scientific Computing 20, 33–61. Google ScholarDigital Library
    8. Donoho, D. 2006. Compressed Sensing. IEEE Trans. Inform. Theory 52, 4, 1289–1306. Google ScholarDigital Library
    9. Duarte-Carvajalino, J., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization. IEEE Trans. Im. Proc. 18, 7, 1395–1408. Google ScholarDigital Library
    10. Elad, M., and Aharon, M. 2006. Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736–3745. Google ScholarDigital Library
    11. Feigin, M., Feldman, D., and Sochen, N. A. 2012. From High Definition Image to Low Space Optimization. In Scale Space and Var. Methods in Comp. Vision, vol. 6667, 459–470. Google ScholarDigital Library
    12. Fergus, R., Torralba, A., and Freeman, W. T. 2006. Random Lens Imaging. Tech. Rep. TR-2006-058, MIT.Google Scholar
    13. Georgiev, T., and Lumsdaine, A. 2006. Spatio-angular Resolution Tradeoffs in Integral Photography. Proc. EGSR, 263–272. Google ScholarDigital Library
    14. Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proc. ACM SIGGRAPH, 43–54. Google ScholarDigital Library
    15. Heide, F., Wetzstein, G., Raskar, R., and Heidrich, W. 2013. Adaptive Image Synthesis for Compressive Displays. ACM Trans. Graph. (SIGGRAPH) 32, 4, 1–11. Google ScholarDigital Library
    16. Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary. In Proc. IEEE ICCV. Google ScholarDigital Library
    17. Ives, H., 1903. Parallax Stereogram and Process of Making Same. US patent 725,567.Google Scholar
    18. Kamal, M., Golbabaee, M., and Vandergheynst, P. 2012. Light Field Compressive Sensing in Camera Arrays. In Proc. ICASSP, 5413–5416.Google Scholar
    19. Lanman, D., Raskar, R., Agrawal, A., and Taubin, G. 2008. Shield Fields: Modeling and Capturing 3D Occluders. ACM Trans. Graph. (SIGGRAPH Asia) 27, 5, 131. Google ScholarDigital Library
    20. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 30, 1–9. Google ScholarDigital Library
    21. Levin, A., Freeman, W. T., and Durand, F. 2008. Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections. In Proc. ECCV, 88–101. Google ScholarDigital Library
    22. Levin, A., Hasinoff, S. W., Green, P., Durand, F., and Freeman, W. T. 2009. 4D Frequency Analysis of Computational Cameras for Depth of Field Extension. ACM Trans. Graph. (SIGGRAPH) 28, 3, 97. Google ScholarDigital Library
    23. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In Proc. ACM SIGGRAPH, 31–42. Google ScholarDigital Library
    24. Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., and Chen, H. H. 2008. Programmable Aperture Photography: Multiplexed Light Field Acquisition. ACM Trans. Graph. (SIGGRAPH) 27, 3, 1–10. Google ScholarDigital Library
    25. Lippmann, G. 1908. La Photographie Intégrale. Academie des Sciences 146, 446–451.Google Scholar
    26. Lumsdaine, A., and Georgiev, T. 2009. The Focused Plenoptic Camera. In Proc. ICCP, 1–8.Google Scholar
    27. Mairal, J., Bach, F., Ponce, G., and Sapiro, G. 2009. Online Dictionary Learning For Sparse Coding. In International Conference on Machine Learning. Google ScholarDigital Library
    28. Marcia, R. F., and Willett, R. M. 2008. Compressive coded aperture video reconstruction. In EUSIPCO.Google Scholar
    29. Natarajan, B. K. 1995. Sparse Approximate Solutions to Linear Systems. SIAM J. Computing 24, 227–234. Google ScholarDigital Library
    30. Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Tech. rep., Stanford University.Google Scholar
    31. Ng, R. 2005. Fourier Slice Photography. ACM Trans. Graph. (SIGGRAPH) 24, 3, 735–744. Google ScholarDigital Library
    32. Park, J. Y., and Wakin, M. B. 2012. A geometric approach to multi-view compressive imaging. EURASIP Journal on Advances in Signal Processing 37.Google Scholar
    33. Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
    34. Perwass, C., and Wietzke, L. 2012. Single Lens 3D-Camera with Extended Depth-of-Field. In Proc. SPIE 8291, 29–36.Google Scholar
    35. Reddy, D., Veeraraghavan, A., and Chellappa, R. 2011. P2C2: Programmable Pixel Compressive Camera for High Speed Imaging. In Proc. IEEE CVPR, 329–336. Google ScholarDigital Library
    36. Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 609–618.Google ScholarCross Ref
    37. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocussing. ACM Trans. Graph. (SIGGRAPH) 26, 3, 69. Google ScholarDigital Library
    38. Wakin, M. B., Laska, J. N., Duarte, M. F., Baron, D., Sarvotham, S., Takhar, D., Kelly, K. F., and Baraniuk, R. G. 2006. Compressive imaging for video representation and coding. In Picture Coding Symposium.Google Scholar
    39. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. (SIGGRAPH). Google ScholarDigital Library
    40. Wetzstein, G., Ihrke, I., and Heidrich, W. 2012. On Plenoptic Multiplexing and Reconstruction. IJCV, 1–16. Google ScholarDigital Library
    41. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1–11. Google ScholarDigital Library
    42. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. (SIGGRAPH) 24, 3, 765–776. Google ScholarDigital Library
    43. Xu, Z., and Lam, E. Y. 2012. A High-resolution Lightfield Camera with Dual-mask Design. In Proc. SPIE 8500, 85000U.Google Scholar
    44. Yang, A., Ganesh, A., Sastry, S., and Ma, Y. 2010. Fast L1-Minimization Algorithms and An Application in Robust Face Recognition: A Review. Tech. rep., UC Berkeley.Google Scholar
    45. Yang, J., Wang, Z., Lin, Z., Cohen, S., and Huang, T. 2012. Coupled Dictionary Training for Image Super-Resolution. IEEE Trans. Im. Proc. 21, 8, 3467–3478.Google ScholarDigital Library

ACM Digital Library Publication: