“Compressive light field photography using overcomplete dictionaries and optimized projections” by Marwah, Wetzstein, Bando and Raskar
Conference:
Type:
Session Title:
- Computational Light Capture
Title:
- Compressive light field photography using overcomplete dictionaries and optimized projections
Moderator(s):
Presenter(s)/Author(s):
Abstract:
Light field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into a single 2D sensor image or require multiple photographs to be taken for acquiring a high-resolution light field. We propose a compressive light field camera architecture that allows for higher-resolution light fields to be recovered than previously possible from a single image. The proposed architecture comprises three key components: light field atoms as a sparse representation of natural light fields, an optical design that allows for capturing optimized 2D light field projections, and robust sparse reconstruction methods to recover a 4D light field from a single coded 2D projection. In addition, we demonstrate a variety of other applications for light field atoms and sparse coding, including 4D light field compression and denoising.
References:
1. Adelson, E., and Wang, J. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Trans. PAMI 14, 2, 99–106. Google ScholarDigital Library
2. Ashok, A., and Neifeld, M. A. 2010. Compressive Light Field Imaging. In Proc. SPIE 7690, 76900Q.Google Scholar
3. Babacan, S., Ansorge, R., Luessi, M., Mataran, P., Molina, R., and Katsaggelos, A. 2012. Compressive Light Field Sensing. IEEE Trans. Im. Proc. 21, 12, 4746–4757.Google ScholarDigital Library
4. Bishop, T., Zanetti, S., and Favaro, P. 2009. Light-Field Superresolution. In Proc. ICCP, 1–9.Google Scholar
5. Candès, E., and Wakin, M. B. 2008. An Introduction to Compressive Sampling. IEEE Signal Processing 25, 2, 21–30.Google ScholarCross Ref
6. Candès, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed Sensing with Coherent and Redundant Dictionaries. Appl. and Comp. Harmonic Analysis 31, 1, 59–73.Google ScholarCross Ref
7. Chen, S. S., Donoho, D. L., Michael, and Saunders, A. 1998. Atomic Decomposition by Basis Pursuit. SIAM J. on Scientific Computing 20, 33–61. Google ScholarDigital Library
8. Donoho, D. 2006. Compressed Sensing. IEEE Trans. Inform. Theory 52, 4, 1289–1306. Google ScholarDigital Library
9. Duarte-Carvajalino, J., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization. IEEE Trans. Im. Proc. 18, 7, 1395–1408. Google ScholarDigital Library
10. Elad, M., and Aharon, M. 2006. Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736–3745. Google ScholarDigital Library
11. Feigin, M., Feldman, D., and Sochen, N. A. 2012. From High Definition Image to Low Space Optimization. In Scale Space and Var. Methods in Comp. Vision, vol. 6667, 459–470. Google ScholarDigital Library
12. Fergus, R., Torralba, A., and Freeman, W. T. 2006. Random Lens Imaging. Tech. Rep. TR-2006-058, MIT.Google Scholar
13. Georgiev, T., and Lumsdaine, A. 2006. Spatio-angular Resolution Tradeoffs in Integral Photography. Proc. EGSR, 263–272. Google ScholarDigital Library
14. Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proc. ACM SIGGRAPH, 43–54. Google ScholarDigital Library
15. Heide, F., Wetzstein, G., Raskar, R., and Heidrich, W. 2013. Adaptive Image Synthesis for Compressive Displays. ACM Trans. Graph. (SIGGRAPH) 32, 4, 1–11. Google ScholarDigital Library
16. Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary. In Proc. IEEE ICCV. Google ScholarDigital Library
17. Ives, H., 1903. Parallax Stereogram and Process of Making Same. US patent 725,567.Google Scholar
18. Kamal, M., Golbabaee, M., and Vandergheynst, P. 2012. Light Field Compressive Sensing in Camera Arrays. In Proc. ICASSP, 5413–5416.Google Scholar
19. Lanman, D., Raskar, R., Agrawal, A., and Taubin, G. 2008. Shield Fields: Modeling and Capturing 3D Occluders. ACM Trans. Graph. (SIGGRAPH Asia) 27, 5, 131. Google ScholarDigital Library
20. Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 30, 1–9. Google ScholarDigital Library
21. Levin, A., Freeman, W. T., and Durand, F. 2008. Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections. In Proc. ECCV, 88–101. Google ScholarDigital Library
22. Levin, A., Hasinoff, S. W., Green, P., Durand, F., and Freeman, W. T. 2009. 4D Frequency Analysis of Computational Cameras for Depth of Field Extension. ACM Trans. Graph. (SIGGRAPH) 28, 3, 97. Google ScholarDigital Library
23. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In Proc. ACM SIGGRAPH, 31–42. Google ScholarDigital Library
24. Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., and Chen, H. H. 2008. Programmable Aperture Photography: Multiplexed Light Field Acquisition. ACM Trans. Graph. (SIGGRAPH) 27, 3, 1–10. Google ScholarDigital Library
25. Lippmann, G. 1908. La Photographie Intégrale. Academie des Sciences 146, 446–451.Google Scholar
26. Lumsdaine, A., and Georgiev, T. 2009. The Focused Plenoptic Camera. In Proc. ICCP, 1–8.Google Scholar
27. Mairal, J., Bach, F., Ponce, G., and Sapiro, G. 2009. Online Dictionary Learning For Sparse Coding. In International Conference on Machine Learning. Google ScholarDigital Library
28. Marcia, R. F., and Willett, R. M. 2008. Compressive coded aperture video reconstruction. In EUSIPCO.Google Scholar
29. Natarajan, B. K. 1995. Sparse Approximate Solutions to Linear Systems. SIAM J. Computing 24, 227–234. Google ScholarDigital Library
30. Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Tech. rep., Stanford University.Google Scholar
31. Ng, R. 2005. Fourier Slice Photography. ACM Trans. Graph. (SIGGRAPH) 24, 3, 735–744. Google ScholarDigital Library
32. Park, J. Y., and Wakin, M. B. 2012. A geometric approach to multi-view compressive imaging. EURASIP Journal on Advances in Signal Processing 37.Google Scholar
33. Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
34. Perwass, C., and Wietzke, L. 2012. Single Lens 3D-Camera with Extended Depth-of-Field. In Proc. SPIE 8291, 29–36.Google Scholar
35. Reddy, D., Veeraraghavan, A., and Chellappa, R. 2011. P2C2: Programmable Pixel Compressive Camera for High Speed Imaging. In Proc. IEEE CVPR, 329–336. Google ScholarDigital Library
36. Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 609–618.Google ScholarCross Ref
37. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocussing. ACM Trans. Graph. (SIGGRAPH) 26, 3, 69. Google ScholarDigital Library
38. Wakin, M. B., Laska, J. N., Duarte, M. F., Baron, D., Sarvotham, S., Takhar, D., Kelly, K. F., and Baraniuk, R. G. 2006. Compressive imaging for video representation and coding. In Picture Coding Symposium.Google Scholar
39. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. (SIGGRAPH). Google ScholarDigital Library
40. Wetzstein, G., Ihrke, I., and Heidrich, W. 2012. On Plenoptic Multiplexing and Reconstruction. IJCV, 1–16. Google ScholarDigital Library
41. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1–11. Google ScholarDigital Library
42. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. (SIGGRAPH) 24, 3, 765–776. Google ScholarDigital Library
43. Xu, Z., and Lam, E. Y. 2012. A High-resolution Lightfield Camera with Dual-mask Design. In Proc. SPIE 8500, 85000U.Google Scholar
44. Yang, A., Ganesh, A., Sastry, S., and Ma, Y. 2010. Fast L1-Minimization Algorithms and An Application in Robust Face Recognition: A Review. Tech. rep., UC Berkeley.Google Scholar
45. Yang, J., Wang, Z., Lin, Z., Cohen, S., and Huang, T. 2012. Coupled Dictionary Training for Image Super-Resolution. IEEE Trans. Im. Proc. 21, 8, 3467–3478.Google ScholarDigital Library
