“A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging” by Manakov, Restrepo, Klehm, Hegedus, Eisemann, et al. …

  • ©Alkhazur Manakov, John Restrepo, Oliver Klehm, Ramon Hegedus, Elmar Eisemann, Hans-Peter Seidel, and Ivo Ihrke

Conference:


Type:


Title:

    A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging

Session/Category Title: Computational Light Capture

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We propose a non-permanent add-on that enables plenoptic imaging with standard cameras. Our design is based on a physical copying mechanism that multiplies a sensor image into a number of identical copies that still carry the plenoptic information of interest. Via different optical filters, we can then recover the desired information. A minor modification of the design also allows for aperture sub-sampling and, hence, light-field imaging. As the filters in our design are exchangeable, a reconfiguration for different imaging purposes is possible. We show in a prototype setup that high dynamic range, multispectral, polarization, and light-field imaging can be achieved with our design.

References:


    1. Adelson, E. H., and Bergen, J. R. 1991. The Plenoptic Function and the Elements of Early Vision. In Computational Models of Visual Processing, MIT Press, 3–20.Google Scholar
    2. Adelson, E., and Wang, J. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Trans. PAMI 14, 2, 99–106. Google ScholarDigital Library
    3. Aggarwal, M., and Ahuja, N. 2001. Split aperture imaging for high dynamic range. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, vol. 2, 10–17 vol.2.Google Scholar
    4. Bando, Y., Chen, B.-Y., and Nishita, T. 2008. Extracting Depth and Matte using a Color-Filtered Aperture. ACM TOG 27, 5, 134. Google ScholarDigital Library
    5. Bayer, B. E., 1976. Color Imaging Array. US Patent 3,971,065.Google Scholar
    6. Bonnet, H., Abuter, R., Baker, A., et al. 2004. First Light of SINFONI at the VLT. ESO Messenger 117, 17, 17–24.Google Scholar
    7. Cao, X., Tong, X., Dai, Q., and Lin, S. 2011. High-Resolution Multi-Spectral Video Capture with a Hybrid Camera System. In Proc. CVPR, 297–304. Google ScholarDigital Library
    8. Cook, R. L., and DeRose, T. 2005. Wavelet Noise. ACM TOG 24, 3, 735–744. Google ScholarDigital Library
    9. Debevec, P. E., and Malik, J. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proc. SIGGRAPH, 369–378. Google ScholarDigital Library
    10. Descour, M., and Dereniak, E. 1995. Computed-tomography Imaging Spectrometer: Experimental Calibration and Reconstruction Results. Appl. Optics 34, 22, 4817–4826.Google Scholar
    11. Du, H., Tong, X., Cao, X., and Lin, S. 2009. A Prism-Based System for Multispectral Video Acquisition. In Proc. ICCV, 175–182.Google Scholar
    12. Gehm, M. E., John, R., Brady, D. J., Willett, R. M., and Schulz, T. J. 2007. Single-Shot Compressive Spectral Imaging with a Dual-Disperser Architecture. Optics Exp. 15, 21, 14013–14027.Google ScholarCross Ref
    13. Georgiev, T., Lumsdaine, A., and Chunev, G. 2011. Using Focused Plenoptic Cameras for Rich Image Capture. IEEE CG&A 31, 1, 62–73. Google ScholarDigital Library
    14. Goldstein, D. H. 2003. Polarized Light, 2nd edition. CRC Press, New York, US.Google Scholar
    15. Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proc. SIGGRAPH, 43–54. Google ScholarDigital Library
    16. Habel, R., Kudenov, M., and Wimmer, M. 2012. Practical Spectral Photography. CGF 31, 2 (May), 449–458. Google ScholarDigital Library
    17. Han, J. Y., and Perlin, K. 2003. Measuring Bidirectional Texture Reflectance with a Kaleidoscope. In Proc. SIGGRAPH, 741–748. Google ScholarDigital Library
    18. Hegedus, R., Szel, G., and Horvath, G. 2006. Imaging polarimetry of the circularly polarizing cuticle of scarab beetles (Coleoptera: Rutelidae, Cetoniidae). Vision Research 46, 2786–2797.Google ScholarCross Ref
    19. Horn, B. K. P., and Schunck, B. G. 1981. Determining Optical Flow. Artif. Intell. 17, (1–3), 185–203.Google Scholar
    20. Horstmeyer, R., Euliss, G., Athale, R., and Levoy, M. 2009. Flexible Multimodal Camera Using a Light Field Architecture. In Proc. ICCP, 1–8.Google Scholar
    21. Ihrke, I., Wetzstein, G., and Heidrich, W. 2010. A Theory of Plenoptic Multiplexing. In Proc. CVPR, 1–8.Google Scholar
    22. Ihrke, I., Reshetouski, I., Manakov, A., Tevs, A., Wand, M., and Seidel, H.-P. 2012. A Kaleidoscopic Approach to Surround Geometry and Reflectance Acquisition. In Proceedings of IEEE International Workshop on Computational Cameras and Displays, 1–8.Google Scholar
    23. Ihrke, I., 2012. Color Calibration Toolbox for MATLAB v2.0. http://giana.mmci.uni-saarland.de/software.html.Google Scholar
    24. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically Reparameterized Light Fields. In Proc. SIGGRAPH, 297–306. Google ScholarDigital Library
    25. Ives, H., 1903. Parallax Stereogram and Process of Making Same. US patent 725,567.Google Scholar
    26. Kudenov, M., and Dereniak, E. 2011. Compact Snapshot Real-Time Imaging Spectrometer. In SPIE Conf. on Elec.-Opt. Rem. Sens., Phot. Tech., and Appl. V, 81860W-1–81860W-12.Google Scholar
    27. Lanman, D., Raskar, R., Agrawal, A., and Taubin, G. 2008. Shield Fields: Modeling and Capturing 3D Occluders. ACM TOG 27, 5, 131. Google ScholarDigital Library
    28. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In Proc. SIGGRAPH, 31–42. Google ScholarDigital Library
    29. Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M. 2004. Synthetic Aperture Confocal Imaging. ACM TOG 23, 3, 825–834. Google ScholarDigital Library
    30. Lippmann, G. 1908. La Photographie Intégrale. Academie des Sciences 146, 446–451.Google Scholar
    31. Mann, S., and Picard, R. W. 1995. Being ‘Undigital’ with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures. In Proc. IS&T, 442–448.Google Scholar
    32. McGuire, M., Matusik, W., Pfister, H., Chen, B., Hughes, J. F., and Nayar, S. K. 2007. Optical Splitting Trees for High-Precision Monocular Imaging. IEEE CG&A 27, 2, 32–42. Google ScholarDigital Library
    33. Mitsunaga, T., and Nayar, S. K. 1999. Radiometric Self Calibration. In Proc. CVPR, 374–380.Google Scholar
    34. Narasimhan, S., and Nayar, S. 2005. Enhancing Resolution along Multiple Imaging Dimensions using Assorted Pixels. IEEE Trans. PAMI 27, 4, 518–530. Google ScholarDigital Library
    35. Nayar, S., and Mitsunaga, T. 2000. High Dynamic Range Imaging: Spatially Varying Pixel Exposures. In Proc. CVPR, vol. 1, 472–479.Google Scholar
    36. Neumann, L., Hegedus, R., Horvath, G., and Garcia, R. 2008. Applications of High Precision Imaging Polarimetry. In Proc. Computational Aesthetics in Graphics, Visualization and Imaging, 89–97. Google ScholarDigital Library
    37. Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Tech. Rep. Computer Science CSTR 2005-02, Stanford University.Google Scholar
    38. Ng, R. 2005. Fourier Slice Photography. ACM TOG 24, 3, 735–744. Google ScholarDigital Library
    39. Okamoto, T., and Yamaguchi, I. 1991. Simultaneous Acquisition of Spectral Image Information. Optics Lett. 16, 16, 1277–1279.Google ScholarCross Ref
    40. Park, J.-I., Lee, M.-H., Grossberg, M. D., and Nayar, S. K. 2007. Multispectral Imaging Using Multiplexed Illumination. In Proc. ICCV, 1–8.Google Scholar
    41. Pezzaniti, J. L., Chenault, D., Roche, M., Reinhardt, J., Pezzaniti, J. P., and Schultz, H. 2008. Four Camera Complete Stokes Imaging Polarimeter. In Proc. SPIE 6972, Polarization: Measurement, Analysis, and Remote Sensing VIII, 69720J-1–69720J-12.Google Scholar
    42. Reinhard, E., Ward, G., Debevec, P., Pattanaik, S., Heidrich, W., and Myszkowski, K. 2010. High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting. Morgan Kaufmann Publishers.Google Scholar
    43. Reshetouski, I., Manakov, A., Seidel, H.-P., and Ihrke, I. 2011. Three-Dimensional Kaleidoscopic Imaging. In Proc. CVPR, 353–360. Google ScholarDigital Library
    44. Rump, M., and Klein, R. 2010. Spectralization: Reconstructing spectra from sparse data. In Proc. EGSR, 1347–1354. Google ScholarDigital Library
    45. Schechner, Y., and Nayar, S. 2005. Generalized Mosaicing: Polarization Panorama. IEEE Trans. PAMI 27, 4, 631–636. Google ScholarDigital Library
    46. Schechner, Y., Narasimhan, S. G., and Nayar, S. K. 2001. Instant Dehazing of Images using Polarization. In Proc. CVPR, 325–332.Google Scholar
    47. Spiering, B. A., 1999. Multispectral Imaging System. US Patent 5,900,942.Google Scholar
    48. Tocci, M. D., Kiser, C., Tocci, N., and Sen, P. 2011. A Versatile HDR Video Production System. ACM TOG 30, 4. Google ScholarDigital Library
    49. Toyooka, S., and Hayasaka, N. 1997. Two-Dimensional Spectral Analysis using Broad-Band Filters. Optical Communications 137 (Apr), 22–26.Google ScholarCross Ref
    50. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask Enhanced Cameras For Heterodyned Light Fields and Coded Aperture Refocussing. ACM TOG 26, 3, 69. Google ScholarDigital Library
    51. Wanner, S., and Goldluecke, B. 2012. Globally Consistent Depth Labeling of 4D Lightfields. In Proc. CVPR, 41–48. Google ScholarDigital Library
    52. Wanner, S., and Goldluecke, B. 2012. Spatial and Angular Variational Super-Resolution of 4D Light Fields. In Proc. ECCV, 608–621. Google ScholarDigital Library
    53. Wetzstein, G., Ihrke, I., Lanman, D., and Heidrich, W. 2011. Computational Plenoptic Imaging. CGF 30, 8, 2397–2426.Google ScholarCross Ref
    54. Wilburn, B., Joshi, N., Vaish, V., et al. 2005. High Performance Imaging using Large Camera Arrays. ACM TOG 24, 3, 765–776. Google ScholarDigital Library
    55. Zhou, C., and Nayar, S. 2011. Computational Cameras: Convergence of Optics and Processing. IEEE Trans. IP 20, 12 (Dec), 3322–3340. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: