“Homogeneous codes for energy-efficient illumination and imaging”

  • ©Matthew O'Toole, Supreeth Achar, Srinivasa Narasimhan, and Kiriakos N. Kutulakos

Conference:


Type:


Title:

    Homogeneous codes for energy-efficient illumination and imaging

Session/Category Title: Computational Illumination

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Programmable coding of light between a source and a sensor has led to several important results in computational illumination, imaging and display. Little is known, however, about how to utilize energy most effectively, especially for applications in live imaging. In this paper, we derive a novel framework to maximize energy efficiency by “homogeneous matrix factorization” that respects the physical constraints of many coding mechanisms (DMDs/LCDs, lasers, etc.). We demonstrate energy-efficient imaging using two prototypes based on DMD and laser illumination. For our DMD-based prototype, we use fast local optimization to derive codes that yield brighter images with fewer artifacts in many transport probing tasks. Our second prototype uses a novel combination of a low-power laser projector and a rolling shutter camera. We use this prototype to demonstrate never-seen-before capabilities such as (1) capturing live structured-light video of very bright scenes—even a light bulb that has been turned on; (2) capturing epipolar-only and indirect-only live video with optimal energy efficiency; (3) using a low-power projector to reconstruct 3D objects in challenging conditions such as strong indirect light, strong ambient light, and smoke; and (4) recording live video from a projector’s—rather than the camera’s—point of view.

References:


    1. Bach, F., Mairal, J., and Ponce, J. 2008. Convex Sparse Matrix Factorizations. Preprint arXiv: 0812.1869.Google Scholar
    2. Cossairt, O., Gupta, M., and Nayar, S. 2012. When Does Computational Imaging Improve Performance? IEEE TIP 22, 2, 447–458. Google ScholarDigital Library
    3. Damberg, G., and Heidrich, W. 2015. Efficient freeform lens optimization for computational caustic displays. Opt. Express 23, 8 (Apr), 10224–10232.Google ScholarCross Ref
    4. Damberg, G., Ballestad, A., Kozak, E., Kumaran, R., and Minor, J. 2014. Efficient, High Brightness, High Dynamic Range Projection. In ACM SIGGRAPH 2014 Emerging Technologies, 18:1–18:1. Google ScholarDigital Library
    5. Decker, Jr, J. A., and Harwit, M. 1969. Experimental operation of a Hadamard spectrometer. Appl. Opt 8, 12, 2552–2554.Google ScholarCross Ref
    6. Gu, J., Nayar, S., Grinspun, E., Belhumeur, P. N., and Ramamoorthi, R. 2013. Compressive Structured Light for Recovering Inhomogeneous Participating Media. IEEE T-PAMI 35, 3, 1–14.Google ScholarCross Ref
    7. Gupta, M., Yin, Q., and Nayar, S. 2013. Structured Light in Sunlight. In Proc. IEEE ICCV, 545–552. Google ScholarDigital Library
    8. Haeffele, B., Young, E., and Vidal, R. 2014. Structured Low-Rank Matrix Factorization: Optimality, Algorithm, and Applications to Image Processing. In Proc. ICML, 2007–015.Google Scholar
    9. Harwit, M., and Sloane, N. J. A. 1979. Hadamard Transform Optics. In Hadamard Transform Optics. Academic Press.Google Scholar
    10. Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a single coded exposure photograph using a learned over-complete dictionary. In Proc. IEEE ICCV, 287–294. Google ScholarDigital Library
    11. Hoskinson, R., Stoeber, B., Heidrich, W., and Fels, S. 2010. Light reallocation for high contrast projection using an analog micromirror array. ACM SIGGRAPH Asia 29, 6 (Dec.), 165:1–165:10. Google ScholarDigital Library
    12. Ibbett, R. N., Aspinall, D., and Grainger, J. F. 1968. Real-Time Multiplexing of Dispersed Spectra in Any Wavelength Region. Appl Optics 7, 6, 1089–1093.Google ScholarCross Ref
    13. Koppal, S., and Narasimhan, S. 2015. Beyond perspective dual photography with illumination masks. IEEE Transactions on Image Processing 24, 7 (July), 2083–2097.Google Scholar
    14. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3d displays using low-rank light field factorization. ACM SIGGRAPH Asia 29, 6 (Dec.), 163:1–163:10. Google ScholarDigital Library
    15. Matsuda, N., Cossairt, O., and Gupta, M. 2015. MC3D: Motion Contrast 3D Scanning. In Proc. IEEE ICCP, 147–156.Google Scholar
    16. Mertz, C., Koppal, S. J., Sia, S., and Narasimhan, S. G. 2012. A low-power structured light sensor for outdoor scene reconstruction and dominant material identification. In Proc. IEEE PROCAMS, 15–22.Google Scholar
    17. Mitra, K., Cossairt, O., and Veeraraghavan, A. 2014. Can we beat Hadamard multiplexing? Data driven design and analysis for computational imaging systems. In Proc. IEEE ICCP, 1–9.Google Scholar
    18. Mitra, K., Cossairt, O. S., and Veeraraghavan, A. 2014. A Framework for Analysis of Computational Imaging Systems: Role of Signal Prior, Sensor Noise and Multiplexing. IEEE T-PAMI 36, 10, 1909–1921.Google ScholarCross Ref
    19. Muller, M., 2012. Confocal imaging device using spatially modulated illumination with electronic rolling shutter detection, Aug. 7. US Patent 8,237,835.Google Scholar
    20. Nayar, S. K., Branzoi, V., and Boult, T. 2004. Programmable imaging using a digital micromirror array. In Proc. CVPR, 436–443.Google Scholar
    21. O’Toole, M., Raskar, R., and Kutulakos, K. N. 2012. Primal-dual coding to probe light transport. ACM SIGGRAPH 31, 4 (July), 39:1–39:11. Google ScholarDigital Library
    22. O’Toole, M., Mather, J., and Kutulakos, K. N. 2014. 3D Shape and Indirect Appearance by Structured Light Transport. In Proc. CVPR, 3246–3253. Google ScholarDigital Library
    23. Parikh, N., and Boyd, S. 2014. Proximal Algorithms. Foundations and Trends in Optimization 1, 3. Google ScholarDigital Library
    24. Raskar, R., Agrawal, A., and Tumblin, J. 2006. Coded exposure photography: Motion deblurring using fluttered shutter. ACM SIGGRAPH 25, 3 (July), 795–804. Google ScholarDigital Library
    25. Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. N. 2007. Multiplexing for optimal lighting. IEEE T-PAMI 29, 8, 1339–1354. Google ScholarDigital Library
    26. Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual photography. ACM SIGGRAPH 24, 3 (July), 745–755. Google ScholarDigital Library
    27. Takhar, D., Laska, J. N., Wakin, M. B., Duarte, M. F., Sarvotham, D. B. S., Kelly, K. F., and Baraniuk, R. G. 2006. A New Compressive Imaging Camera Architecture using Optical-Domain Compression. In Proc. Computational Imaging IV, 43–52.Google Scholar
    28. Veeraraghavan, A., Reddy, D., and Raskar, R. 2011. Coded strobing photography: compressive sensing of high speed periodic videos. IEEE T-PAMI 33, 4, 671–686. Google ScholarDigital Library
    29. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM SIGGRAPH 31, 4 (July), 80:1–80:11. Google ScholarDigital Library
    30. Xu, Y., and Yin, W. 2013. A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 3, 1758–1789.Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: