“Occluded Imaging With Time-of-Flight Sensor” by Kadambi, Zhao, Shi and Raskar

  • ©

Conference:


Type(s):


Title:

    Occluded Imaging With Time-of-Flight Sensor

Session/Category Title:   COMPUTATIONAL CAMERAS

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We explore the question of whether phase-based time-of-flight (TOF) range cameras can be used for looking around corners and through scattering diffusers. By connecting TOF measurements with theory from array signal processing, we conclude that performance depends on two primary factors: camera modulation frequency and the width of the specular lobe (“shininess”) of the wall. For purely Lambertian walls, commodity TOF sensors achieve resolution on the order of meters between targets. For seemingly diffuse walls, such as posterboard, the resolution is drastically reduced, to the order of 10cm. In particular, we find that the relationship between reflectance and resolution is nonlinear—a slight amount of shininess can lead to a dramatic improvement in resolution. Since many realistic scenes exhibit a slight amount of shininess, we believe that off-the-shelf TOF cameras can look around corners.

References:


    1. Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C. Miller. 2014. 3D tracking via body radio reflections. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI’14). 317–329. 
    2. Ayush Bhandari, Achuta Kadambi, Refael Whyte, Christopher Barsi, Micha Feigin, Adrian Dorrington, and Ramesh Raskar. 2014. Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization. Optics Letters 39, 6, 1705–1708.
    3. Petros T. Boufounos, Paris Smaragdis, and Bhiksha Raj. 2011. Joint sparsity models for wideband array processing. In SPIE Optical Engineering + Applications. International Society for Optics and Photonics.
    4. G. C. Carter. 1981. Time delay estimation for passive sonar signal processing. IEEE Transactions on Acoustics, Speech and Signal Processing 29, 3, 463–470.
    5. Volkan Cevher, Marco Duarte, and Richard G. Baraniuk. 2008. Distributed target localization via spatial sparsity. In Proceedings of the 16th European Signal Processing Conference (EUSIPCO’08). 1–5.
    6. Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders. 2001. Atomic decomposition by Basis Pursuit. SIAM Review 43, 1, 129–159. 
    7. Chaitanya Ekanadham, Daniel Tranchina, and Eero P. Simoncelli. 2011. Recovery of sparse translation-invariant signals with Continuous Basis Pursuit. IEEE Transactions on Signal Processing 59, 10, 4735–4744. 
    8. Michael Elad. 2010. Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer. 
    9. Ioannis Gkioulekas, Anat Levin, Frédo Durand, and Todd Zickler. 2015. Micron-scale light transport decomposition using interferometry. ACM Transactions on Graphics 34, 4, 37. 
    10. John Clifford Gower. 1985. Properties of Euclidean and non-Euclidean distance matrices. Linear Algebra and Its Applications 67, 81–97.
    11. Mohit Gupta, Shree K. Nayar, Matthias B. Hullin, and Jaime Martin. 2015. Phasor imaging: A generalization of correlation-based time-of-flight imaging. ACM Transactions on Graphics 34, 5, Article No. 156. 
    12. Charles Han, Bo Sun, Ravi Ramamoorthi, and Eitan Grinspun. 2007. Frequency domain normal map filtering. ACM Transactions on Graphics 26, 28. 
    13. Chinmay Hegde, Piotr Indyk, and Ludwig Schmidt. 2014. Approximation-tolerant model-based compressive sensing. In Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms. 1544–1561. 
    14. Felix Heide, Matthias B. Hullin, James Gregson, and Wolfgang Heidrich. 2013. Low-budget transient imaging using photonic mixer devices. ACM Transactions on Graphics 32, 4, Article No. 45. 
    15. Felix Heide, Lei Xiao, Wolfgang Heidrich, and Matthias B. Hullin. 2014. Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 
    16. Suren Jayasuriya, Adithya Pediredla, Sriram Sivaramakrishnan, Alyosha Molnar, and Ashok Veeraraghavan. 2015. Depth fields: Extending light field techniques to time-of-flight imaging. Proceedings of the International Conference on 3D Vision (3DV’15). 
    17. Achuta Kadambi, Refael Whyte, Ayush Bhandari, Lee Streeter, Christopher Barsi, Adrian Dorrington, and Ramesh Raskar. 2013. Coded time of flight cameras: Sparse deconvolution to address multipath interference and recover time profiles. ACM Transactions on Graphics 32, 6, 167. 
    18. Jingyu Lin, Yebin Liu, Matthias B. Hullin, and Qionghai Dai. 2014. Fourier analysis on transient imaging with a multifrequency time-of-flight camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, Los Alamitos, CA, 3230–3237. 
    19. Dmitry Malioutov, Müjdat Çetin, and Alan S. Willsky. 2005. A sparse signal reconstruction perspective for source localization with sensor arrays. IEEE Transactions on Signal Processing 53, 8, 3010–3022. 
    20. Stephane G. Mallat and Zhifeng Zhang. 1993. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41, 12, 3397–3415. 
    21. B. Masia. 2014. Computational imaging: Combining optics, computation and perception. ACM Transactions on Graphics 32, 6.
    22. Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003. A data-driven reflectance model. ACM Transactions on Graphics 22, 3, 759–769. 
    23. J. D. Maynard, E. G. Williams, and Y. Lee. 1985. Nearfield acoustic holography: I. Theory of generalized holography and the development of NAH. Journal of the Acoustical Society of America 78, 1395.
    24. Nikhil Naik, Shuang Zhao, Andreas Velten, Ramesh Raskar, and Kavita Bala. 2011. Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Transactions on Graphics 30, 171. 
    25. Addy Ngan, Frédo Durand, and Wojciech Matusik. 2005. Experimental analysis of BRDF models. In Proceedings of the 16th Eurographics Conference on Rendering Techniques. 117–126. 
    26. Matthew O’Toole, Felix Heide, Lei Xiao, Matthias B. Hullin, Wolfgang Heidrich, and Kiriakos N. Kutulakos. 2014. Temporal frequency probing for 5d transient analysis of global light transport. ACM Transactions on Graphics 33, 4, 87. 
    27. Ravi Ramamoorthi and Pat Hanrahan. 2001. A signal-processing framework for inverse rendering. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, 117–128. 
    28. Ilya Reshetouski and Ivo Ihrke. 2013. Mirrors in Computer Graphics, Computer Vision and Time-of-Flight Imaging. Springer.
    29. I. Reshetouski, A. Manakov, H.-P. Seidel, and I. Ihrke. 2011. Three-dimensional kaleidoscopic imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE, Los Alamitos, CA, 353–360. 
    30. Yair Rivenson, Adrian Stern, and Joseph Rosen. 2013. Reconstruction guarantees for compressive tomographic holography. Optics Letters 38, 14, 2509–2511.
    31. Satyabrata Sen and Arye Nehorai. 2011. Adaptive OFDM radar for target detection in multipath scenarios. IEEE Transactions on Signal Processing 59, 1, 78–90. 
    32. Ain Sume, Magnus Gustafsson, Magnus Herberthson, Anna Janis, Stefan Nilsson, Jonas Rahm, and Anders Orbom. 2011. Radar detection of moving targets behind corners. IEEE Transactions on Geoscience and Remote Sensing 49, 6, 2259–2267.
    33. Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological) 58, 1, 267–288.
    34. Harry L. Van Trees. 2004. Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory. John Wiley & Sons.
    35. Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan, Moungi G. Bawendi, and Ramesh Raskar. 2012a. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communications 3, 745.
    36. Andreas Velten, Di Wu, Adrian Jarabo, Belen Masia, Christopher Barsi, Chinmaya Joshi, Everett Lawson, Moungi Bawendi, Diego Gutierrez, and Ramesh Raskar. 2013. Femto-photography: Capturing and visualizing the propagation of light. ACM Transactions on Graphics 32, 4, 44. 
    37. Andreas Velten, Di Wu, Adrian Jarabo, Belen Masia, Christopher Barsi, Everett Lawson, Chinmaya Joshi, Diego Gutierrez, Moungi G. Bawendi, and Ramesh Raskar. 2012b. Relativistic ultrafast rendering using time-of-flight imaging. In Proceedings of ACM SIGGRAPH 2012 Talks. ACM, New York, NY, Article No. 41. 
    38. Gregory Ward. 1992. Measuring and modeling anisotropic reflection. Computer Graphics 26, 2, 265–272. 
    39. Di Wu, Gordon Wetzstein, Christopher Barsi, Thomas Willwacher, Qionghai Dai, and Ramesh Raskar. 2014. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. International Journal of Computer Vision 110, 2, 128–140.

ACM Digital Library Publication:


Overview Page: