“What Are Optimal Coding Functions for Time-of-Flight Im” by Gupta, Velten, Nayar and Breitbach

  • ©Mohit Gupta, Andreas Velten, Shree K. Nayar, and Eric Breitbach

Conference:


Type:


Title:

    What Are Optimal Coding Functions for Time-of-Flight Im

Session/Category Title: Computational Cameras

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    The depth resolution achieved by a continuous wave time-of-flight (C-ToF) imaging system is determined by the coding (modulation and demodulation) functions that it uses. Almost all current C-ToF systems use sinusoid or square coding functions, resulting in a limited depth resolution. In this article, we present a mathematical framework for exploring and characterizing the space of C-ToF coding functions in a geometrically intuitive space. Using this framework, we design families of novel coding functions that are based on Hamiltonian cycles on hypercube graphs. Given a fixed total source power and acquisition time, the new Hamiltonian coding scheme can achieve up to an order of magnitude higher resolution as compared to the current state-of-the-art methods, especially in low signal-to-noise ratio (SNR) settings. We also develop a comprehensive physically-motivated simulator for C-ToF cameras that can be used to evaluate various coding schemes prior to a real hardware implementation. Since most off-the-shelf C-ToF sensors use sinusoid or square functions, we develop a hardware prototype that can implement a wide range of coding functions. Using this prototype and our software simulator, we demonstrate the performance advantages of the proposed Hamiltonian coding functions in a wide range of imaging settings.

References:


    1. Amit Adam, Christoph Dann, Omer Yair, Shai Mazor, and Sebastian Nowozin. 2016. Bayesian time-of-flight for realtime shape, illumination and Albedo. IEEE PAMI 38, 5 (2016), 851–864. 
    2. B. Buxbaum, R. Schwarte, T. Ringbeck, M. Grothof, and X. Luan. 2002. MSM-PMD as correlation receiver in a new 3D-ranging system. Proc. SPIE 4546 (2002).
    3. D. A. Carnegie, J. R. K. McClymont, A. P. P. Jongenelen, A. A. Dorrington B. Drayto and, and A. D. Payne. 2011. Design and construction of a configurable full-field range imaging system for mobile robotic applications. Lecture Notes in Electrical Engineering 83 (2011).
    4. David Droeschel, Dirk Holz, and Sven Behnke. 2010. Multi-frequency phase unwrapping for time-of-flight cameras. In Proc. IROS.
    5. R. Ferriere, J. Cussey, and J. Dudley. 2008. Time-of-flight range detection using low frequency intensity modulation of a CW laser diode: Application to fiber length measurement. Optical Engineering 47, 9 (2008), 93602–1 to 93602–6.
    6. D. Freedman, E. Krupka, Y. Smolin, I. Leichter, and M. Schmidt. 2014. SRA: Fast removal of general multipath for ToF sensors. In Proc. ECCV.
    7. J. P. Godbaz, A. A. Dorrington, and M. J. Cree. 2013. Understanding and ameliorating mixed pixels and multipath interference in AMCW lidar. In TOF Range-Imaging Cameras, Fabio Remondino and D. Stoppa (Eds.). Springer Berlin.
    8. B. S. Goldstein and G. F. Dalrymple. 1967. Gallium arsenide injection laser radar. Proc. of the IEEE 55, 2 (1967).
    9. C. Gotsman and M. Lindenbaum. 1996. On the metric properties of discrete space-filling curves. IEEE TIP 5, 5 (1996), 794–797. 
    10. Frank Gray. 1953. Pulse code communication. US Patent 2,632,058 (1953).
    11. R. Grootjans, W. Van der Tempel, D. Van Nieuwenhovec, C. de Tandt, and M. Kuijk. 2006. Improved modulation techniques for time-of-flight ranging cameras using pseudo random binary sequences. In Proc. IEEE LEOS Benelux Chapter.
    12. M. Gupta, S. K. Nayar, M. Hullin, and J. Martin. 2015. Phasor imaging: A generalization of correlation-based time-of-flight imaging. ACM Trans. Graphics 34, 5 (2015), 156:1–156:18. 
    13. S. W. Hasinoff, F. Durand, and W. T. Freeman. 2010. Noise-optimal capture for high dynamic range photography. In Proc. IEEE CVPR.
    14. F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich. 2013. Low-budget transient imaging using photonic mixer devices. ACM SIGGRAPH 32, 4 (2013), 45:1–45:10. 
    15. E. Horn and N. Kiryati. 1997. Toward optimal structured light patterns. In International Conference on Recent Advances in 3D Digital Imaging and Modeling. 
    16. S. Inokuchi, K. Sato, and F. Matsuda. 1984. Range imaging system for 3-D object recognition. In Proc. IEEE ICPR.
    17. A. P. P. Jongenelen, D. G. Bailey, A. D. Payne, A. A. Dorrington, and D. A. Carnegie. 2011. Analysis of errors in ToF range imaging with dual-frequency modulation. IEEE Transactions on Instrumentation and Measurement 60, 5 (2011), 1861–1868.
    18. A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar. 2013. Coded ToF cameras: Sparse deconvolution to address multipath interference and recover time profiles. ACM SIGGRAPH Asia 32, 6 (2013), 167:1–167:10. 
    19. M. Kawakita, K. Iizuka, R. Iwama, K. Takizawa, H. Kikuchi, and F. Sato. 2004. Gain-modulated Axi-vision camera. Opt. Express 12, 22 (2004), 5336–5344.
    20. Maik Keller and Andreas Kolb. 2009. Real-time simulation of time-of-flight sensors. Simulation Modelling Practice and Theory 17, 5 (2009), 967–978.
    21. W. Koechner. 1968. Optical ranging system employing a high power injection laser diode. IEEE Trans. AES 4, 1 (1968), 81–91.
    22. A. Kolb, E. Barth, R. Koch, and R. Larsen. 2010. Time-of-flight cameras in computer graphics. Eurographics (2010).
    23. R. Lange. 2000. 3D ToF distance measurement with custom solid-state image sensors in CMOS-CCD-technology. Ph.D. Thesis (2000).
    24. R. Lange and P. Seitz. 2001. Solid state time-of-flight range camera. IEEE J. Quantum Electronics 37, 3 (2001), 390–397.
    25. Robert Lange, Peter Seitz, Alice Biber, and Stefan C. Lauxtermann. 2000. Demodulation pixels in CCD and CMOS technologies for time-of-flight ranging. Proc. SPIE 3965 (2000).
    26. Stephan Meister, Rahul Nair, and Daniel Kondermann. 2013. Simulation of time-of-flight sensors using global illumination. In Vision, Modeling and Visualization, Michael Bronstein, Jean Favre, and Kai Hormann (Eds.). The Eurographics Association.
    27. Microsoft-Kinect. 2014. March NPD Results: Titanfall on Xbox One is Number One Selling Game. Retrieved from http://news.xbox.com/2014/04/xbox-one-march-npd.
    28. W. H. Mills. 1963. Some complete cycles on the n-cube. Proc. Amer. Math. Soc. 14, 4 (1963), 640–643.
    29. M. O’Toole, S. Achar, S. G. Narasimhan, and K. N. Kutulakos. 2015. Homogeneous codes for energy-efficient illumination and imaging. ACM SIGGRAPH 34, 4 (2015), 35:1–35:13. 
    30. M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos. 2014. Temporal frequency probing for 5D transient analysis of global light transport. ACM SIGGRAPH 33, 4 (2014), 87:1–87:11. 
    31. A. D. Payne, A. A. Dorrington, and M. J. Cree. 2010. Illumination waveform optimization for time-of-flight range imaging cameras. In Proc. SPIE 8085.
    32. J. M. Payne. 1973. An optical distance measuring instrument. Review of Scientific Instruments 44, 3 (1973).
    33. H. Sagan. 1994. Space Filling Curves. Springer, New York.
    34. Mirko Schmidt and Bernd Jähne. 2009. A Physical Model of Time-of-Flight 3D Imaging Systems, Including Suppression of Ambient Light. Springer Berlin, Berlin, Germany, 1–15.
    35. R. Schwarte, Z. Xu, H. Heinol, J. Olk, R. Klein, B. Buxbaum, H. Fischer, and J. Schulte. 1997. New electro-optical mixing and correlating sensor: Facilities and applications of the photonic mixer device. In Proc. SPIE (3100).
    36. O. Shcherbakova, L. Pancheri, G.-F. Dalla Betta, N. Massari, and D. Stoppa. 2013. 3D camera based on linear-mode gain-modulated photodiodes. In Proc. IEEE ISSCC.
    37. Shikhar Shrestha, Felix Heide, Wolfgang Heidrich, and Gordon Wetzstein. 2016. Computational imaging with multi-camera time-of-flight systems. ACM SIGGRAPH 35, 4 (2016), 33:1–33:11. 
    38. E. Tadmor, I. Bakish, S. Felzenshtein, E. Larry, G. Yahav, and D. Cohen. 2014. A fast global shutter image sensor based on the VOD mechanism. In IEEE SENSORS. 618–621.
    39. Texas-Instruments. 2017. OPT8241 3D Time-of-Flight Sensor. Retrieved from http://www.ti.com/lit/ds/sbas704b/sbas704b.pdf.
    40. A. Torralba and A. Oliva. 2003. Statistics of natural image categories. Network 14, 3 (2003), 391–412.
    41. John F. Wakerly. 2005. Digital Design: Principles and Practices. 

ACM Digital Library Publication:


Overview Page: