“Light-Efficient Holographic Illumination for Continuous-Wave Time-of-Flight Imaging” by Chan and O’Toole
Conference:
Type(s):
Title:
- Light-Efficient Holographic Illumination for Continuous-Wave Time-of-Flight Imaging
Session/Category Title: Computer Vision
Presenter(s)/Author(s):
Abstract:
Time-of-flight (TOF) cameras have seen widespread adoption in recent years across the entire spectrum of commodity devices. However, these devices are fundamentally limited by their dynamic range, struggling with saturation from nearby bright objects and noisy depth from farther darker objects. In this work, we explore overcoming these limitations in the context of continuous-wave time-of-flight (CWTOF) devices, by using a holographic light source capable of redistributing light according to arbitrary patterns. In particular, we propose using such a system to move light from overexposed to underexposed regions of a scene, such that the entire scene is well exposed. Such a methodology can be easily integrated with existing illumination schemes for TOF. Our proof-of-concept prototype is constructed from off-the-shelf optical components, and demonstrated on a number of lab scenes.
References:
[1]
Supreeth Achar, Joseph R Bartels, William L’Red’ Whittaker, Kiriakos N Kutulakos, and Srinivasa G Narasimhan. 2017. Epipolar time-of-flight imaging. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–8.
[2]
Gianluca Agresti and Pietro Zanuttigh. 2018. Combination of spatially-modulated ToF and structured light for MPI-free depth estimation. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 0–0.
[3]
Toshiyuki Amano and Hirokazu Kato. 2008. Real world dynamic appearance enhancement with procam feedback. In Proceedings of the 5th ACM/IEEE International Workshop on Projector camera systems. 1–2.
[4]
Toshiyuki Amano and Ryo Suzuki. 2007. Virtual recovery of the deteriorated art object based on AR technology. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–2.
[5]
Joseph R Bartels, Jian Wang, William Whittaker, Srinivasa G Narasimhan, 2019. Agile depth sensing using triangulation light curtains. In Int. Conf. Comput. Vis.7900–7908.
[6]
Terry A Bartlett, William C McDonald, and James N Hall. 2019. Adapting Texas Instruments DLP technology to demonstrate a phase spatial light modulator. In Emerging Digital Micromirror Device Based Systems and Applications XI, Vol. 10932. SPIE, 161–173.
[7]
Alexander W Bergman, David B Lindell, and Gordon Wetzstein. 2020. Deep adaptive lidar: End-to-end optimization of sampling and depth completion at low sampling rates. In International conference on computational photography (ICCP). IEEE, 1–11.
[8]
Oliver Bimber and Daisuke Iwai. 2008. Superimposing dynamic range. ACM Transactions on Graphics (TOG) 27, 5 (2008), 1–8.
[9]
Oliver Bimber, Daniel Klöck, Toshiyuki Amano, Anselm Grundhöfer, and Daniel Kurz. 2010. Closed-loop feedback illumination for optical inverse tone-mapping in light microscopy. IEEE Transactions on Visualization and Computer Graphics 17, 6 (2010), 857–870.
[10]
Praneeth Chakravarthula, Yifan Peng, Joel Kollin, Henry Fuchs, and Felix Heide. 2019. Wirtinger holography for near-eye displays. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–13.
[11]
Dorian Chan, Srinivasa G Narasimhan, and Matthew O’Toole. 2022. Holocurtains: Programming Light Curtains via Binary Holography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17886–17895.
[12]
Miguel Heredia Conde, Klaus Hartmann, and Otmar Loffeld. 2014. Adaptive high dynamic range for time-of-flight cameras. IEEE Transactions on Instrumentation and Measurement 64, 7 (2014), 1885–1906.
[13]
Qiqin Dai, Fengqiang Li, Oliver Cossairt, and Aggelos K Katsaggelos. 2022. Adaptive Illumination Based Depth Sensing Using Deep Superpixel and Soft Sampling Approximation. IEEE Transactions on Computational Imaging 8 (2022), 224–235.
[14]
Gerwin Damberg, Anders Ballestad, Eric Kozak, Johannes Minor, Raveen Kumaran, and James Gregson. 2015. High brightness HDR projection using dynamic phase modulation. In ACM SIGGRAPH 2015 Emerging Technologies. 1–1.
[15]
Gerwin Damberg, James Gregson, and Wolfgang Heidrich. 2016. High brightness HDR projection using dynamic freeform lensing. ACM Transactions on Graphics (TOG) 35, 3 (2016), 1–11.
[16]
Paul E. Debevec and Jitendra Malik. 1997. Recovering High Dynamic Range Radiance Maps from Photographs. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., USA, 369–378.
[17]
Dragos Falie and Vasile Buzuloiu. 2007. Noise characteristics of 3D time-of-flight cameras. In 2007 International Symposium on Signals, Circuits and Systems, Vol. 1. IEEE, 1–4.
[18]
David Ferstl, Christian Reinbacher, Gernot Riegler, Matthias Rüther, and Horst Bischof. 2015. Learning Depth Calibration of Time-of-Flight Cameras. In BMVC. 102–1.
[19]
James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758–2769.
[20]
Chunyu Gao, Narendra Ahuja, and Hong Hua. 2007. Active aperture control and sensor modulation for flexible imaging. In CVPR. IEEE, 1–8.
[21]
R. W. Gerchberg and W. O. Saxton. 1972. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35 (1972), 237–246.
[22]
Michael D Grossberg, Harish Peri, Shree K Nayar, and Peter N Belhumeur. 2004. Making one object look like another: Controlling appearance using a projector-camera system. In Computer Vision and Pattern Recognition, Vol. 1. IEEE, I–I.
[23]
Mohit Gupta, Qi Yin, and Shree K Nayar. 2013. Structured light in sunlight. In Proceedings of the IEEE International Conference on Computer Vision. 545–552.
[24]
Felipe Gutierrez-Barragan, Huaijin Chen, Mohit Gupta, Andreas Velten, and Jinwei Gu. 2021. itof2dtof: A robust and flexible representation for data-driven time-of-flight imaging. IEEE Transactions on Computational Imaging 7 (2021), 1205–1214.
[25]
Reynald Hoskinson, Boris Stoeber, Wolfgang Heidrich, and Sidney Fels. 2010. Light reallocation for high contrast projection using an analog micromirror array. ACM Transactions on Graphics (TOG) 29, 6 (2010), 1–10.
[26]
Marvin Lindner and Andreas Kolb. 2007. Calibration of the intensity-related distance error of the PMD ToF-camera. In Intelligent Robots and Computer Vision XXV: Algorithms, Techniques, and Active Vision, Vol. 6764. SPIE, 338–345.
[27]
Tian Yu Liu, Parth Agrawal, Allison Chen, Byung-Woo Hong, and Alex Wong. 2022. Monitored distillation for positive congruent depth completion. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part II. Springer, 35–53.
[28]
Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–16.
[29]
Hidetoshi Mannami, Ryusuke Sagawa, Yasuhiro Mukaigawa, Tomio Echigo, and Yasushi Yagi. 2007. High dynamic range camera using reflective liquid crystal. In International Conference on Computer Vision. IEEE, 1–8.
[30]
Christoph Mertz, Sanjeev J Koppal, Solomon Sia, and Srinivasa Narasimhan. 2012. A low-power structured light sensor for outdoor scene reconstruction and dominant material identification. In CVPR Workshops. IEEE, 15–22.
[31]
James Mure-Dubois and Heinz Hügli. 2007. Real-time scattering compensation for time-of-flight camera. In International Conference on Computer Vision Systems: Proceedings (2007).
[32]
Shree K Nayar and Vlad Branzoi. 2003. Adaptive Dynamic Range Imaging: Optical Control of Pixel Exposures Over Space and Time. In ICCV, Vol. 2. 1168–1175.
[33]
Shree K Nayar, Vlad Branzoi, and Terrance E Boult. 2004. Programmable imaging using a digital micromirror array. In 2004 IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. IEEE, I–I.
[34]
Shree K Nayar, Harish Peri, Michael D Grossberg, and Peter N Belhumeur. 2003. A projection system with radiometric compensation for screen imperfections. In ICCV workshop on projector-camera systems (PROCAMS), Vol. 3.
[35]
Matthew O’Toole, Supreeth Achar, Srinivasa G Narasimhan, and Kiriakos N Kutulakos. 2015. Homogeneous codes for energy-efficient illumination and imaging. ACM Transactions on Graphics (ToG) 34, 4 (2015), 1–13.
[36]
Matthew O’Toole, Felix Heide, Lei Xiao, Matthias B Hullin, Wolfgang Heidrich, and Kiriakos N Kutulakos. 2014. Temporal frequency probing for 5D transient analysis of global light transport. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–11.
[37]
Andrew D Payne, Adrian A Dorrington, Michael J Cree, and Dale A Carnegie. 2010. Improved measurement linearity and precision for AMCW time-of-flight range imaging cameras. Applied optics 49, 23 (2010), 4392–4403.
[38]
Yifan Peng, Suyeon Choi, Nitish Padmanaban, and Gordon Wetzstein. 2020. Neural holography with camera-in-the-loop training. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.
[39]
Francesco Pittaluga, Zaid Tasneem, Justin Folden, Brevin Tilmon, Ayan Chakrabarti, and Sanjeev J Koppal. 2020. Towards a mems-based adaptive lidar. In 2020 International Conference on 3D Vision (3DV). IEEE, 1216–1226.
[40]
Jochen Radmer, Pol Moser Fusté, Henning Schmidt, and Jorg Kruger. 2008. Incident light related distance error study and calibration of the PMD-range imaging camera. In Computer Vision and Pattern Recognition Workshops. IEEE, 1–6.
[41]
Malcolm Reynolds, Jozef Doboš, Leto Peel, Tim Weyrich, and Gabriel J Brostow. 2011. Capturing time-of-flight data with confidence. In CVPR 2011. IEEE, 945–952.
[42]
Gal Shtendel and Ayush Bhandari. 2022. HDR-TOF: HDR Time-of-Flight Imaging via Modulo Acquisition. In 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 3808–3812.
[43]
Zaid Tasneem, Charuvahan Adhivarahan, Dingkang Wang, Huikai Xie, Karthik Dantu, and Sanjeev J Koppal. 2020. Adaptive fovea for scanning depth sensors. The International Journal of Robotics Research 39, 7 (2020), 837–855.
[44]
Brevin Tilmon, Zhanghao Sun, Sanjeev J Koppal, Yicheng Wu, Georgios Evangelidis, Ramzi Zahreddine, Gurunandan Krishnan, Sizhuo Ma, and Jian Wang. 2023. Energy-Efficient Adaptive 3D Sensing. In CVPR 2023.
[45]
Mayuka Tsuji, Hiroyuki Kubo, Suren Jayasuriya, Takuya Funatomi, and Yasuhiro Mukaigawa. 2021. Touch Sensing for a Projected Screen Using Slope Disparity Gating. IEEE Access 9 (2021), 106005–106013.
[46]
Tomoki Ueda, Hiroyuki Kubo, Suren Jayasuriya, Takuya Funatomi, and Yasuhiro Mukaigawa. 2019. Slope disparity gating using a synchronized projector-camera system. In International Conference on Computational Photography (ICCP). IEEE, 1–9.
[47]
Jian Wang, Joseph Bartels, William Whittaker, Aswin C Sankaranarayanan, and Srinivasa G Narasimhan. 2018. Programmable triangulation light curtains. In Eur. Conf. Comput. Vis.19–34.
[48]
Taeseong Woo, Hye Yun Kim, Su Yeon Kim, Byungjae Hwang, Cheolwoo Ahn, Seok-Kyu Kwon, Jae-Ick Kim, and Jung-Hoon Park. 2022. High-throughput high-dynamic range imaging by spatiotemporally structured illumination. APL Photonics 7, 10 (2022), 106106.
