“A Dataset for Benchmarking Time-Resolved Non-Line-of-Sight Imaging” by Galindo, Wetzstein, Marco, Gutierrez, O’Toole, et al. …

  • ©Miguel Galindo, Julio Marco, Diego Gutierrez, Matthew O'Toole, and Adrian Jarabo

  • ©Miguel Galindo, Julio Marco, Diego Gutierrez, Matthew O'Toole, and Adrian Jarabo


Entry Number: 73


    A Dataset for Benchmarking Time-Resolved Non-Line-of-Sight Imaging



    Time-resolved imaging has made it possible to look around corners by exploiting information from diffuse light bounces. While there have been successive improvements in the field since its conception, so far it has only been proven to work in very simple and controlled scenarios. We present a public dataset of synthetic time-resolved Non-Line-of-Sight (NLOS) scenes with varied complexity aimed at benchmarking reconstructions. It includes scenes that are common in the real world but remain a challenge for NLOS reconstruction methods due to the ambiguous nature of higher-order diffuse bounces naturally occurring in them. With over 300 re-constructible scenes, the dataset contains an order of magnitude more scenes than what is available currently. The final objective of the dataset it to boost NLOS research to take it closer to its real-world applications.


    • Victor Arellano, Diego Gutierrez, and Adrian Jarabo. 2017. Fast Back-Projection for Non-Line of Sight Reconstruction. Optics Express 25, 10 (2017).
    • D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. 2012. A naturalistic open source movie for optical flow evaluation. In The European Conference on Computer Vision (ECCV), A. Fitzgibbon et al. (Eds.) (Ed.). Springer-Verlag.
    • Mauro Buttafava, Jessica Zeman, Alberto Tosi, Kevin Eliceiri, and Andreas Velten. 2015. Non-line-of-sight imaging using a time-gated single photon avalanche diode. Optics Express 23, 16 (Aug 2015), 20997–21011. 
    • Qi Guo, Iuri Frosio, Orazio Gallo, Todd Zickler, and Jan Kautz. 2018. Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset. In The European Conference on Computer Vision (ECCV). 
    • Felix Heide, Matthew O’Toole, Kai Zang, David Lindell, Steven Diamond, and Gordon Wetzstein. 2018. Non-line-of-sight Imaging with Partial Occluders and Surface Normals. arXiv:cs.CV/1711.07134 arXiv:1711.07134. 
    • Felix Heide, Lei Xiao, Wolfgang Heidrich, and Matthias B. Hullin. 2014. Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using inexpensive Time-ofFlight Sensors. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). 
    • Adrian Jarabo, Julio Marco, Adolfo Muñoz, Raul Buisan, Wojciech Jarosz, and Diego Gutierrez. 2014. A Framework for Transient Rendering. ACM Transactions on Graphics (SIGGRAPH Asia 2014) 33, 6 (2014). 
    • Adrian Jarabo, Belen Masia, Julio Marco, and Diego Gutierrez. 2017. Recent Advances in Transient Imaging: A Computer Graphics and Vision Perspective. Visual Informatics 1, 1 (2017). Ahmed Kirmani, Tyler Hutchison, James Davis, and Ramesh Raskar. 2009. Looking around the corner using transient imaging. In Computer Vision, 2009 IEEE 12th International Conference on. IEEE, 159–166. 
    • Jonathan Klein, Martin Laurenzis, Dominik L. Michels, and Matthias B. Hullin. 2018. A Quantitative Platform for Non-Line-of-Sight Imaging Problems. In British Machine Vision Conference 2018 (BMVC). 104. 
    • Julio Marco, Quercus Hernandez, Adolfo Munoz, Yue Dong, Adrian Jarabo, Min Kim, Xin Tong, and Diego Gutierrez. 2017. DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging. ACM Transactions on Graphics (SIGGRAPH Asia 2017) 36, 6 (2017). 
    • Matthew O’Toole, David B. Lindell, and Gordon Wetzstein. 2018. Confocal Non-Lineof-Sight Imaging Based on the Light-Cone Transform. Nature (2018). 
    • Andreas Velten, Thomas Willwacher, Otkrist Gupta, Ashok Veeraraghavan, Moungi G Bawendi, and Ramesh Raskar. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature communications 3 (2012), 745.



    This project has been funded by DARPA (project REVEAL), the European Research Council (ERC) under the EU’s Horizon 2020 research and innovation programme (project CHAMELEON, grant No 682080), the Spanish Ministry of Economy and Competitiveness (project TIN2016-78753-P), and the BBVA Foundation through a Leonardo grant.