“IR Surface Reflectance Estimation and Material Type Recognition using Two-stream Net and Kinect Camera” by Lee, Lim, Ahn and Lee

  • ©SeokYeong Lee, HwaSup Lim, SangChul Ahn, and Seungkyu Lee

  • ©SeokYeong Lee, HwaSup Lim, SangChul Ahn, and Seungkyu Lee

Conference:


Type:


Entry Number: 43

Title:

    IR Surface Reflectance Estimation and Material Type Recognition using Two-stream Net and Kinect Camera

Presenter(s)/Author(s):



Abstract:


    Recently, material type recognition using color or light field camera has been studied. However, visual pattern based approaches for material type recognition without direct acquisition of surface reflectance show limited performance. In this work, we propose IR surface reflectance estimation using off-the-shelf ToF (Time-of- Flight) active sensor such as Kinect and perform surface material type recognition based on both color and reflectance clues. Two stream deep neural network consists of convolutional neural network encoding visual clue and recurrent neural network encoding reflectance characteristic is proposed for material classification. Estimated IR surface reflectance and material type recognition evaluation on our Color-IR Material Data set show promising performance compared to prior approaches.

References:


    • Sean Bell, Paul Upchurch, Noah Snavely, and Kavita Bala. 2015. Material Recognition in the Wild With the Materials in Context Database. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015). 
    • Victor Campos, Brendan Jou, Xavier Giró i Nieto, Jordi Torres, and Shih-Fu Chang. 2017. Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks. CoRR abs/1708.06834 (2017). arXiv:1708.06834 http://arxiv.org/abs/1708.06834
    •  Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. 2016. Densely Connected Convolutional Networks. CoRR abs/1608.06993 (2016). arXiv:1608.06993 
    • J. Kim, H. Lim, S. C. Ahn, and S. Lee. 2018. RGBD Camera Based Material Recognition via Surface Roughness Estimation. (March 2018), 1963–1971. https://doi.org/10. 1109/WACV.2018.00217 
    • Ting-Chun Wang, Jun-Yan Zhu, Hiroaki Ebi, Manmohan Chandraker, Alexei A. Efros, and Ravi Ramamoorthi. 2016. A 4D Light-Field Dataset and CNN Architectures for Material Recognition. CoRR abs/1608.06985 (2016). arXiv:1608.06985 
    • Hang Zhang, Jia Xue, and Kristin Dana. 2017. Deep TEN: Texture Encoding Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Keyword(s):



Acknowledgements:


    This work was supported by the Global Frontier RD Program on Human-centered Interaction for Coexistence funded by the National Research Foundation of Korea grant funded by the Korean Government (MSIT) (2010-0029752)


PDF:



Overview Page: