“Full 3D reconstruction of transparent objects” by Wu, Zhou, Lin, Gong and Huang
Conference:
Type:
Entry Number: 103
Title:
- Full 3D reconstruction of transparent objects
Session/Category Title: 3D Capture
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Numerous techniques have been proposed for reconstructing 3D models for opaque objects in past decades. However, none of them can be directly applied to transparent objects. This paper presents a fully automatic approach for reconstructing complete 3D shapes of transparent objects. Through positioning an object on a turntable, its silhouettes and light refraction paths under different viewing directions are captured. Then, starting from an initial rough model generated from space carving, our algorithm progressively optimizes the model under three constraints: surface and refraction normal consistency, surface projection and silhouette consistency, and surface smoothness. Experimental results on both synthetic and real objects demonstrate that our method can successfully recover the complex shapes of transparent objects and faithfully reproduce their light refraction properties.
References:
1. Kfir Aberman, Oren Katzir, Qiang Zhou, Zegang Luo, Andrei Sharf, Chen Greif, Baoquan Chen, and Daniel Cohen-Or. 2017. Dip Transform for 3D Shape Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH) 36, 4 (2017), 79:1–79:11. Google ScholarDigital Library
2. Bradley Atcheson, Ivo Ihrke, Wolfgang Heidrich, Art Tevs, Derek Bradley, Marcus Magnor, and Hans-Peter Seidel. 2008. Time-resolved 3D Capture of Non-stationary Gas Flows. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 27, 5 (2008), 132:1–132:9. Google ScholarDigital Library
3. Moshe Ben-Ezra and Shree K. Nayar. 2003. What Does Motion Reveal About Transparency? Proc. Int. Conf. on Computer Vision (2003), 1025–1032. Google ScholarDigital Library
4. Matthew Berger, Joshua A. Levine, Luis Gustavo Nonato, Gabriel Taubin, and Claudio T. Silva. 2013. A Benchmark for Surface Reconstruction. ACM Trans. on Graphics 32, 2 (2013), 20:1–20:17. Google ScholarDigital Library
5. Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Joshua A. Levine, Andrei Sharf, and Claudio Silva. 2014. State of the Art in Surface Reconstruction from Point Clouds. Eurographics STAR (2014), 165–185.Google Scholar
6. Max Born and Emil Wolf. 2013. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier.Google Scholar
7. Tongbo Chen, Hendrik PA Lensch, Christian Fuchs, and Hans-Peter Seidel. 2007. Polarization and Phase-Shifting for 3D Scanning of Translucent Objects. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2007), 1–8.Google ScholarCross Ref
8. Yung-Yu Chuang, Douglas E. Zongker, Joel Hindorff, Brian Curless, David H. Salesin, and Richard Szeliski. 2000. Environment Matting Extensions: Towards Higher Accuracy and Real-time Capture. ACM Trans. on Graphics (Proc. of SIGGRAPH) (2000), 121–130. Google ScholarDigital Library
9. Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. 2012. Efficient and Flexible Sampling with Blue Noise Properties of Triangular Meshes. IEEE Trans. Visualization & Computer Graphics 18, 6 (2012), 914–924. Google ScholarDigital Library
10. Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, and Jan Kautz. 2017. Polarimetric Multi-View Stereo. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 1558–1567.Google ScholarCross Ref
11. Qi Duan, Jianfei Cai, and Jianmin Zheng. 2015. Compressive Environment Matting. The Visual Computer 31, 12 (2015), 1587–1600. Google ScholarDigital Library
12. Silvano Galliani, Katrin Lasinger, and Konrad Schindler. 2015. Massively Parallel Multiview Stereopsis by Surface Normal Diffusion. Proc. Int. Conf. on Computer Vision (2015), 873–881. Google ScholarDigital Library
13. James Gregson, Michael Krimerman, Matthias B. Hullin, and Wolfgang Heidrich. 2012. Stochastic Tomography and Its Applications in 3D Imaging of Mixing Fluids. ACM Trans. on Graphics (Proc. of SIGGRAPH) 31, 4 (2012), 52:1–52:10. Google ScholarDigital Library
14. Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of Unorganized Point Clouds for Surface Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 28, 5 (2009), 176:1–176:7. Google ScholarDigital Library
15. Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang. 2013. Edge-aware Point Set Resampling. ACM Trans. on Graphics 32, 1 (2013), 9:1–9:12. Google ScholarDigital Library
16. Matthias B. Hullin, Martin Fuchs, Ivo Ihrke, Hans-Peter Seidel, and Hendrik P. A. Lensch. 2008. Fluorescent Immersion Range Scanning. ACM Trans. on Graphics (Proc. of SIGGRAPH) 27, 3 (2008), 87:1–87:10. Google ScholarDigital Library
17. Cong Phuoc Huynh, Antonio Robles-Kelly, and Edwin Hancock. 2010. Shape and refractive index recovery from single-view polarisation images. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2010), 1229–1236.Google ScholarCross Ref
18. Ivo Ihrke, Kiriakos N. Kutulakos, Hendrik Lensch, Marcus Magnor, and Wolfgang Heidrich. 2010. Transparent and specular object reconstruction. Computer Graphics Forum 29, 8 (2010), 2400–2426.Google ScholarCross Ref
19. Ivo Ihrke and Marcus Magnor. 2004. Image-based Tomographic Reconstruction of Flames. Proc. Eurographics Symp. on Computer Animation (2004), 365–373. Google ScholarDigital Library
20. Julian Iseringhausen, Bastian Goldlücke, Nina Pesheva, Stanimir Iliev, Alexander Wender, Martin Fuchs, and Matthias B. Hullin. 2017. 4D Imaging Through Spray-on Optics. ACM Trans. on Graphics (Proc. of SIGGRAPH) 36, 4 (2017), 35:1–35:11. Google ScholarDigital Library
21. Yu Ji, Jinwei Ye, and Jingyi Yu. 2013. Reconstructing gas flows using light-path approximation. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2013), 2507–2514. Google ScholarDigital Library
22. Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson Surface Reconstruction. ACM Trans. on Graphics 32, 3 (2013), 29:1–29:13. Google ScholarDigital Library
23. Jaewon Kim, Ilya Reshetouski, and Abhijeet Ghosh. 2017. Acquiring Axially-Symmetric Transparent Objects Using Single-View Transmission Imaging. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 1484–1492.Google ScholarCross Ref
24. Kiriakos N. Kutulakos and Steven M. Seitz. 2000. A Theory of Shape by Space Carving. Int. J. Computer Vision 38, 3 (2000), 199–218. Google ScholarDigital Library
25. Kiriakos N. Kutulakos and Eron Steger. 2008. A theory of refractive and specular 3D shape by light-path triangulation. Int. J. Computer Vision 76, 1 (2008), 13–29. Google ScholarDigital Library
26. Aviad Levis, Yoav Y Schechner, Amit Aides, and Anthony B Davis. 2015. Airborne three-dimensional cloud tomography. Proc. Int. Conf. on Computer Vision (2015), 3379–3387. Google ScholarDigital Library
27. Aviad Levis, Yoav Y Schechner, and Anthony B Davis. 2017. Multiple-scattering microphysics tomography. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 5797–5806.Google ScholarCross Ref
28. Yaron Lipman, Daniel Cohen-Or, David Levin, and Hillel Tal-Ezer. 2007. Parameterization-free Projection for Geometry Reconstruction. ACM Trans. on Graphics (Proc. of SIGGRAPH) 26, 3 (2007), 22:1–22:6. Google ScholarDigital Library
29. Daisuke Miyazaki and Katsushi Ikeuchi. 2005. Inverse polarization raytracing: estimating surface shapes of transparent objects. Proc. IEEE Conf. on Computer Vision & Pattern Recognition 2 (2005), 910–917. Google ScholarDigital Library
30. Nigel JW Morris and Kiriakos N. Kutulakos. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter-trace photography. Proc. Int. Conf. on Computer Vision (2007), 1–8.Google Scholar
31. Nigel JW Morris and Kiriakos N. Kutulakos. 2011. Dynamic refraction stereo. IEEE Trans. Pattern Analysis & Machine Intelligence 33, 8 (2011), 1518–1531. Google ScholarDigital Library
32. Pieter Peers and Philip Dutré. 2003. Wavelet environment matting. Proc. Eurographics Workshop on Rendering (2003), 157–166. Google ScholarDigital Library
33. Yiming Qian, Minglun Gong, and Yee Hong Yang. 2016. 3D Reconstruction of Transparent Objects with Position-Normal Consistency. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2016), 4369–4377.Google ScholarCross Ref
34. Yiming Qian, Minglun Gong, and Yee-Hong Yang. 2015. Frequency-based environment matting by compressive sensing. Proc. Int. Conf. on Computer Vision (2015), 3532–3540. Google ScholarDigital Library
35. Yiming Qian, Minglun Gong, and Yee-Hong Yang. 2017. Stereo-Based 3D Reconstruction of Dynamic Fluid Surfaces by Global Optimization. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2017), 6650–6659.Google ScholarCross Ref
36. Yuliy Schwartzburg, Romain Testuz, Andrea Tagliasacchi, and Mark Pauly. 2014. High-contrast Computational Caustic Design. ACM Trans. on Graphics (Proc. of SIGGRAPH) 33, 4 (2014), 74:1–74:11. Google ScholarDigital Library
37. Qi Shan, Sameer Agarwal, and Brian Curless. 2012. Refractive height fields from single and multiple images. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2012), 286–293. Google ScholarDigital Library
38. Kenichiro Tanaka, Yasuhiro Mukaigawa, Hiroyuki Kubo, Yasuyuki Matsushita, and Yasushi Yagi. 2016. Recovering Transparent Shape from Time-of-Flight Distortion. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2016), 4387–4395.Google ScholarCross Ref
39. Borislav Trifonov, Derek Bradley, and Wolfgang Heidrich. 2006. Tomographic reconstruction of transparent objects. Proc. Eurographics Conf. on Rendering Techniques (2006), 51–60. Google ScholarDigital Library
40. Chia-Yin Tsai, Ashok Veeraraghavan, and Aswin C Sankaranarayanan. 2015. What does a single light-ray reveal about a transparent object? Proc. IEEE Int. Conf. on Image Processing (2015), 606–610.Google ScholarDigital Library
41. Gordon Wetzstein, David Roodnick, Wolfgang Heidrich, and Ramesh Raskar. 2011. Refractive shape from light field distortion. Proc. Int. Conf. on Computer Vision (2011), 1180–1186. Google ScholarDigital Library
42. Yonatan Wexler, Andrew. W. Fitzgibbon, and Andrew. Zisserman. 2002. Image-based Environment Matting. Proc. Eurographics Workshop on Rendering (2002), 279–290. Google ScholarDigital Library
43. Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. 2015a. Deep Points Consolidation. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 34, 6 (2015), 176:1–176:13. Google ScholarDigital Library
44. Shihao Wu, Wei Sun, Pinxin Long, Hui Huang, Daniel Cohen-Or, Minglun Gong, Oliver Deussen, and Baoquan Chen. 2014. Quality-driven Poisson-guided Autoscanning. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 33, 6 (2014), 203:1–203:12. Google ScholarDigital Library
45. Zhaohui Wu, Zhong Zhou, Delei Tian, and Wei Wu. 2015b. Reconstruction of Three-dimensional Flame with Color Temperature. The Visual Computer 31, 5 (2015), 613–625. Google ScholarDigital Library
46. Sai-Kit Yeung, Tai-Pang Wu, Chi-Keung Tang, Tony F Chan, and Stanley Osher. 2011. Adequate reconstruction of transparent objects on a shoestring budget. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2011), 2513–2520. Google ScholarDigital Library
47. Yonghao Yue, Kei Iwasaki, Bing-Yu Chen, Yoshinori Dobashi, and Tomoyuki Nishita. 2014. Poisson-Based Continuous Surface Generation for Goal-Based Caustics. ACM Trans. on Graphics 33, 3 (2014), 31:1–31:7. Google ScholarDigital Library
48. Mingjie Zhang, Xing Lin, Mohit Gupta, Jinli Suo, and Qionghai Dai. 2014. Recovering Scene Geometry under Wavy Fluid via Distortion and Defocus Analysis. Proc. Euro. Conf. on Computer Vision (2014), 234–250.Google ScholarCross Ref
49. Zhengyou Zhang. 2000. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Analysis & Machine Intelligence 22, 11 (2000), 1330–1334. Google ScholarDigital Library
50. Ciyou Zhu, Richard H. Byrd, Peihuang Lu, and Jorge Nocedal. 1997. Algorithm 778: L-BFGS-B: Fortran Subroutines for Large-scale Bound-constrained Optimization. ACM Trans. Mathematical Software 23, 4 (1997), 550–560. Google ScholarDigital Library
51. Douglas E. Zongker, Dawn M. Werner, Brian Curless, and David H. Salesin. 1999. Environment Matting and Compositing. ACM Trans. on Graphics (Proc. of SIGGRAPH) (1999), 205–214. Google ScholarDigital Library
52. Xinxin Zuo, Chao Du, Sen Wang, Jiangbin Zheng, and Ruigang Yang. 2015. Interactive visual hull refinement for specular and transparent object surface reconstruction. Proc. Int. Conf. on Computer Vision (2015), 2237–2245. Google ScholarDigital Library
