“Neural dual contouring” by Chen, Tagliasacchi, Funkhouser and Zhang

We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC). Like traditional DC, it produces exactly one vertex per grid cell and one quad for each grid edge intersection, a natural and efficient structure for reproducing sharp features. However, rather than computing vertex locations and edge crossings with hand-crafted functions that depend directly on difficult-to-obtain surface gradients, NDC uses a neural network to predict them. As a result, NDC can be trained to produce meshes from signed or unsigned distance fields, binary voxel grids, or point clouds (with or without normals); and it can produce open surfaces in cases where the input represents a sheet or partial surface. During experiments with five prominent datasets, we find that NDC, when trained on one of the datasets, generalizes well to the others. Furthermore, NDC provides better surface reconstruction accuracy, feature preservation, output complexity, triangle quality, and inference time in comparison to previous learned (e.g., neural marching cubes, convolutional occupancy networks) and traditional (e.g., Poisson) methods. Code and data are available at https://github.com/czq142857/NDC.

1. Nina Amenta, Marshall Bern, and Manolis Kamvysselis. 1998. A New Voronoi-Based Surface Reconstruction Algorithm. In SIGGRAPH. 415–421.Google Scholar
2. Matan Atzmon and Yaron Lipman. 2020. SAL: Sign Agnostic Learning of Shapes From Raw Data. In CVPR. 2562–2571.Google Scholar
3. Abhishek Badki, Orazio Gallo, Jan Kautz, and Pradeep Sen. 2020. Meshlet priors for 3D mesh reconstruction. In CVPR. 2849–2858.Google Scholar
4. Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Gaël Guennebaud, Joshua A. Levine, Andrei Sharf, and Claudio T Silva. 2017. A Survey of Surface Reconstruction from Point Clouds. In Computer Graphics Forum, Vol. 36. 301–329.Google ScholarDigital Library
5. Fausto Bernardini, Joshua Mittleman, Holly Rushmeier, Cláudio Silva, and Gabriel Taubin. 1999. The ball-pivoting algorithm for surface reconstruction. IEEE TVCG 5, 4 (1999), 349–359.Google Scholar
6. Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. 2019. Multi-Garment Net: Learning to Dress 3D People from Images. In ICCV. 5420–5430.Google Scholar
7. Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR. 3794–3801.Google Scholar
8. Alexandre Boulch and Renaud Marlet. 2012. Fast and Robust Normal Estimation for Point Clouds with Sharp Features. Computer Graphics Forum 31, 5 (2012), 1765–1774.Google ScholarDigital Library
9. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D Data in Indoor Environments. In 3DV. 667–676.Google Scholar
10. Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. arXiv preprint arXiv:1512.03012 (2015).Google Scholar
11. Zhiqin Chen and Hao Zhang. 2019. Learning Implicit Fields for Generative Shape Modeling. In CVPR. 5932–5941.Google Scholar
12. Zhiqin Chen and Hao Zhang. 2021. Neural Marching Cubes. ACM Transactions on Graphics 40, 6 (2021), 1–15.Google ScholarDigital Library
13. Evgeni Chernyaev. 1995. Marching Cubes 33: construction of topologically correct isosurfaces. Technical Report CN/95-17. CERN.Google Scholar
14. David Cohen-Steiner and Tran Kai Frank Da. 2004. A greedy Delaunay-based surface reconstruction algorithm. The Visual Computer 20, 1 (2004), 4–16.Google ScholarDigital Library
15. Brian Curless and Marc Levoy. 1996. A volumetric method for building complex models from range images. In SIGGRAPH. 303–312.Google Scholar
16. Bruno Rodrigues De Araújo, Daniel S. Lopes, Pauline Jepp, Joaquim A. Jorge, and Brian Wyvill. 2015. A Survey on Implicit Surface Polygonization. ACM Computing Surveys (CSUR) 47, 4 (2015), 1–39.Google ScholarDigital Library
17. Thomas Funkhouser, Michael Kazhdan, Philip Shilane, Patrick Min, William Kiefer, Ayellet Tal, Szymon Rusinkiewicz, and David Dobkin. 2004. Modeling by example. ACM Transactions on Graphics 23, 3 (2004), 652–663.Google ScholarDigital Library
18. Jun Gao, Wenzheng Chen, Tommy Xiang, Alec Jacobson, Morgan McGuire, and Sanja Fidler. 2020. Learning deformable tetrahedral meshes for 3d reconstruction. In NeurIPS, Vol. 33. 9936–9947.Google Scholar
19. Michael Garland and Paul S. Heckbert. 1997. Surface simplification using quadric error metrics. In SIGGRAPH. 209–216.Google Scholar
20. Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, and Mathieu Aubry. 2018. A papier-mâché approach to learning 3D surface generation. In CVPR. 216–224.Google Scholar
21. Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. 2020. Point2Mesh: A Self-Prior for Deformable Meshes. ACM Transactions on Graphics 39, 4 (2020).Google ScholarDigital Library
22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770–778.Google Scholar
23. Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, and Thomas Funkhouser. 2020. Local implicit grid representations for 3d scenes. In CVPR. 6001–6010.Google Scholar
24. Tao Ju, Frank Losasso, Scott Schaefer, and Joe Warren. 2002. Dual Contouring of Hermite Data. ACM Transactions on graphics 21, 3 (2002), 339–346.Google Scholar
25. Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson surface reconstruction. ACM Transactions on Graphics 32, 3 (2013), 1–13.Google ScholarDigital Library
26. Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: a big CAD model dataset for geometric deep learning. In CVPR. 9601–9611.Google Scholar
27. Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. 2018. Noise2Noise: Learning Image Restoration without Clean Data. In ICML. 2965–2974.Google Scholar
28. Thomas Lewiner, Hélio Lopes, Antônio Wilson Vieira, and Geovan Tavares. 2003. Efficient implementation of marching cubes’ cases with topological guarantees. Journal of Graphics Tools 8, 2 (2003), 1–15.Google ScholarCross Ref
29. Ruosi Li, Lu Liu, Ly Phan, Sasakthi Abeysinghe, Cindy Grimm, and Tao Ju. 2010. Polygonizing extremal surfaces with manifold guarantees. In Proceedings of ACM Symposium on Solid and Physical Modeling. 189–194.Google ScholarDigital Library
30. Yiyi Liao, Simon Donne, and Andreas Geiger. 2018. Deep marching cubes: Learning explicit surface representations. In CVPR. 2916–2925.Google Scholar
31. Minghua Liu, Xiaoshuai Zhang, and Hao Su. 2020. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. In ECCV. 68–84.Google Scholar
32. William E. Lorensen and Harvey E. Cline. 1987. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. In SIGGRAPH. 163–169.Google Scholar
33. Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019. Occupancy Networks: Learning 3D Reconstruction in Function Space. In CVPR. 4455–4465.Google Scholar
34. Zhenxing Mi, Yiming Luo, and Wenbing Tao. 2020. SSRNet: Scalable 3D Surface Reconstruction Network. In CVPR. 970–979.Google Scholar
35. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV. 405–421.Google Scholar
36. Gregory M. Nielson. 2004. Dual Marching Cubes. In IEEE Visualization. 489–496.Google Scholar
37. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Love-grove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In CVPR. 165–174.Google Scholar
38. Mark Pauly, Niloy J. Mitra, Joachim Giesen, Markus H. Gross, and Leonidas J. Guibas. 2005. Example-based 3D scan completion. In Symp. on Geometry Processing. 23–32.Google Scholar
39. Songyou Peng, Chiyu Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger. 2021. Shape as points: A differentiable Poisson solver. In NeurIPS, Vol. 34.Google Scholar
40. Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. 2020. Convolutional occupancy networks. In ECCV. 523–540.Google Scholar
41. Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS, Vol. 30. 5105–5114.Google Scholar
42. Marie-Julie Rakotosaona, Paul Guerrero, Noam Aigerman, Niloy Mitra, and Maks Ovsjanikov. 2021. Learning Delaunay Surface Elements for Mesh Reconstruction. In CVPR. 22–31.Google Scholar
43. Edoardo Remelli, Artem Lukoianov, Stephan R. Richter, Benoît Guillard, Timur Bagautdinov, Pierre Baque, and Pascal Fua. 2020. MeshSDF: differentiable iso-surface extraction. In NeurIPS, Vol. 33. 22468–22478.Google Scholar
44. Scott Schaefer, Tao Ju, and Joe Warren. 2007. Manifold dual contouring. IEEE TVCG 13, 3 (2007), 610–619.Google Scholar
45. Ruwen Schnabel, Patrick Degener, and Reinhard Klein. 2009. Completion and Reconstruction with Primitive Shapes. Computer Graphics Forum 28 (2009), 503–512.Google ScholarCross Ref
46. Nicholas Sharp and Maks Ovsjanikov. 2020. PointTriNet: Learned Triangulation of 3D Point Sets. In ECCV. 762–778.Google Scholar
47. Chao-Hui Shen, Hongbo Fu, Kang Chen, and Shi-Min Hu. 2012. Structure recovery by part assembly. ACM Transactions on Graphics 31, 6 (2012), 1–11.Google ScholarDigital Library
48. Vincent Sitzmann, Julien NP Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. 2020. Implicit neural representations with periodic activation functions. In NeurIPS, Vol. 33. 7462–7473.Google Scholar
49. Jiapeng Tang, Jiabao Lei, Dan Xu, Feiying Ma, Kui Jia, and Lei Zhang. 2021. SAConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks. In ICCV. 6484–6493.Google Scholar
50. Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2019. Deep geometric prior for surface reconstruction. In CVPR. 10130–10139.Google Scholar
51. Geoff Wyvill, Craig McPheeters, and Brian Wyvill. 1986. Data Structure for Soft Objects. The Visual Computer 2 (1986), 227–234.Google ScholarCross Ref
52. Qingnan Zhou and Alec Jacobson. 2016. Thingi10K: a dataset of 10,000 3D-printing models. arXiv preprint arXiv:1605.04797 (2016).Google Scholar