“Neural Progressive Meshes” by Chen, Kim, Aigerman and Jacobson
Conference:
Type:
Title:
- Neural Progressive Meshes
Session/Category Title: Deep Geometric Learning
Presenter(s)/Author(s):
Moderator(s):
Abstract:
The recent proliferation of 3D content that can be consumed on hand-held devices necessitates efficient tools for transmitting large geometric data, e.g., 3D meshes, over the Internet. Detailed high-resolution assets can pose a challenge to storage as well as transmission bandwidth, and level-of-detail techniques are often used to transmit an asset using an appropriate bandwidth budget. It is especially desirable for these methods to transmit data progressively, improving the quality of the geometry with more data. Our key insight is that the geometric details of 3D meshes often exhibit similar local patterns even across different shapes, and thus can be effectively represented with a shared learned generative space. We learn this space using a subdivision-based encoder-decoder architecture trained in advance on a large collection of surfaces. We further observe that additional residual features can be transmitted progressively between intermediate levels of subdivision that enable the client to control the tradeoff between bandwidth cost and quality of reconstruction, providing a neural progressive mesh representation. We evaluate our method on a diverse set of complex 3D shapes and demonstrate that it outperforms baselines in terms of compression ratio and reconstruction quality.
References:
1. Pierre Alliez and Mathieu Desbrun. 2001. Valence-driven connectivity encoding for 3D meshes. In Computer graphics forum.
2. Pierre Baldi. 2012. Autoencoders, unsupervised learning, and deep architectures. In ICMLW.
3. Yoshua Bengio 2009. Learning deep architectures for AI. Foundations and trends in Machine Learning (2009).
4. Edwin Catmull and James Clark. 1978. Recursively generated B-spline surfaces on arbitrary topological meshes. Computer-aided design (1978).
5. Zhiqin Chen and Hao Zhang. 2019. Learning implicit fields for generative shape modeling. In CVPR.
6. Michael Deering. 1995. Geometry compression. In Conference on Computer graphics and interactive techniques.
7. Michael Garland and Paul S Heckbert. 1997. Surface simplification using quadric error metrics. In Computer graphics and interactive techniques.
8. Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018a. 3d-coded: 3d correspondences by deep deformation. In ECCV.
9. Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018b. A papier-mâché approach to learning 3d surface generation. In CVPR.
10. Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. Meshcnn: a network with an edge. ACM TOG (2019).
11. Amir Hertz, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. 2020. Deep Geometric Texture Synthesis. ACM TOG (2020).
12. Hugues Hoppe. 1996. Progressive meshes. In Conference on Computer graphics and interactive techniques.
13. Hugues Hoppe, Tony DeRose, Tom Duchamp, Mark Halstead, Hubert Jin, John McDonald, Jean Schweitzer, and Werner Stuetzle. 1994. Piecewise smooth surface reconstruction. In Annual conference on Computer graphics and interactive techniques.
14. Shi-Min Hu, Zheng-Ning Liu, Meng-Hao Guo, Jun-Xiong Cai, Jiahui Huang, Tai-Jiang Mu, and Ralph R Martin. 2022. Subdivision-based mesh convolution networks. TOG (2022).
15. Yixin Hu, Qingnan Zhou, Xifeng Gao, Alec Jacobson, Denis Zorin, and Daniele Panozzo. 2018. Tetrahedral meshing in the wild.ACM TOG (2018).
16. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML.
17. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In ICLR.
18. Thibault Lescoat, Hsueh-Ti Derek Liu, Jean-Marc Thiery, Alec Jacobson, Tamy Boubekeur, and Maks Ovsjanikov. 2020. Spectral mesh simplification. In Computer Graphics Forum.
19. Hsueh-Ti Derek Liu, Vladimir G Kim, Siddhartha Chaudhuri, Noam Aigerman, and Alec Jacobson. 2020. Neural subdivision. ACM TOG (2020).
20. Charles Loop. 1987. Smooth subdivision surfaces based on triangles. Master’s thesis, University of Utah, Department of Mathematics (1987).
21. Julien NP Martel, David B Lindell, Connor Z Lin, Eric R Chan, Marco Monteiro, and Gordon Wetzstein. 2021. Acorn: Adaptive coordinate networks for neural scene representation. ACM TOG (2021).
22. John McCormac, Ankur Handa, Andrew Davison, and Stefan Leutenegger. 2017. Semanticfusion: Dense 3d semantic mapping with convolutional neural networks. In ICRA.
23. Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019. Occupancy networks: Learning 3d reconstruction in function space. In CVPR.
24. Thomas W Mitchel, Vladimir G Kim, and Michael Kazhdan. 2021. Field convolutions for surface CNNs. In ICCV.
25. Luca Morreale, Noam Aigerman, Paul Guerrero, Vladimir G. Kim, and Niloy Mitra. 2022. Neural Convolutional Surfaces. In CVPR.
26. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML.
27. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR.
28. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS.
29. Rolandos Alexandros Potamias, Stylianos Ploumpis, and Stefanos Zafeiriou. 2022. Neural mesh simplification. In CVPR.
30. Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR.
31. Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In NeurIPS.
32. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In MICCAI.
33. Jarek Rossignac. 1999. Edgebreaker: Connectivity compression for triangle meshes. TVCG (1999).
34. Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. 2015. Multi-view convolutional neural networks for 3d shape recognition. In ICCV.
35. Vitaly Surazhsky and Craig Gotsman. 2003. Explicit surface remeshing. In SGP.
36. Andrzej Szymczak, Davis King, and Jarek Rossignac. 2001. An Edgebreaker-based efficient compression scheme for regular meshes. Computational Geometry (2001).
37. Andrzej Szymczak, Jarek Rossignac, and Davis King. 2002. Piecewise regular meshes: Construction and compression. Graphical Models (2002).
38. Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, and Sanja Fidler. 2021. Neural geometric level of detail: Real-time rendering with implicit 3D shapes. In CVPR.
39. Gabriel Taubin and Jarek Rossignac. 1998. Geometric compression through topological surgery. ACM TOG (1998).
40. Costa Touma and Craig Gotsman. 1998. Triangle mesh compression. In Proceedings-Graphics Interface.
41. Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks. In ICCV.
42. Qingnan Zhou and Alec Jacobson. 2016. Thingi10k: A dataset of 10,000 3d-printing models. In SGP.
43. Denis Zorin, Peter Schröder, and Wim Sweldens. 1996. Interpolating subdivision for meshes with arbitrary topology. In Computer graphics and interactive techniques.
