“DEF: deep estimation of sharp geometric features in 3D shapes” by Matveev, Rakhimov, Artemov, Bobrovskikh, Egiazarian, et al. …

We propose Deep Estimators of Features (DEFs), a learning-based framework for predicting sharp geometric features in sampled 3D shapes. Differently from existing data-driven methods, which reduce this problem to feature classification, we propose to regress a scalar field representing the distance from point samples to the closest feature line on local patches. Our approach is the first that scales to massive point clouds by fusing distance-to-feature estimates obtained on individual patches.We extensively evaluate our approach against related state-of-the-art methods on newly proposed synthetic and real-world 3D CAD model benchmarks. Our approach not only outperforms these (with improvements in Recall and False Positives Rates), but generalizes to real-world scans after training our model on synthetic data and fine-tuning it on a small dataset of scanned data.We demonstrate a downstream application, where we reconstruct an explicit representation of straight and curved sharp feature lines from range scan data.We make code, pre-trained models, and our training and evaluation datasets available at https://github.com/artonson/def.

1. D. Bazazian, J. R. Casas, and J. Ruiz-Hidalgo. 2015. Fast and Robust Edge Extraction in Unorganized Point Clouds. In 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA). 1–8. Google ScholarCross Ref
2. D. Bazazian and ME. Parés. 2021. EDC-Net: Edge Detection Capsule Network for 3D Point Clouds. Applied Sciences 11, 4: 1833 (2021), 1–16. Google ScholarCross Ref
3. Yuanhao Cao, Liangliang Nan, and Peter Wonka. 2016. Curve networks for surface reconstruction. arXiv preprint arXiv:1603.08753 (2016).Google Scholar
4. Paolo Cignoni, Marco Callieri, Massimiliano Corsini, Matteo Dellepiane, Fabio Ganovelli, and Guido Ranzuglia. 2008. Meshlab: an open-source mesh processing tool.. In Eurographics Italian chapter conference, Vol. 2008. Salerno, Italy, 129–136.Google Scholar
5. Joel II Daniels, Linh K Ha, Tilo Ochotta, and Claudio T Silva. 2007. Robust smooth feature extraction from point clouds. In IEEE International Conference on Shape Modeling and Applications 2007 (SMI’07). IEEE, 123–136.Google ScholarDigital Library
6. Joel Daniels Ii, Tilo Ochotta, Linh K Ha, and Cláudio T Silva. 2008. Spline-based feature curves from point-sampled geometry. The Visual Computer 24, 6 (2008), 449–462.Google ScholarDigital Library
7. Kris Demarsin, Denis Vanderstraeten, Tim Volodine, and Dirk Roose. 2007. Detection of closed sharp edges in point clouds using normal estimation and graph theory. Computer-Aided Design 39, 4 (2007), 276–283.Google ScholarDigital Library
8. WA Falcon. 2019. PyTorch Lightning. GitHub. Note: https://github.com/PyTorchLightning/pytorch-lightning 3 (2019).Google Scholar
9. Shachar Fleishman, Daniel Cohen-Or, and Cláudio T Silva. 2005. Robust moving least-squares fitting with sharp features. ACM transactions on graphics (TOG) 24, 3 (2005), 544–552.Google Scholar
10. Adrien Gaidon, Qiao Wang, Yohann Cabon, and Eleonora Vig. 2016. Virtual worlds as proxy for multi-object tracking analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4340–4349.Google Scholar
11. T. Hackel, J. D. Wegner, and K. Schindler. 2016. Contour Detection in Unstructured 3D Point Clouds. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1610–1618. Google ScholarCross Ref
12. Timo Hackel, Jan D. Wegner, and Konrad Schindler. 2017. Joint classification and contour extraction of large 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 130 (2017), 231–245. Google ScholarCross Ref
13. Ankur Handa, Viorica Patraucean, Vijay Badrinarayanan, Simon Stent, and Roberto Cipolla. 2016. Understanding real world indoor scenes with synthetic data. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4077–4085.Google Scholar
14. JH Hannay and JF Nye. 2004. Fibonacci numerical integration on a sphere. Journal of Physics A: Mathematical and General 37, 48 (2004), 11591.Google ScholarCross Ref
15. Richard Hartley and Andrew Zisserman. 2004. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press. Google ScholarCross Ref
16. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.Google ScholarCross Ref
17. Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research 21, 248 (2020), 1–43.Google Scholar
18. Chems-Eddine Himeur, Thibault Lejemble, Thomas Pellegrini, Mathias Paulin, Loic Barthe, and Nicolas Mellado. 2021. PCEDNet: A Lightweight Neural Network for Fast and Interactive Edge Detection in 3D Point Clouds. ACM Transactions on Graphics (TOG) 41, 1 (2021), 1–21.Google ScholarDigital Library
19. Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Richard Zhang. 2013. Edge-aware point set resampling. ACM transactions on graphics (TOG) 32, 1 (2013), 9.Google Scholar
20. Peter J Huber et al. 1973. Robust regression: asymptotics, conjectures and Monte Carlo. The annals of statistics 1, 5 (1973), 799–821.Google Scholar
21. Ehsan Imani and Martha White. 2018. Improving Regression Performance with Distributional Losses (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, Stockholmsmässan, Stockholm Sweden, 2157–2166. http://proceedings.mlr.press/v80/imani18a.htmlGoogle Scholar
22. Tejas Khot, Shubham Agrawal, Shubham Tulsiani, Christoph Mertz, Simon Lucey, and Martial Hebert. 2019. Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency. arXiv:1905.02706 [cs.CV]Google Scholar
23. Sangpil Kim, Hyung-gun Chi, Xiao Hu, Qixing Huang, and Karthik Ramani. 2020. A Large-scale Annotated Mechanical Components Benchmark for Classification and Retrieval Tasks with Deep Neural Networks. In Proceedings of 16th European Conference on Computer Vision (ECCV).Google ScholarDigital Library
24. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
25. Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A big CAD model dataset for geometric deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9601–9611.Google ScholarCross Ref
26. Eric-Tuan Lê, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, and Niloy J Mitra. 2021. CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7457–7466.Google ScholarCross Ref
27. Kai Wah Lee and Pengbo Bo. 2016. Feature curve extraction from point clouds via developable strip intersection. Journal of Computational Design and Engineering 3, 2 (2016), 102–111. Google ScholarCross Ref
28. Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas J Guibas. 2019. Supervised fitting of geometric primitives to 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2652–2660.Google ScholarCross Ref
29. Y. Lin, C. Wang, B. Chen, D. Zai, and J. Li. 2017. Facet Segmentation-Based Line Segment Extraction for Large-Scale Point Clouds. IEEE Transactions on Geoscience and Remote Sensing 55, 9 (2017), 4839–4854. Google ScholarCross Ref
30. Yangbin Lin, Cheng Wang, Jun Cheng, Bili Chen, Fukai Jia, Zhonggui Chen, and Jonathan Li. 2015. Line segment extraction for large scale unorganized point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 102 (2015), 172–183. Google ScholarCross Ref
31. Yujia Liu, Stefano D’Aronco, Konrad Schindler, and Jan Dirk Wegner. 2021. PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds. CoRR abs/2103.02766 (2021). arXiv:2103.02766 https://arxiv.org/abs/2103.02766Google Scholar
32. Albert Matveev, Alexey Artemov, Denis Zorin, and Evgeny Burnaev. 2021. 3D Parametric Wireframe Extraction Based on Distance Fields. In 2021 4th International Conference on Artificial Intelligence and Pattern Recognition (Xiamen, China) (AIPR 2021). Association for Computing Machinery, New York, NY, USA, 316–322. Google ScholarDigital Library
33. Quentin Mérigot, Maks Ovsjanikov, and Leonidas J Guibas. 2010. Voronoi-based curvature and feature estimation from point clouds. IEEE Transactions on Visualization and Computer Graphics 17, 6 (2010), 743–756.Google ScholarDigital Library
34. Open CASCADE Technology OCCT 2021. Open CASCADE Technology OCCT. https://www.opencascade.com/. Accessed: 2021-06-01.Google Scholar
35. Parasolid: 3D Geometric Modeling Engine 2021. Parasolid: 3D Geometric Modeling Engine. https://www.plm.automation.siemens.com/global/en/products/plm-components/parasolid.html. Accessed: 2021-06-01.Google Scholar
36. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alche-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdfGoogle Scholar
37. Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. Pointnet++ : Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems. 5099–5108.Google Scholar
38. Prashant Raina, Sudhir Mudur, and Tiberiu Popa. 2019. Sharpness fields in point clouds using deep learning. Computers & Graphics 78 (2019), 37–53.Google ScholarCross Ref
39. Range Vision Spectrum 2021. RangeVision Spectrum – a new 3D high-resolution scanner. https://rangevision.com/en/products/spectrum/. Accessed: 2021-06-01.Google Scholar
40. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.Google ScholarCross Ref
41. Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Měch. 2020. Parsenet: A parametric surface fitting network for 3d point clouds. In European Conference on Computer Vision. Springer, 261–276.Google ScholarDigital Library
42. Maria-Laura Torrente, Silvia Biasotti, and Bianca Falcidieno. 2018. Recognition of feature curves on 3D shapes using an algebraic approach to Hough transforms. Pattern Recognition 73 (2018), 111–130.Google ScholarCross Ref
43. Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, and Hao Zhang. 2020. PIE-NET: Parametric Inference of Point Cloud Edges. Advances in Neural Information Processing Systems 33 (2020).Google Scholar
44. Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG) 38, 5 (2019), 1–12.Google ScholarDigital Library
45. Christopher Weber, Stefanie Hahmann, and Hans Hagen. 2010. Sharp feature detection in point clouds. In 2010 Shape Modeling International Conference. IEEE, 175–186.Google ScholarDigital Library
46. Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik. 2020. Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Reconstruction. arXiv preprint arXiv:2010.02392 (2020).Google Scholar
47. Nan Xue, Song Bai, Fudong Wang, Gui-Song Xia, Tianfu Wu, and Liangpei Zhang. 2019. Learning attraction field representation for robust line segment detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1595–1603.Google ScholarCross Ref
48. Omry Yadan. 2019. Hydra – A framework for elegantly configuring complex applications. Github. https://github.com/facebookresearch/hydraGoogle Scholar
49. Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018. EC-Net: an Edge-aware Point set Consolidation Network. In Proceedings of the European Conference on Computer Vision (ECCV). 386–402.Google ScholarDigital Library
50. Igor Zacharov, Rinat Arslanov, Maksim Gunin, Daniil Stefonishin, Andrey Bykov, Sergey Pavlov, Oleg Panarin, Anton Maliutin, Sergey Rykovanov, and Maxim Fedorov. 2019. “Zhores”-Petaflops supercomputer for data-driven modeling, machine learning and artificial intelligence installed in Skolkovo Institute of Science and Technology. Open Engineering 9, 1 (2019), 512–520.Google ScholarCross Ref