“Orienting point clouds with dipole propagation” by Metzer, Hanocka, Zorin, Giryes, Panozzo, et al. …

Establishing a consistent normal orientation for point clouds is a notoriously difficult problem in geometry processing, requiring attention to both local and global shape characteristics. The normal direction of a point is a function of the local surface neighborhood; yet, point clouds do not disclose the full underlying surface structure. Even assuming known geodesic proximity, calculating a consistent normal orientation requires the global context. In this work, we introduce a novel approach for establishing a globally consistent normal orientation for point clouds. Our solution separates the local and global components into two different sub-problems. In the local phase, we train a neural network to learn a coherent normal direction per patch (i.e., consistently oriented normals within a single patch). In the global phase, we propagate the orientation across all coherent patches using a dipole propagation. Our dipole propagation decides to orient each patch using the electric field defined by all previously orientated patches. This gives rise to a global propagation that is stable, as well as being robust to nearby surfaces, holes, sharp features and noise.

1. Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. 2018. Learning representations and generative models for 3d point clouds. In International conference on machine learning. PMLR, 40–49.Google Scholar
2. Gavin Barill, Neil Dickson, Ryan Schmidt, David I.W. Levin, and Alec Jacobson. 2018. Fast Winding Numbers for Soups and Clouds. ACM Transactions on Graphics (2018).Google Scholar
3. Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. 2020. Learning Gradient Fields for Shape Generation. In Proceedings of the European Conference on Computer Vision (ECCV).Google ScholarDigital Library
4. Frédéric Cazals and Marc Pouget. 2005. Estimating differential quantities using polynomial fitting of osculating jets. Computer Aided Geometric Design 22, 2 (2005), 121–146.Google ScholarDigital Library
5. Yi-Ling Chen, Bing-Yu Chen, Shang-Hong Lai, and Tomoyuki Nishita. 2010. Binary orientation trees for volume and surface reconstruction from unoriented point clouds. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 2011–2019.Google Scholar
6. Haoqiang Fan, Hao Su, and Leonidas J Guibas. 2017. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition. 605–613.Google ScholarCross Ref
7. Gerald B Folland. 1995. Introduction to partial differential equations. Vol. 102. Princeton university press.Google Scholar
8. Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018. A papier-mâché approach to learning 3d surface generation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 216–224.Google ScholarCross Ref
9. Gaël Guennebaud and Markus Gross. 2007. Algebraic point set surfaces. In ACM SIGGRAPH 2007 papers. 23–es.Google ScholarDigital Library
10. Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, and Niloy J Mitra. 2018. Pcpnet learning local shape properties from raw point clouds. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 75–85.Google Scholar
11. Swaminathan Gurumurthy and Shubham Agrawal. 2019. High fidelity semantic shape completion for point clouds using latent optimization. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1099–1108.Google ScholarCross Ref
12. Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. 2019. MeshCNN: A Network with an Edge. ACM Trans. Graph. 38, 4, Article 90 (July 2019), 12 pages. Google ScholarDigital Library
13. Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. 2020. Point2Mesh: A Self-Prior for Deformable Meshes. ACM Trans. Graph. 39, 4, Article 126 (July 2020), 12 pages. Google ScholarDigital Library
14. Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. 2019. Total Denoising: Unsupervised learning of 3D point cloud cleaning. In Proceedings of the IEEE International Conference on Computer Vision. 52–60.Google Scholar
15. Amir Hertz, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. 2020. PointGMM: a Neural GMM Network for Point Clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12054–12063.Google ScholarCross Ref
16. Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques. 71–78.Google ScholarDigital Library
17. Chao Huang, Ruihui Li, Xianzhi Li, and Chi-Wing Fu. 2020. Non-Local Part-Aware Point Cloud Denoising. arXiv preprint arXiv:2003.06631 (2020).Google Scholar
18. Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of unorganized point clouds for surface reconstruction. ACM transactions on graphics (TOG) 28, 5 (2009), 1–7.Google Scholar
19. Zhiyang Huang, Nathan Carr, and Tao Ju. 2019. Variational implicit point set surfaces. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1–13.Google ScholarDigital Library
20. Johannes Jakob, Christoph Buchenau, and Michael Guthe. 2019. Parallel globally consistent normal orientation of raw unorganized point clouds. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 163–173.Google Scholar
21. Li Jiang, Shaoshuai Shi, Xiaojuan Qi, and Jiaya Jia. 2018. Gal: Geometric adversarial loss for single-view 3d-object reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV). 802–816.Google ScholarDigital Library
22. Sagi Katz, Ayellet Tal, and Ronen Basri. 2007. Direct visibility of point sets. In ACM SIGGRAPH 2007 papers. 24–es.Google ScholarDigital Library
23. Michael Kazhdan. 2005. Reconstruction of solid models from oriented point sets. In Proceedings of the third Eurographics symposium on Geometry processing. 73–es.Google ScholarDigital Library
24. Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, Vol. 7.Google ScholarDigital Library
25. Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Transactions on Graphics (ToG) 32, 3 (2013), 1–13.Google ScholarDigital Library
26. S. König and S. Gumhold. 2009. Consistent Propagation of Normal Orientations in Point Clouds. In VMV.Google Scholar
27. Itai Lang, Uriel Kotlicki, and Shai Avidan. 2020. Geometric Adversarial Attacks and Defenses on 3D Point Clouds. arXiv preprint arXiv:2012.05657 (2020).Google Scholar
28. Oliver Laric. 2012. Three D Scans. http://threedscans.com/info/ (2012).Google Scholar
29. David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. Springer, 37–49.Google Scholar
30. Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. 2018c. Point cloud gan. arXiv preprint arXiv:1810.05795 (2018).Google Scholar
31. Jiaxin Li, Ben M Chen, and Gim Hee Lee. 2018b. So-net: Self-organizing network for point cloud analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9397–9406.Google ScholarCross Ref
32. Ruihui Li, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2019. Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE International Conference on Computer Vision. 7203–7212.Google ScholarCross Ref
33. Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. 2018a. Pointcnn: Convolution on x-transformed points. In Advances in neural information processing systems. 820–830.Google Scholar
34. Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu. 2020. Morphing and sampling network for dense point cloud completion. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11596–11603.Google ScholarCross Ref
35. Xinhai Liu, Zhizhong Han, Xin Wen, Yu-Shen Liu, and Matthias Zwicker. 2019. L2g auto-encoder: Understanding point clouds by local-to-global reconstruction with hierarchical self-attention. In Proceedings of the 27th ACM International Conference on Multimedia. 989–997.Google ScholarDigital Library
36. Shitong Luo and Wei Hu. 2020. Differentiable Manifold Reconstruction for Point Cloud Denoising. In Proceedings of the 28th ACM International Conference on Multimedia. 1330–1338.Google ScholarDigital Library
37. Viní cius Mello, Luiz Velho, and Gabriel Taubin. 2003. Estimating the in/out function of a surface represented by points. In Proceedings of the eighth ACM symposium on Solid modeling and applications. 108–114.Google ScholarDigital Library
38. Gal Metzer, Rana Hanocka, Raja Giryes, and Daniel Cohen-Or. 2020. Self-Sampling for Neural Point Cloud Consolidation. arXiv preprint arXiv:2008.06471 (2020).Google Scholar
39. Niloy J Mitra and An Nguyen. 2003. Estimating surface normals in noisy point cloud data. In Proceedings of the nineteenth annual symposium on Computational geometry. 322–328.Google ScholarDigital Library
40. Patrick Mullen, Fernando De Goes, Mathieu Desbrun, David Cohen-Steiner, and Pierre Alliez. 2010. Signing the unsigned: Robust surface reconstruction from raw pointsets. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 1733–1741.Google Scholar
41. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Love-grove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
42. Mark Pauly, Richard Keiser, Leif P Kobbelt, and Markus Gross. 2003. Shape modeling with point-sampled geometry. ACM Transactions on Graphics (TOG) 22, 3 (2003), 641–650.Google ScholarDigital Library
43. Francesca Pistilli, Giulia Fracastoro, Diego Valsesia, and Enrico Magli. 2020. Learning Graph-Convolutional Representations for Point Cloud Denoising. In European Conference on Computer Vision. Springer, 103–118.Google Scholar
44. Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017a. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 652–660.Google Scholar
45. Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017b. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems 30 (2017), 5099–5108.Google Scholar
46. Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J Mitra, and Maks Ovsjanikov. 2020. Pointcleannet: Learning to denoise and remove outliers from dense point clouds. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 185–203.Google Scholar
47. Javier Romero, Dimitrios Tzionas, and Michael J. Black. 2017. Embodied Hands: Modeling and Capturing Hands and Bodies Together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) 36, 6 (Nov. 2017).Google Scholar
48. Muhammad Sarmad, Hyunjoo Jenny Lee, and Young Min Kim. 2019. Rl-gan-net: A reinforcement learning agent controlled gan network for real-time point cloud shape completion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5898–5907.Google ScholarCross Ref
49. Nico Schertler, Bogdan Savchynskyy, and Stefan Gumhold. 2017. Towards globally optimal normal orientations for large point clouds. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 197–208.Google Scholar
50. Lee M Seversky, Matt S Berger, and Lijun Yin. 2011. Harmonic point cloud orientation. Computers & Graphics 35, 3 (2011), 492–499.Google ScholarDigital Library
51. Oana Sidi, Oliver van Kaick, Yanir Kleiman, Hao Zhang, and Daniel Cohen-Or. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. In Proceedings of the 2011 SIGGRAPH Asia Conference. 1–10.Google ScholarDigital Library
52. Yongbin Sun, Yue Wang, Ziwei Liu, Joshua Siegel, and Sanjay Sarma. 2020. Pointgrow: Autoregressively learned point cloud generation with self-attention. In The IEEE Winter Conference on Applications of Computer Vision. 61–70.Google ScholarCross Ref
53. Lyne P Tchapmi, Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. 2019. Topnet: Structural point cloud decoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 383–392.Google ScholarCross Ref
54. The CGAL Project. 2020. CGAL User and Reference Manual (5.2 ed.). CGAL Editorial Board. https://doc.cgal.org/5.2/Manual/packages.htmlGoogle Scholar
55. Christian Walder, Olivier Chapelle, and Bernhard Schölkopf. 2005. Implicit surface modelling as an eigenvalue problem. In Proceedings of the 22nd international conference on Machine learning. 936–939.Google ScholarDigital Library
56. Xiaogang Wang, Marcelo H Ang Jr, and Gim Hee Lee. 2020. Point Cloud Completion by Learning Shape Priors. arXiv preprint arXiv:2008.00394 (2020).Google Scholar
57. Yunhai Wang, Shmulik Asafi, Oliver Van Kaick, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Active co-analysis of a set of shapes. ACM Transactions on Graphics (TOG) 31, 6 (2012), 1–10.Google ScholarDigital Library
58. Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38, 5 (2019), 1–12.Google ScholarDigital Library
59. Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. 2020. Point cloud completion by skip-attention network with hierarchical folding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1939–1948.Google ScholarCross Ref
60. Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2019. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10130–10139.Google ScholarCross Ref
61. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912–1920.Google Scholar
62. Hui Xie, Kevin T McDonnell, and Hong Qin. 2004. Surface reconstruction of noisy and defective data sets. In IEEE Visualization 2004. IEEE, 259–266.Google ScholarDigital Library
63. Hui Xie, Jianning Wang, Jing Hua, Hong Qin, and Arie Kaufman. 2003. Piecewise C/sup 1/continuous surface reconstruction of noisy point clouds via local implicit quadric regression. In IEEE Visualization, 2003. VIS 2003. IEEE, 91–98.Google ScholarDigital Library
64. Minfeng Xu, Shiqing Xin, and Changhe Tu. 2018. Towards globally optimal normal orientations for thin surfaces. Computers & Graphics 75 (2018), 36–43.Google ScholarDigital Library
65. Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. 2019. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE International Conference on Computer Vision. 4541–4550.Google ScholarCross Ref
66. Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. 2018. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 206–215.Google ScholarCross Ref
67. Wang Yifan, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine-Hornung. 2019a. Differentiable surface splatting for point-based geometry processing. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–14.Google ScholarDigital Library
68. Wang Yifan, Shihao Wu, Hui Huang, Daniel Cohen-Or, and Olga Sorkine-Hornung. 2019b. Patch-based progressive 3d point set upsampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5958–5967.Google ScholarCross Ref
69. Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018a. Ec-net: an edge-aware point set consolidation network. In Proceedings of the European Conference on Computer Vision (ECCV). 386–402.Google ScholarDigital Library
70. Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018b. Pu-net: Point cloud upsampling network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2790–2799.Google ScholarCross Ref
71. Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. 2018. Pcn: Point completion network. In 2018 International Conference on 3D Vision (3DV). IEEE, 728–737.Google ScholarCross Ref
72. Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 2019. 3D point capsule networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1009–1018.Google ScholarCross Ref
73. Qingnan Zhou and Alec Jacobson. 2016. Thingi10K: A Dataset of 10,000 3D-Printing Models. arXiv preprint arXiv:1605.04797 (2016).Google Scholar