“Neural Stochastic Poisson Surface Reconstruction” by Sellán and Jacobson
Conference:
Type(s):
Title:
- Neural Stochastic Poisson Surface Reconstruction
Session/Category Title: Reconstruction
Presenter(s)/Author(s):
Abstract:
Reconstructing a surface from a point cloud is an underdetermined problem. We propose using a neural network to study and quantify this reconstruction uncertainty under a Poisson smoothness prior. Our algorithm addresses the main limitations of existing work and can be fully integrated into the 3D scanning pipeline, from deciding on the next best sensor position to iteratively updating the reconstruction upon capturing more data.
References:
[1]
Marc Alexa, Johannes Behr, Daniel Cohen-Or, Shachar Fleishman, David Levin, and Claudio T. Silva. 2003. Computing and rendering point set surfaces. IEEE TVCG 9, 1 (2003), 3–15.
[2]
Matthew Berger, Andrea Tagliasacchi, Lee M Seversky, Pierre Alliez, Gael Guennebaud, Joshua A Levine, Andrei Sharf, and Claudio T Silva. 2017. A survey of surface reconstruction from point clouds. In Comput. Graph. Forum, Vol. 36. Wiley Online Library, 301–329.
[3]
Simeon M Berman. 1987. An extension of Plackett’s differential equation for the multivariate normal density. SIAM Journal on Algebraic Discrete Methods 8, 2 (1987), 196–197.
[4]
Andreas Bircher, Mina Kamel, Kostas Alexis, Helen Oleynikova, and Roland Siegwart. 2016. Receding horizon” next-best-view” planner for 3d exploration. In 2016 IEEE international conference on robotics and automation (ICRA). IEEE, 1462–1468.
[5]
Jonathan C Carr, Richard K Beatson, Jon B Cherrie, Tim J Mitchell, W Richard Fright, Bruce C McCallum, and Tim R Evans. 2001. Reconstruction and representation of 3D objects with radial basis functions. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. 67–76.
[6]
Shengyong Chen, Youfu Li, and Ngai Ming Kwok. 2011. Active vision in robotic systems: A survey of recent developments. The International Journal of Robotics Research 30, 11 (2011), 1343–1377.
[7]
Cl Connolly. 1985. The determination of next best views. In Proceedings. 1985 IEEE international conference on robotics and automation, Vol. 2. IEEE, 432–435.
[8]
Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. 2022. Scientific machine learning through physics–informed neural networks: where we are and what’s next. Journal of Scientific Computing 92, 3 (2022), 88.
[9]
Angela Dai and Matthias Nießner. 2022. Neural Poisson: Indicator Functions for Neural Fields. arXiv preprint arXiv:2211.14249 (2022).
[10]
Jonathan Daudelin and Mark Campbell. 2017. An adaptable, probabilistic, next-best view algorithm for reconstruction of unknown 3-d objects. IEEE Robotics and Automation Letters 2, 3 (2017), 1540–1547.
[11]
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018. A papier-mâché approach to learning 3d surface generation. In Proc. CVPR. 216–224.
[12]
Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. In Proceedings of the 19th annual conference on computer graphics and interactive techniques. 71–78.
[13]
Stefan Isler, Reza Sabzevari, Jeffrey Delmerico, and Davide Scaramuzza. 2016. An information gain formulation for active volumetric 3D reconstruction. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3477–3484.
[14]
Liren Jin, Xieyuanli Chen, Julius Rückin, and Marija Popović. 2023. NeU-NBV: Next Best View Planning Using Uncertainty Estimation in Image-Based Neural Rendering. arXiv preprint arXiv:2303.01284 (2023).
[15]
Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson surface reconstruction. In Proc. SGP, Vol. 7. 0.
[16]
Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Trans. Graph. 32, 3 (2013), 1–13.
[17]
Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision?Proc. NeurIPS 30 (2017).
[18]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[19]
Xin Kong, Shikun Liu, Marwan Taher, and Andrew J Davison. 2023. vMAP: Vectorised Object Mapping for Neural Field SLAM. arXiv preprint arXiv:2302.01838 (2023).
[20]
Sören König and Stefan Gumhold. 2009. Consistent Propagation of Normal Orientations in Point Clouds. In VMV. 83–92.
[21]
Mikko Lauri, Joni Pajarinen, Jan Peters, and Simone Frintrop. 2020. Multi-Sensor Next-Best-View Planning as Matroid-Constrained Submodular Maximization. IEEE Robotics and Automation Letters 5, 4 (Oct. 2020), 5323–5330. https://doi.org/10.1109/LRA.2020.3007445 arxiv:2007.02084
[22]
David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. Springer, 37–49.
[23]
Sébastien Marmin. 2023. Torch-MvNorm. https://github.com/SebastienMarmin/torch-mvnorm.
[24]
Sébastien Marmin, Clément Chevalier, and David Ginsbourger. 2015. Differentiating the multipoint expected improvement for optimal batch design. In International Workshop on Machine Learning, Optimization and Big Data. Springer, 37–48.
[25]
Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth. 2021. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7210–7219.
[26]
Gal Metzer, Rana Hanocka, Denis Zorin, Raja Giryes, Daniele Panozzo, and Daniel Cohen-Or. 2021. Orienting point clouds with dipole propagation. ACM Trans. Graph. 40, 4 (2021), 1–14.
[27]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99–106.
[28]
Yutaka Ohtake, Alexander Belyaev, Marc Alexa, Greg Turk, and Hans-Peter Seidel. 2005. Multi-level partition of unity implicits. In Acm Siggraph 2005 Courses. 173–es.
[29]
Ahmed AA Osman, Timo Bolkart, and Michael J Black. 2020. Star: Sparse trained articulated human body regressor. In Proc. ECCV. Springer, 598–613.
[30]
Onur Özyeşil, Vladislav Voroninski, Ronen Basri, and Amit Singer. 2017. A survey of structure from motion*.Acta Numerica 26 (2017), 305–364.
[31]
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In Proc. CVPR. 165–174.
[32]
Mark Pauly, Niloy J Mitra, and Leonidas J Guibas. 2004. Uncertainty and variability in point cloud surface data. In PBG. 77–84.
[33]
Mark Pauly, Niloy J Mitra, Johannes Wallner, Helmut Pottmann, and Leonidas J Guibas. 2008. Discovering structural regularity in 3D geometry. ACM Trans. Graph. (2008), 1–11.
[34]
Thinal Raj, Fazida Hanim Hashim, Aqilah Baseri Huddin, Mohd Faisal Ibrahim, and Aini Hussain. 2020. A survey on LiDAR scanning mechanisms. Electronics 9, 5 (2020), 741.
[35]
Oussama Remil, Qian Xie, Xingyu Xie, Kai Xu, and Jun Wang. 2017. Surface reconstruction with data-driven exemplar priors. 88 (2017), 31–41.
[36]
Nico Schertler, Bogdan Savchynskyy, and Stefan Gumhold. 2017. Towards globally optimal normal orientations for large point clouds. In Comput. Graph. Forum, Vol. 36. Wiley Online Library, 197–208.
[37]
Ruwen Schnabel, Patrick Degener, and Reinhard Klein. 2009. Completion and reconstruction with primitive shapes. In Comput. Graph. Forum, Vol. 28. Wiley Online Library, 503–512.
[38]
William R Scott, Gerhard Roth, and Jean-François Rivest. 2003. View planning for automated three-dimensional object reconstruction and inspection. ACM Computing Surveys (CSUR) 35, 1 (2003), 64–96.
[39]
Silvia Sellán and Alec Jacobson. 2022. Stochastic Poisson Surface Reconstruction. ACM Trans. Graph. 41, 6 (2022), 1–12.
[40]
Silvia Sellán, Oded Stein, 2023. gptyoolbox: A Python Geometry Processing Toolbox. https://gpytoolbox.org/.
[41]
Andrei Sharf, Thomas Lewiner, Gil Shklarski, Sivan Toledo, and Daniel Cohen-Or. 2007. Interactive topology-aware surface reconstruction. ACM Trans. Graph. 26, 3 (2007), 43–es.
[42]
Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. 2020. Implicit Neural Representations with Periodic Activation Functions. In Proc. NeurIPS.
[43]
Edward J Smith, Michal Drozdzal, Derek Nowrouzezahrai, David Meger, and Adriana Romero-Soriano. 2022. Uncertainty-Driven Active Vision for Implicit Scene Reconstruction. arXiv preprint arXiv:2210.00978 (2022).
[44]
Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J Davison. 2021. iMAP: Implicit mapping and positioning in real-time. In Proc. ICCV. 6229–6238.
[45]
J Irving Vasquez-Gomez, L Enrique Sucar, Rafael Murrieta-Cid, and Efrain Lopez-Damian. 2014. Volumetric next-best-view planning for 3D object reconstruction with positioning error. International Journal of Advanced Robotic Systems 11, 10 (2014), 159.
[46]
Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2019. Deep geometric prior for surface reconstruction. In Proc. CVPR. 10130–10139.
[47]
Kai Xu, Hui Huang, Yifei Shi, Hao Li, Pinxin Long, Jianong Caichen, Wei Sun, and Baoquan Chen. 2015. Autoscanning for coupled scene reconstruction and proactive object analysis. ACM Trans. Graph. 34, 6 (2015), 1–14.
[48]
Brian Yamauchi. 1997. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’. IEEE, 146–151.
[49]
Bing Yu 2018. The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics 6, 1 (2018), 1–12.
[50]
Han Zhang, Yucong Yao, Ke Xie, Chi-Wing Fu, Hao Zhang, and Hui Huang. 2021. Continuous aerial path planning for 3D urban scene reconstruction.ACM Trans. Graph. 40, 6 (2021), 225–1.
[51]
Xiaohui Zhou, Ke Xie, Kai Huang, Yilin Liu, Yang Zhou, Minglun Gong, and Hui Huang. 2020. Offsite aerial path planning for efficient urban scene reconstruction. ACM Trans. Graph. 39, 6 (2020), 1–16.
