“P2P-NET: bidirectional point displacement net for shape transform” by Yin, Huang, Cohen-Or and Zhang
Conference:
Type(s):
Entry Number: 152
Title:
- P2P-NET: bidirectional point displacement net for shape transform
Session/Category Title: Shape Analysis
Presenter(s)/Author(s):
Abstract:
We introduce P2P-NET, a general-purpose deep neural network which learns geometric transformations between point-based shape representations from two domains, e.g., meso-skeletons and surfaces, partial and complete scans, etc. The architecture of the P2P-NET is that of a bi-directional point displacement network, which transforms a source point set to a prediction of the target point set with the same cardinality, and vice versa, by applying point-wise displacement vectors learned from data. P2P-NET is trained on paired shapes from the source and target domains, but without relying on point-to-point correspondences between the source and target point sets. The training loss combines two uni-directional geometric losses, each enforcing a shape-wise similarity between the predicted and the target point sets, and a cross-regularization term to encourage consistency between displacement vectors going in opposite directions. We develop and present several different applications enabled by our general-purpose bidirectional P2P-NET to highlight the effectiveness, versatility, and potential of our network in solving a variety of point-based shape transformation problems.
References:
1. Marc Alexa, Johannes Behr, Daniel Cohen-Or, Shachar Fleishman, David Levin, and Claudio T Silva. 2003. Computing and rendering point set surfaces. IEEE Trans. Vis. & Comp. Graphics 9, 1 (2003), 3–15. Google ScholarDigital Library
2. Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Gaël Guennebaud, Joshua A. Levine, Andrei Sharf, and Claudio T. Silva. 2017. A Survey of Surface Reconstruction from Point Clouds. Computer Graphics Forum 36, 1 (2017), 301–329. Google ScholarDigital Library
3. Sema Berkiten, Maciej Halber, Justin Solomon, Chongyang Ma, Hao Li, and Szymon Rusinkiewicz. 2017. Learning detail transfer based on geometric features. In Computer Graphics Forum (Eurographics), Vol. 36. 361–373. Google ScholarDigital Library
4. Jeannette Bohg, Javier Romero, Alexander Herzog, and Stefan Schaal. 2014. Robot arm pose estimation through pixel-wise part classification. In Proc. of ICRA. IEEE, 3143–3150. https://github.com/jbohg/render_kinectGoogle ScholarCross Ref
5. Arunkumar Byravan and Dieter Fox. 2016. SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks. In Proc. of ICRA.Google Scholar
6. Junjie Cao, Andrea Tagliasacchi, Matt Olson, Hao Zhang, and Zhinxun Su. 2010. Point cloud skeletons via laplacian based contraction. In Proc. IEEE Int. Conf. on Shape Modeling and Applications. 187–197. Google ScholarDigital Library
7. Jonathan C Carr, Richard K Beatson, Jon B Cherrie, Tim J Mitchell, W Richard Fright, Bruce C McCallum, and Tim R Evans. 2001. Reconstruction and representation of 3D objects with radial basis functions. In ACM Trans. on Graph (SIGGRAPH). 67–76. Google ScholarDigital Library
8. Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. 2012. Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Vis. & Comp. Graphics 18, 6 (2012), 914–924. Google ScholarDigital Library
9. Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. 2017. Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis. In Proc. of CVPR.Google ScholarCross Ref
10. Haoqiang Fan, Hao Su, and Leonidas Guibas. 2017. A point set generation network for 3d object reconstruction from a single image. In Proc. of CVPR.Google ScholarCross Ref
11. Matheus Gadelha, Subhransu Maji, and Rui Wang. 2017. Shape Generation using Spatially Partitioned Point Clouds. In Proc. of BMVC.Google ScholarCross Ref
12. Markus Gross and Hanspeter Pfister. 2007. Point-Based Graphics. Morgan Kaufmann. Google ScholarDigital Library
13. Michael Gschwandtner, Roland Kwitt, Andreas Uhl, and Wolfgang Pree. 2011. BlenSor: blender sensor simulation toolbox. Advances in visual computing (2011), 199–208. Google ScholarDigital Library
14. Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, and Niloy J Mitra. 2017. PCP-NET: Learning Local Shape Properties from Raw Point Clouds. arXiv preprint arXiv:1710.04954 (2017).Google Scholar
15. Ruizhen Hu, Wenchao Li, Oliver van Kaick, Hui Huang, Melinos Averkiou, Daniel Cohen-Or, and Hao Zhang. 2017. Co-Locating Style-Defining Elements on 3D Shapes. ACM Trans. on Graph 36, 3 (2017), 33:1–33:15. Google ScholarDigital Library
16. Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of unorganized point clouds for surface reconstruction. In ACM Trans. on Graph (SIGGRAPH Asia), Vol. 28. 176:1–176:7. Google ScholarDigital Library
17. Hui Huang, Shihao Wu, Daniel Cohen-Or, Minglun Gong, Hao Zhang, Guiqing Li, and Baoquan Chen. 2013a. L1-medial skeleton of point cloud. ACM Trans. on Graph (SIGGRAPH) 32, 4 (2013), 65:1–65:8. Google ScholarDigital Library
18. Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang. 2013b. Edge-aware point set resampling. ACM Trans. on Graph 32, 1 (2013), 9:1–9:12. Google ScholarDigital Library
19. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proc. of CVPR.Google ScholarCross Ref
20. Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. 2015. Spatial transformer networks. In Proc. of NIPS. 2017–2025. Google ScholarDigital Library
21. Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson surface reconstruction. ACM Trans. on Graph 32, 3 (2013), 29:1–29:13. Google ScholarDigital Library
22. Chen-Hsuan Lin, Chen Kong, and Simon Lucey. 2018. Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction. In Proc. of AAAI.Google Scholar
23. Yaron Lipman, Daniel Cohen-Or, David Levin, and Hillel Tal-Ezer. 2007. Parameterization-free projection for geometry reconstruction. In ACM Trans. on Graph (SIGGRAPH), Vol. 26. 22:1–22:6. Google ScholarDigital Library
24. Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised Image-to-Image Translation Networks. In Proc. of NIPS. 700–708.Google Scholar
25. Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks. In Proc. of 3DV.Google ScholarCross Ref
26. Niloy J Mitra and An Nguyen. 2003. Estimating surface normals in noisy point cloud data. In Symp. on Geom. Proc. 322–328. Google ScholarDigital Library
27. Matthias Müller, Richard Keiser, Andrew Nealen, Mark Pauly, Markus Gross, and Marc Alexa. 2004. Point based animation of elastic, plastic and melting objects. In Symp. on Computer Animation. 141–151. Google ScholarDigital Library
28. Mark Pauly, Richard Keiser, Leif P Kobbelt, and Markus Gross. 2003. Shape modeling with point-sampled geometry. ACM Trans. on Graph (SIGGRAPH) 22, 3 (2003), 641–650. Google ScholarDigital Library
29. Reinhold Preiner, Oliver Mattausch, Murat Arikan, Renato Pajarola, and Michael Wimmer. 2014. Continuous Projection for Fast L1 Reconstruction. ACM Trans. on Graph 33, 4 (July 2014), 47:1–47:13. Google ScholarDigital Library
30. Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017a. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proc. of CVPR.Google Scholar
31. Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017b. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metrie Space. In Proc. of NIPS.Google Scholar
32. Minhyuk Sung, Hao Su, Vladimir G Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling. ACM Trans. on Graph (SIGGRAPH Asia) 36, 6 (2017), 226:1–226:12. Google ScholarDigital Library
33. Andrea Tagliasacchi, Thomas Delame, Michela Spagnuolo, Nina Amenta, and Alexandru Telea. 2016. 3D Skeletons. In Eurographics State of the Art Report.Google Scholar
34. Andrea Tagliasacchi, Hao Zhang, and Daniel Cohen-Or. 2009. Curve skeleton extraction from incomplete point cloud. In ACM Trans. on Graph (SIGGRAPH Asia), Vol. 28. 71:1–71:9. Google ScholarDigital Library
35. Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. 2015a. Deep Points Consolidation. ACM Trans. on Graph 34, 6 (2015), 176:1–176:13. Google ScholarDigital Library
36. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015b. 3d shapenets: A deep representation for volumetric shapes. In Proc. of CVPR. 1912–1920.Google Scholar
37. Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. 2017. DualGAN: Unsupervised Dual Learning for Image-to-image Translation. In Proc. of ICCV.Google ScholarCross Ref
38. Tinghui Zhou, Shubham Tulsiani, Weilun Sun, Jitendra Malik, and Alexei A Efros. 2016. View synthesis by appearance flow. In Proc. Euro. Conf. on Comp. Vis. 286–301.Google ScholarCross Ref
39. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. of ICCV.Google ScholarCross Ref
