“Learning to group and label fine-grained shape components”
Conference:
Type(s):
Title:
- Learning to group and label fine-grained shape components
Session/Category Title: Learning to compose & decompose
Presenter(s)/Author(s):
Moderator(s):
Abstract:
A majority of stock 3D models in modern shape repositories are assembled with many fine-grained components. The main cause of such data form is the component-wise modeling process widely practiced by human modelers. These modeling components thus inherently reflect some function-based shape decomposition the artist had in mind during modeling. On the other hand, modeling components represent an over-segmentation since a functional part is usually modeled as a multi-component assembly. Based on these observations, we advocate that labeled segmentation of stock 3D models should not overlook the modeling components and propose a learning solution to grouping and labeling of the fine-grained components. However, directly characterizing the shape of individual components for the purpose of labeling is unreliable, since they can be arbitrarily tiny and semantically meaningless. We propose to generate part hypotheses from the components based on a hierarchical grouping strategy, and perform labeling on those part groups instead of directly on the components. Part hypotheses are mid-level elements which are more probable to carry semantic information. A multi-scale 3D convolutional neural network is trained to extract context-aware features for the hypotheses. To accomplish a labeled segmentation of the whole shape, we formulate higher-order conditional random fields (CRFs) to infer an optimal label assignment for all components. Extensive experiments demonstrate that our method achieves significantly robust labeling results on raw 3D models from public shape repositories. Our work also contributes the first benchmark for component-wise labeling.
References:
1. 2017. 3D Warehouse. https://3dwarehouse.sketchup.com/. (2017). Accessed: 2017-05-18.Google Scholar
2. 2017. GAMMA mesh database. https://www.rocq.inria.fr/gamma/gamma/download/download.php. (2017). Accessed: 2017-09-10.Google Scholar
3. Ibraheem Alhashim, Kai Xu, Yixin Zhuang, Junjie Cao, Patricio Simari, and Hao Zhang. 2015. Deformation-driven topology-varying 3D shape correspondence. Acm Transactions on Graphics 34, 6 (2015), 236. Google ScholarDigital Library
4. Oscar Kin-Chung Au, Youyi Zheng, Menglin Chen, Pengfei Xu, and Chiew-Lan Tai. 2012. Mesh Segmentation with Concavity-Aware Fields. IEEE Transactions on Visualization and Computer Graphics (TVCG) 18, 7 (2012), 1125–1134. Google ScholarDigital Library
5. João Carreira and Cristian Sminchisescu. 2012. CPMC: Automatic Object Segmentation Using Constrained Parametric Min-Cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 34, 7 (2012), 1312–1328. Google ScholarDigital Library
6. Angel X. Chang, Thomas Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 {cs.GR}. Stanford University — Princeton University — Toyota Technological Institute at Chicago.Google Scholar
7. Ding-Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Ouhyoung. 2010. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum (Proc. Eurographics) 22, 3 (2010), 223–232.Google ScholarCross Ref
8. Ross Girshick. 2015. Fast R-CNN. arXiv preprint arXiv:1504.08083 (2015). Google ScholarDigital Library
9. Kan Guo, Dongqing Zou, and Xiaowu Chen. 2015. 3D Mesh Labeling via Deep Convolutional Neural Networks. ACM Transactions on Graphics 35, 1 (2015), 3:1–3:12. Google ScholarDigital Library
10. Ruizhen Hu, Lubin Fan, and Ligang Liu. 2012. Co-Segmentation of 3D Shapes via Subspace Clustering. Computer Graphics Forum (Proc. SGP) 31, 5 (2012), 1703–1713. Google ScholarDigital Library
11. Qixing Huang, Vladlen Koltun, and Leonidas J. Guibas. 2011. Joint Shape Segmentation with Linear Programming. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 30, 6 (2011), 125:1–125:12. Google ScholarDigital Library
12. Qixing Huang, Martin Wicke, Bart Adams, and Leonidas J. Guibas. 2009. Shape Decomposition using Modal Analysis. Computer Graphics Forum (Proc. Eurographics) 28, 2 (2009), 407–416.Google ScholarCross Ref
13. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe:Convolutional Architecture for Fast Feature Embedding. (2014), 675–678. Google ScholarDigital Library
14. Evangelos Kalogerakis, Melinos Averkiou, Subhransu Maji, and Siddhartha Chaudhuri. 2017. 3D shape segmentation with projective convolutional networks. In Proc. CVPR, Vol. 1. 8.Google Scholar
15. Evangelos Kalogerakis, Aaron Hertzmann, and Karan Singh. 2010. Learning 3D Mesh Segmentation and Labeling. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 4 (2010), 102:1–102:12. Google ScholarDigital Library
16. Sagi Katz and Ayellet Tal. 2003. Hierarchical Mesh Decomposition using Fuzzy Clustering and Cuts. ACM Transactions on Graphics (Proc. SIGGRAPH) 22, 3 (2003), 954–961. Google ScholarDigital Library
17. Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. 2003. Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors. In Proc. SGP. 156–164. Google ScholarDigital Library
18. Diederik P Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. Computer Science (2014).Google Scholar
19. Pushmeet Kohli, L’ubor Ladický, and Phillip H. S. Torr. 2008. Graph Cuts for Minimizing Robust Higher Order Potentials. Technical Report. Oxford Brookes University Uk.Google Scholar
20. Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. 2017. GRASS: Generative Recursive Autoencoders for Shape Structures. ACM Transactions on Graphics (Proc. of SIGGRAPH) 36, 4 (2017), 52. Google ScholarDigital Library
21. Tianqiang Liu, Siddhartha Chaudhuri, Vladimir G. Kim, Qi-Xing Huang, Niloy J. Mitra, and Thomas Funkhouser. 2014. Creating Consistent Scene Graphs Using a Probabilistic Grammar. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 33, 6 (2014), 211:1–211:12. Google ScholarDigital Library
22. Jiajun Lv, Xinlei Chen, Jin Huang, and Hujun Bao. 2012. Semi-supervised Mesh Segmentation and Labeling. Computer Graphics Forum (Proc. Pacific Graphics) 31, 7 (2012), 2241–2248. Google ScholarDigital Library
23. Santiago Manen, Matthieu Guillaumin, and Luc Van Gool. 2014. Prime Object Proposals with Randomized Prim’s Algorithm. In Proc. IEEE International Conference on Computer Vision (ICCV). 2536–2543. Google ScholarDigital Library
24. Kyoungup Park and Stephen Gould. 2003. On Learning Higher-Order Consistency Potentials for Multi-class Pixel Labeling. In Proc. European Conference on Computer Vision (ECCV). 202–215.Google Scholar
25. Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. arXiv preprint arXiv:1706.02413 (2017).Google Scholar
26. Lior Shapira, Shy Shalom, Ariel Shamir, Daniel Cohen-Or, and Hao Zhang. 2010. Contextual Part Analogies in 3D Objects. International Journal of Computer Vision (IJCV) 89, 1–2 (2010), 309–326. Google ScholarDigital Library
27. Oana Sidi, Oliver van Kaick, Yanir Kleiman, Hao Zhang, and Daniel Cohen-Or. 2011. Unsupervised Co-Segmentation of a Set of Shapes via Descriptor-Space Spectral Clustering. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 30, 6 (2011), 126:1–126:10. Google ScholarDigital Library
28. Saurabh Singh, Abhinav Gupta, and Alexei A. Efros. 2012. Unsupervised Discovery of Mid-level Discriminative Patches. In European Conference on Computer Vision. arXiv:cs.CV/1205.3137 http://arxiv.org/abs/1205.3137Google Scholar
29. Hao Su, Charles Ruizhongtai Qi, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). to appear.Google Scholar
30. Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-Supervised Component Suggestions for3DModeling. ACM Transactions on Graphics (Proc. of SIGGRAPH Asia) 36, 6 (2017). Google ScholarDigital Library
31. Koen E. A. van de Sande, Jasper R. R. Uijlings, Theo Gevers, and Arnold W. M. Smeulders. 2011. Segmentation as Selective Search for Object Recognition. In Proc. IEEE International Conference on Computer Vision (ICCV). 1879–1886. Google ScholarDigital Library
32. Oliver van Kaick, Kai Xu, Hao Zhang, Yanzhen Wang, Shuyang Sun, Ariel Shamir, and Daniel Cohen-Or. 2013. Co-Hierarchical Analysis of Shape Structures. ACM Transactions on Graphics (Proc. SIGGRAPH) 32, 4 (2013), 69:1–69:10. Google ScholarDigital Library
33. Yunhai Wang, Shmulik Asafi, Oliver van Kaick, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Active Co-Analysis of a Set of Shapes. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 31, 6 (2012), 165:1–165:10. Google ScholarDigital Library
34. Yunhai Wang, Minglun Gong, Tianhua Wang, Daniel Cohen-Or, Hao Zhang, and Baoquan Chen. 2013. Projective Analysis for 3D Shape Segmentation. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 32, 6 (2013), 192:1–192:12. Google ScholarDigital Library
35. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3D ShapeNets: A Deep Representation for Volumetric Shapes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1912–1920.Google Scholar
36. Zhige Xie, Kai Xu, and Ligang Liu abd Yueshan Xiong. 2014. 3D Shape Segmentation and Labeling via Extreme Learning Machine. Computer Graphics Forum (Proc. SGP) 33, 5 (2014), 85–95.Google ScholarDigital Library
37. Kai Xu, Vladimir G Kim, Qixing Huang, Niloy Mitra, and Evangelos Kalogerakis. 2016. Data-driven shape analysis and processing. In SIGGRAPH ASIA 2016 Courses. ACM, 4. Google ScholarDigital Library
38. Kai Xu, Honghua Li, Hao Zhang, Daniel Cohen-Or, Yueshan Xiong, and Zhi-Quan Cheng. 2010. Style-content separation by anisotropic part scales. ACM Transactions on Graphics (TOG) 29, 6 (2010), 184. Google ScholarDigital Library
39. Li Yi, Leonidas Guibas, Aaron Hertzmann, Vladimir G. Kim, Hao Su, and Ersin Yumer. 2017a. Learning Hierarchical Shape Segmentation and Labeling from Online Repositories. SIGGRAPH (2017). Google ScholarDigital Library
40. Li Yi, Hao Su, Xingwen Guo, and Leonidas J. Guibas. 2017b. SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). to appear.Google Scholar
41. Juyong Zhang, Jianmin Zheng, Chunlin Wu, and Jianfei Cai. 2012. Variational Mesh Decomposition. ACM Transactions on Graphics 31, 3 (2012), 21:1–31:15. Google ScholarDigital Library
42. Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip HS Torr. 2015. Conditional random fields as recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 1529–1537. Google ScholarDigital Library
43. Chenyang Zhu, Renjiao YI, Wallace LIRA, Ibraheem ALHASHIM, Kai XU, and Hao ZHANG. 2017. Deformation-driven shape correspondence via shape recognition. Acm Transactions on Graphics 36, 4 (2017), 51. Google ScholarDigital Library
