“SCORES: shape composition with recursive substructure priors”
Conference:
Type(s):
Title:
- SCORES: shape composition with recursive substructure priors
Session/Category Title: Learning to compose & decompose
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We introduce SCORES, a recursive neural network for shape composition. Our network takes as input sets of parts from two or more source 3D shapes and a rough initial placement of the parts. It outputs an optimized part structure for the composed shape, leading to high-quality geometry construction. A unique feature of our composition network is that it is not merely learning how to connect parts. Our goal is to produce a coherent and plausible 3D shape, despite large incompatibilities among the input parts. The network may significantly alter the geometry and structure of the input parts and synthesize a novel shape structure based on the inputs, while adding or removing parts to minimize a structure plausibility loss. We design SCORES as a recursive autoencoder network. During encoding, the input parts are recursively grouped to generate a root code. During synthesis, the root code is decoded, recursively, to produce a new, coherent part assembly. Assembled shape structures may be novel, with little global resemblance to training exemplars, yet have plausible substructures. SCORES therefore learns a hierarchical substructure shape prior based on per-node losses. It is trained on structured shapes from ShapeNet, and is applied iteratively to reduce the plausibility loss. We show results of shape composition from multiple sources over different categories of man-made shapes and compare with state-of-the-art alternatives, demonstrating that our network can significantly expand the range of composable shapes for assembly-based modeling.
References:
1. Ibraheem Alhashim, Honghua Li, Kai Xu, Junjie Cao, Rui Ma, and Hao Zhang. 2014. Topology-Varying 3D Shape Creation via Structural Blending. In SIGGRAPH. Google ScholarDigital Library
2. Sara Ball. 1985. Croc-gu-phant. Ragged Bears.Google Scholar
3. Martin Bokeloh, Michael Wand, and Hans-Peter Seidel. 2010. A Connection Between Partial Symmetry and Inverse Procedural Modeling. In SIGGRAPH. Google ScholarDigital Library
4. Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. CoRR abs/1512.03012 (2015).Google Scholar
5. Siddhartha Chaudhuri, Evangelos Kalogerakis, Leonidas Guibas, and Vladlen Koltun. 2011. Probabilistic Reasoning for Assembly-based 3D Modeling. ACM Trans. Graph. 30, 4 (2011), 35:1–35:10. Google ScholarDigital Library
6. Siddhartha Chaudhuri and Vladlen Koltun. 2010. Data-Driven Suggestions for Creativity Support in 3D Modeling. ACM Trans. Graph. 29, 6 (2010), 183:1–9. Google ScholarDigital Library
7. Daniel Cohen-Or and Hao Zhang. 2016. From inspired modeling to creative modeling. The Visual Computer 32, 1 (2016), 1–8. Google ScholarDigital Library
8. Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum. 2016. Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. In NIPS. Google ScholarDigital Library
9. Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3D ShapeNets: A deep representation for volumetric shapes. In CVPR.Google Scholar
10. Carl Doersch. 2016. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016).Google Scholar
11. Noah Duncan, Lap-Fai Yu, and Sai-Kit Yeung. 2016. Interchangeable Components for Hands-on Assembly Based Modelling. ACM Trans. Graph. 35, 6 (2016), 234:1–234:14. Google ScholarDigital Library
12. Thomas Funkhouser, Michael Kazhdan, Philip Shilane, Patrick Min, William Kiefer, Ayellet Tal, Szymon Rusinkiewicz, and David Dobkin. 2004. Modeling by Example. ACM Trans. Graph. 23, 3 (2004), 652–663. Google ScholarDigital Library
13. Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. 2016. Learning a predictable and generative vector representation for objects. In ECCV.Google Scholar
14. Haibin Huang, Evangelos Kalogerakis, and Benjamin Marlin. 2015. Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces. In SGP. Google ScholarDigital Library
15. Arjun Jain, Thorsten Thormählen, Tobias Ritschel, and Hans-Peter Seidel. 2012. Exploring Shape Variations by 3D-Model Decomposition and Part-based Recombination. In Eurographics. Google ScholarDigital Library
16. Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, and Vladlen Koltun. 2012. A probabilistic model for component-based shape synthesis. ACM Trans. Graph. (SIGGRAPH) 31, 4 (2012). Google ScholarDigital Library
17. Vladislav Kraevoy, Dan Julius, and Alla Sheffer. 2007. Shuffler: Modeling with Interchangeable Parts. In Pacific Graphics.Google Scholar
18. Hamid Laga, Michela Mortara, and Michela Spagnuolo. 2013. Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes. ACM Trans. Graph. 32, 5 (2013), 150. Google ScholarDigital Library
19. Phong Le and Willem Zuidema. 2014. The Inside-Outside Recursive Neural Network model for Dependency Parsing.. In on EMNLP. 729–739.Google Scholar
20. Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. 2017. GRASS: Generative Recursive Autoencoders for Shape Structures. ACM Trans. Graph. 36, 4 (2017), 52:1–52:14. Google ScholarDigital Library
21. Niloy Mitra, Michael Wand, Hao Zhang, Daniel Cohen-Or, and Martin Bokeloh. 2013. Structure-aware shape processing. In Eurographics State-of-the-art Report (STAR).Google Scholar
22. Daniel Ritchie, Sarah Jobalia, and Anna Thomas. 2018. Example-based Authoring of Procedural Modeling Programs with Structural and Continuous Variability. Comp. Graph. For. 37, 2 (2018), 401–413.Google ScholarCross Ref
23. Adriana Schulz, Ariel Shamir, David I. W. Levin, Pitchaya Sitthi-amorn, and Wojciech Matusik. 2014. Design and Fabrication by Example. ACM Trans. Graph. 33, 4 (July 2014), 62:1–62:11. Google ScholarDigital Library
24. Andrei Sharf, Marina Blumenkrants, Ariel Shamir, and Daniel Cohen-Or. 2006. Snap-Paste: An Interactive Technique for Easy Mesh Composition. In Pacific Graphics.Google Scholar
25. Chao-Hui Shen, Hongbo Fu, Kang Chen, and Shi-Min Hu. 2012. Structure Recovery by Part Assembly. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6 (2012), 180:1–180:11. Google ScholarDigital Library
26. Richard Socher, Cliff C. Lin, Andrew Y. Ng, and Christopher D. Manning. 2011. Parsing Natural Scenes and Natural Language with Recursive Neural Networks. In ICML. Google ScholarDigital Library
27. Minhyuk Sung, Vladimir G. Kim, Roland Angst, and Leonidas Guibas. 2015. Data-driven Structural Priors for Shape Completion. ACM Trans. Graph. 34, 6 (2015). Google ScholarDigital Library
28. Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-Supervised Component Suggestions for3DModeling. ACM Trans. Graph. (SIGGRAPH Asia) (2017). Google ScholarDigital Library
29. Kenshi Takayama, Ryan Schmidt, Karan Singh, Takeo Igarashi, Tamy Boubekeur, and Olga Sorkine. 2011. GeoBrush: Interactive Mesh Geometry Cloning. Comp. Graph. For. (Eurographics) 30 (2011), 613–622.Google ScholarCross Ref
30. Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. CoRR abs/1711.00937 (2017).Google Scholar
31. Wikipedia. 2017. Kitbashing — Wikipedia, The Free Encyclopedia. (2017). https://en.wikipedia.org/wiki/Kitbashing {Online; accessed 23-December-2017}.Google Scholar
32. Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Fit and Diverse: Set Evolution for Inspiring 3D Shape Galleries. ACM Trans. Graph. 31, 4 (2012), 57:1–10. Google ScholarDigital Library
33. Youyi Zheng, Daniel Cohen-Or, Melinos Averkiou, and Niloy J Mitra. 2014. Recurring part arrangements in shape collections. Comp. Graph. For. 33, 2 (2014), 115–124. Google ScholarDigital Library
34. Youyi Zheng, Daniel Cohen-Or, and Niloy J. Mitra. 2013. Smart Variations: Functional Substructures for Part Compatibility. Comp. Graph. For. 32, 2 (2013).Google Scholar
35. Chenyang Zhu, Renjiao Yi, Wallace Lira, Ibraheem Alhashim, Kai Xu, and Hao Zhang. 2017. Deformation-driven shape correspondence via shape recognition. ACM Trans. Graph. 36, 4 (2017), 51. Google ScholarDigital Library
