“Adaptive synthesis of indoor scenes via activity-associated object relation graphs” by Fu, Chen, Wang, Wen, Zhou, et al. …
Conference:
Type(s):
Title:
- Adaptive synthesis of indoor scenes via activity-associated object relation graphs
Session/Category Title: 3D Scenes
Presenter(s)/Author(s):
Abstract:
We present a system for adaptive synthesis of indoor scenes given an empty room and only a few object categories. Automatically suggesting indoor objects and proper layouts to convert an empty room to a 3D scene is challenging, since it requires interior design knowledge to balance the factors like space, path distance, illumination and object relations, in order to insure the functional plausibility of the synthesized scenes. We exploit a database of 2D floor plans to extract object relations and provide layout examples for scene synthesis. With the labeled human positions and directions in each plan, we detect the activity relations and compute the coexistence frequency of object pairs to construct activity-associated object relation graphs. Given the input room and user-specified object categories, our system first leverages the object relation graphs and the database floor plans to suggest more potential object categories beyond the specified ones to make resulting scenes functionally complete, and then uses the similar plan references to create the layout of synthesized scenes. We show various synthesis results to demonstrate the practicability of our system, and validate its usability via a user study. We also compare our system with the state-of-the-art furniture layout and activity-centric scene representation methods, in terms of functional plausibility and user friendliness.
References:
1. Kang Chen, Yu-Kun Lai, Yu-Xin Wu, Ralph Martin, and Shi-Min Hu. 2014. Automatic Semantic Modeling of Indoor Scenes from Low-quality RGB-D Data Using Contextual Information. ACM Transactions on Graphics 33, 6 (2014), 208:1–208:12.
2. Kang Chen, Kun Xu, Yizhou Yu, Tian-Yi Wang, and Shi-Min Hu. 2015b. Magic Decorator: Automatic Material Suggestion for Indoor Digital Scenes. ACM Transactions on Graphics 34, 6 (2015), 232:1–232:11.
3. Xiaowu Chen, Jianwei Li, Qing Li, Bo Gao, Dongqing Zou, and Qinping Zhao. 2015a. Image2Scene: Transforming Style of 3D Room. In Proceedings of the 23rd Annual ACM Conference on Multimedia Conference. ACM, 321–330.
4. Matthew Fisher and Pat Hanrahan. 2010. Context-based Search for 3D Models. ACM Transactions on Graphics 29, 6 (2010), 182:1–182:10.
5. Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan. 2012. Example-based Synthesis of 3D Object Arrangements. ACM Transactions on Graphics 31, 6 (2012), 135:1–135:11.
6. Matthew Fisher, Manolis Savva, and Pat Hanrahan. 2011. Characterizing Structural Relationships in Scenes Using Graph Kernels. ACM Transactions on Graphics 30, 4 (2011), 34:1–34:12.
7. Matthew Fisher, Manolis Savva, Yangyan Li, Pat Hanrahan, and Matthias Nießner. 2015. Activity-centric Scene Synthesis for Functional 3D Scene Modeling. ACM Transactions on Graphics 34, 6 (2015).
8. H. Grabner, J. Gall, and L. Van Gool. 2011. What makes a chair a chair?. In IEEE Conference on Computer Vision and Pattern Recognition. 1529–1536.
9. Ruizhen Hu, Chenyang Zhu, Oliver van Kaick, Ligang Liu, Ariel Shamir, and Hao Zhang. 2015. Interaction Context (ICON): Towards a Geometric Functionality Descriptor. ACM Transactions on Graphics 34, 4 (2015), 83:1–83:12.
10. Yun Jiang, Marcus Lim, and Ashutosh Saxena. 2012. Learning Object Arrangements in 3D Scenes using Human Context. In International Conference on Machine Learning.
11. Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas, and Thomas Funkhouser. 2014. Shape2Pose: Human-centric Shape Analysis. ACM Transactions on Graphics 33, 4 (2014), 120:1–120:12.
12. Tianqiang Liu, Siddhartha Chaudhuri, Vladimir G. Kim, Qixing Huang, Niloy J. Mitra, and Thomas Funkhouser. 2014. Creating Consistent Scene Graphs Using a Probabilistic Grammar. ACM Transactions on Graphics 33, 6 (2014), 211:1–211:12.
13. Rui Ma, Honghua Li, Changqing Zou, Zicheng Liao, Xin Tong, and Hao Zhang. 2016. Action-Driven 3D Indoor Scene Evolution. ACM Transactions on Graphics 35, 6 (2016), 173:1–173:13.
14. Paul Merrell, Eric Schkufza, Zeyang Li, Maneesh Agrawala, and Vladlen Koltun. 2011. Interactive Furniture Layout Using Interior Design Guidelines. ACM Transactions on Graphics 30, 4 (2011), 87.
15. Chi-Han Peng, Yong-Liang Yang, and Peter Wonka. 2014. Computing Layouts with Deformable Templates. ACM Transactions on Graphics 33, 4 (2014), 99:1–99:11.
16. Manolis Savva, Angel X. Chang, Pat Hanrahan, Matthew Fisher, and Matthias Nießner. 2014. SceneGrok: Inferring Action Maps in 3D Environments. ACM Transactions on Graphics 33, 6 (2014), 212:1–212:10.
17. Manolis Savva, Angel X. Chang, Pat Hanrahan, Matthew Fisher, and Matthias Nießner. 2016. PiGraphs: Learning Interaction Snapshots from Observations. ACM Transactions on Graphics (2016), 5:1–5:2.
18. Tianjia Shao, Aron Monszpart, Youyi Zheng, Bongjin Koo, Weiwei Xu, Kun Zhou, and Niloy J. Mitra. 2014. Imagining the Unseen: Stability-based Cuboid Arrangements for Scene Understanding. ACM Transactions on Graphics 33, 6 (2014), 209:1–209:11.
19. Zhige Xie, Yueshan Xiong, and Kai Xu. 2014. AB3D: action-based 3D descriptor for shape analysis. The Visual Computer 30, 6–8 (2014), 591–601.
20. Kun Xu, Kang Chen, Hongbo Fu, Wei-Lun Sun, and Shi-Min Hu. 2013. Sketch2Scene: Sketch-based Co-retrieval and Co-placement of 3D Models. ACM Transactions on Graphics 32, 4 (2013), 123:1–123:15.
21. Kai Xu, Hui Huang, Yifei Shi, Hao Li, Pinxin Long, Jianong Caichen, Wei Sun, and Baoquan Chen. 2015. Autoscanning for Coupled Scene Reconstruction and Proactive Object Analysis. ACM Transactions on Graphics 34, 6 (2015), 177:1–177:14.
22. Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu, Ariel Shamir, Daniel Cohen-Or, and Hui Huang. 2014. Organizing Heterogeneous Scene Collections Through Contextual Focal Points. ACM Transactions on Graphics 33, 4 (2014), 35:1–35:12.
23. Yong-Liang Yang, Jun Wang, Etienne Vouga, and Peter Wonka. 2013. Urban Pattern: Layout Design by Hierarchical Domain Splitting. ACM Transactions on Graphics 32, 6 (2013), 181:1–181:12.
24. Lap-Fai Yu, Sai Kit Yeung, Chi-Keung Tang, Demetri Terzopoulos, Tony F. Chan, and Stanley Osher. 2011. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics 30, 4 (2011), 86.
25. Y. Zheng, H. Liu, J. Dorsey, and N. Mitra. 2016. Ergonomics-inspired Reshaping and Exploration of Collections of Models. IEEE Transactions on Visualization and Computer Graphics 22, 6 (2016), 1732–1744.
