“Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models” by Xu, Chen, Fu, Sun and Hu
Conference:
Type:
Session Title:
- Structures, Faces & Building
Title:
- Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models
Moderator(s):
Presenter(s)/Author(s):
Abstract:
This work presents Sketch2Scene, a framework that automatically turns a freehand sketch drawing inferring multiple scene objects to semantically valid, well arranged scenes of 3D models. Unlike the existing works on sketch-based search and composition of 3D models, which typically process individual sketched objects one by one, our technique performs co-retrieval and co-placement of 3D relevant models by jointly processing the sketched objects. This is enabled by summarizing functional and spatial relationships among models in a large collection of 3D scenes as structural groups. Our technique greatly reduces the amount of user intervention needed for sketch-based modeling of 3D scenes and fits well into the traditional production pipeline involving concept design followed by 3D modeling. A pilot study indicates that it is promising to use our technique as an alternative but more efficient tool of standard 3D modeling for 3D scene construction.
References:
1. Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 4 (Apr.), 509–522. Google ScholarDigital Library
2. Chaudhuri, S., and Koltun, V. 2010. Data-driven suggestions for creativity support in 3d modeling. ACM Transactions on Graphics 29, 6, 183:1–183:9. Google ScholarDigital Library
3. Chaudhuri, S., Kalogerakis, E., Guibas, L., and Koltun, V. 2011. Probabilistic reasoning for assembly-based 3d modeling. ACM Transactions on Graphics 30, 4 (Aug.), 35:1–35:10. Google ScholarDigital Library
4. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., and Hu, S.-M. 2009. Sketch2photo: internet image montage. ACM Transactions on Graphics 28, 5, 124:1–124:10. Google ScholarDigital Library
5. Cheng, M.-M., Zhang, F.-L., Mitra, N. J., Huang, X., and Hu, S.-M. 2010. Repfinder: finding approximately repeated scene elements for image editing. ACM Transactions on Graphics 29, 4 (July), 83:1–83:8. Google ScholarDigital Library
6. Chopine, A. 2011. 3D Art Essentials: The Fundamentals of 3D Modeling, Texturing, and Animation. Focal Press. Google ScholarDigital Library
7. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. ACM Transactions on Graphics 22, 3 (July), 848–855. Google ScholarDigital Library
8. Divvala, S. K., Hoiem, D., Hays, J. H., Efros, A. A., and Hebert, M. 2009. An empirical study of context in object detection. In Proceedings of CVPR, 1271–1278.Google Scholar
9. Dutagaci, H., Cheung, C. P., and Godil, A. 2010. A benchmark for best view selection of 3d objects. In Proceedings of the ACM workshop on 3D object retrieval, ACM, New York, NY, USA, 45–50. Google ScholarDigital Library
10. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Transactions on Graphics 31, 4, 31:1–31:10. Google ScholarDigital Library
11. Fisher, M., and Hanrahan, P. 2010. Context-based search for 3d models. ACM Transactions on Graphics 29, 6 (Dec.), 182:1–182:10. Google ScholarDigital Library
12. Fisher, M., Savva, M., and Hanrahan, P. 2011. Characterizing structural relationships in scenes using graph kernels. ACM Transactions on Graphics 30, 4 (July), 34:1–34:12. Google ScholarDigital Library
13. Fisher, M., Ritchie, D., Savva, M., Funkhouser, T., and Hanrahan, P. 2012. Example-based synthesis of 3d object arrangements. ACM Transactions on Graphics 31, 6 (Nov.), 135:1–135:11. Google ScholarDigital Library
14. Funkhouser, T., Kazhdan, M., Shilane, P., Min, P., Kiefer, W., Tal, A., Rusinkiewicz, S., and Dobkin, D. 2004. Modeling by example. ACM Transactions on Graphics 23, 3, 652–663. Google ScholarDigital Library
15. Galleguillos, C., and Belongie, S. 2010. Context based object categorization: A critical survey. Computer Vision and Image Understanding 114, 6, 712–722. Google ScholarDigital Library
16. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics 26, 3, 4:1–4:7. Google ScholarDigital Library
17. Huang, Q., Koltun, V., and Guibas, L. 2011. Joint shape segmentation with linear programming. ACM Transactions on Graphics 30, 6, 125:1–125:11. Google ScholarDigital Library
18. Johnson, M., Brostow, G., Shotton, J., Arandjelovic, O., Kwatra, V., and Cipolla, R. 2006. Semantic photo synthesis. Computer Graphics Forum 25, 3, 407–413.Google ScholarCross Ref
19. Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM Transactions on Graphics 31, 4, 55:1–55:13. Google ScholarDigital Library
20. Lalonde, J., Hoiem, D., Efros, A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Transactions on Graphics 26, 3 (Aug.), 3:1–3:10. Google ScholarDigital Library
21. Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Proceedings of EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, 97–104. Google ScholarDigital Library
22. Lee, Y. J., Zitnick, C. L., and Cohen, M. F. 2011. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics 30, 27:1–27:10. Google ScholarDigital Library
23. Liu, H., Zhang, L., and Huang, H. 2012. Web-image driven best views of 3d shapes. the Visual Computer 28, 3, 279–287. Google ScholarDigital Library
24. Merrell, P., Schkufza, E., Li, Z., Agrawala, M., and Koltun, V. 2011. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics 30, 4 (July), 87:1–87:10. Google ScholarDigital Library
25. Noris, G., Sýkora, D., Shamir, A., Coros, S., Whited, B., Simmons, M., Hornung, A., Gross, M., and Sumner, R. 2012. Smart scribbles for sketch segmentation. Computer Graphics Forum 31, 8, 2516–2527. Google ScholarDigital Library
26. Olsen, L., Samavati, F., Sousa, M., and Jorge, J. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85–103. Google ScholarDigital Library
27. Russell, B., and Torralba, A. 2009. Building a database of 3d scenes from user annotations. In Proceedings of CVPR, 2711–2718.Google Scholar
28. Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2008. Labelme: a database and web-based tool for image annotation. International journal of computer vision 77, 1-3, 157–173. Google ScholarDigital Library
29. Secord, A., Lu, J., Finkelstein, A., Singh, M., and Nealen, A. 2011. Perceptual models of viewpoint preference. ACM Trans. Graph. 30, 5 (Oct.), 109:1–109:12. Google ScholarDigital Library
30. Shen, C.-H., Fu, H., Chen, K., and Hu, S.-M. 2012. Structure recovery by part assembly. ACM Transactions on Graphics 31, 6, 180:1–180:11. Google ScholarDigital Library
31. Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3-d scene prototype from freehand sketches. In Proceedings of Graphics Interface, 63–70. Google ScholarDigital Library
32. Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Transactions on Graphics 30, 126:1–126:9. Google ScholarDigital Library
33. Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Active co-analysis of a set of shapes. ACM Transactions on Graphics 31, 6 (Nov.), 165:1–165:10. Google ScholarDigital Library
34. Xie, X., Xu, K., Mitra, N. J., Cohen-Or, D., and Chen, B. 2012. Sketch-to-design: Context-based part assembly. ArXiv e-prints (Dec.).Google Scholar
35. Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: set evolution for inspiring 3d shape galleries. ACM Transactions on Graphics 31, 4, 57:1–57:10. Google ScholarDigital Library
36. Yu, L.-F., Yeung, S.-K., Tang, C.-K., Terzopoulos, D., Chan, T. F., and Osher, S. J. 2011. Make it home: automatic optimization of furniture arrangement. ACM Transactions on Graphics 30, 4 (July), 86:1–86:12. Google ScholarDigital Library
37. Yumer, M., and Kara, L. 2012. Co-abstraction of shape collections. ACM Transactions on Graphics 31, 6, 166:1–166:11. Google ScholarDigital Library
38. Zheng, Y., Fu, H., Cohen-Or, D., Au, O. K.-C., and Tai, C.-L. 2011. Component-wise controllers for structure-preserving shape manipulation. Computer Graphics Forum 30, 2, 563–572.Google ScholarCross Ref
39. Zhu, X., Jin, X., Liu, S., and Zhao, H. 2012. Analytical solutions for sketch-based convolution surface modeling on the gpu. the Visual Computer 28, 11, 1115–1125.Google Scholar
