“Data-driven segmentation and labeling of freehand sketches” by Huang, Fu and Lau
Conference:
Type(s):
Title:
- Data-driven segmentation and labeling of freehand sketches
Session/Category Title: Paintings, Sketches and Buildings
Presenter(s)/Author(s):
Abstract:
We present a data-driven approach to derive part-level segmentation and labeling of free-hand sketches, which depict single objects with multiple parts. Our method performs segmentation and labeling simultaneously, by inferring a structure that best fits the input sketch, through selecting and connecting 3D components in the database. The problem is formulated using Mixed Integer Programming, which optimizes over both the local fitness of the selected components and the global plausibility of the connected structure. Evaluations show that our algorithm is significantly better than the straightforward approaches based on direct retrieval or part assembly, and can effectively handle challenging variations in the sketch.
References:
1. Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. IEEE TPAMI 24, 4, 509–522.
2. Chaudhuri, S., Kalogerakis, E., Guibas, L., and Koltun, V. 2011. Probabilistic reasoning for assembly-based 3D modeling. ACM TOG 30.
3. Chen, T., Cheng, M., Tan, P., Shamir, A., and Hu, S. 2009. Sketch2photo: internet image montage. ACM TOG 28, 5.
4. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM TOG 31, 4.
5. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM TOG 31, 4.
6. Fish, N., Averkiou, M., van Kaick, O., Sorkine-Hornung, O., Cohen-Or, D., and Mitra, N. 2014. Meta-representation of shape families. ACM TOG 33, 4.
7. Funkhouser, T., Kazhdan, M., Shilane, P., Min, P., Kiefer, W., Tal, A., Rusinkiewicz, S., and Dobkin, D. 2004. Modeling by example. ACM TOG 23, 3.
8. Galleguillos, C., and Belongie, S. 2010. Context based object categorization: A critical survey. Computer Vision and Image Understanding 114, 6, 712–722.
9. Gennari, L., Kara, L., Stahovich, T., and Shimada, K. 2005. Combining geometry and domain knowledge to interpret hand-drawn diagrams. Computers & Graphics 29, 4, 547–562.
10. He, X., Zemel, R., and Carreira-Perpinán, M. 2004. Multiscale conditional random fields for image labeling. In Proc. IEEE CVPR, vol. 2, 695–702.
11. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D mesh segmentation and labeling. TOG 29, 4.
12. Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM TOG 31, 4.
13. Kraevoy, V., Sheffer, A., and van de Panne, M. 2009. Modeling from contour drawings. In Proc. SBIM, 37–44.
14. LaViola, J., Davis, R., and Igarashi, T. 2006. An introduction to sketch-based interfaces. ACM SIGGRAPH Course Notes.
15. Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Proc. SBIM.
16. Noris, G., Sykora, D., Shamir, A., Coros, S., Whited, B., Simmons, M., Hornung, A., Gross, M., and Sumner, R. 2012. Smart scribbles for sketch segmentation. Computer Graphics Forum.
17. Olsen, L., Samavati, F., Sousa, M., and Jorge, J. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85–103.
18. Pusch, R., Samavati, F., Nasri, A., and Wyvill, B. 2007. Improving the sketch-based interface. The Visual Computer 23, 9–11 (Sept.), 955–962.
19. Ren, X., Bo, L., and Fox, D. 2012. Rgb-(d) scene labeling: Features and algorithms. In Proc. IEEE CVPR.
20. Rivers, A., Durand, F., and Igarashi, T. 2010. 3D modeling with silhouettes. ACM TOG 29, 4.
21. Secord, A., Lu, J., Finkelstein, A., Singh, M., and Nealen, A. 2011. Perceptual models of viewpoint preference. ACM TOG 30, 5.
22. Sezgin, T., Stahovich, T., and Davis, R. 2001. Sketch based interfaces: Early processing for sketch understanding. In Proc. PUI.
23. Shao, T., Xu, W., Zhou, K., Wang, J., Li, D., and Guo, B. 2012. An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM TOG 31, 6.
24. Shen, C., Fu, H., Chen, K., and Hu, S. 2012. Structure recovery by part assembly. ACM TOG 31, 6.
25. Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In Proc. Graphics Interface, 63–70.
26. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In Proc. ECCV.
27. Sun, Z., Wang, C., Zhang, L., and Zhang, L. 2012. Free hand-drawn sketch segmentation. In Proc. ECCV. 626–639.
28. Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Active co-analysis of a set of shapes. ACM TOG 31, 6.
29. Xie, X., Xu, K., Mitra, N., Cohen-Or, D., Su, Q., Gong, W., and Chen, B. 2013. Sketch-to-design: Context-based part assembly. Computer Graphics Forum.
30. Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3D object modeling. ACM TOG 30.
31. Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: set evolution for inspiring 3D shape galleries. ACM TOG 31, 4.
32. Xu, K., Chen, K., Fu, H., Sun, W., and Hu, S. 2013. Sketch2scene: Sketch-based co-retrieval and co-placement of 3D models. ACM TOG 32, 4.
