“Sketch classification and classification-driven analysis using Fisher vectors” by Schneider and Tuytelaars
Conference:
Type(s):
Title:
- Sketch classification and classification-driven analysis using Fisher vectors
Session/Category Title: Paintings, Sketches and Buildings
Presenter(s)/Author(s):
Abstract:
We introduce an approach for sketch classification based on Fisher vectors that significantly outperforms existing techniques. For the TU-Berlin sketch benchmark [Eitz et al. 2012a], our recognition rate is close to human performance on the same task. Motivated by these results, we propose a different benchmark for the evaluation of sketch classification algorithms. Our key idea is that the relevant aspect when recognizing a sketch is not the intention of the person who made the drawing, but the information that was effectively expressed. We modify the original benchmark to capture this concept more precisely and, as such, to provide a more adequate tool for the evaluation of sketch classification techniques. Finally, we perform a classification-driven analysis which is able to recover semantic aspects of the individual sketches, such as the quality of the drawing and the importance of each part of the sketch for the recognition.
References:
1. Barla, P., Thollot, J., and Sillion, F. X. 2005. Geometric clustering for line drawing simplification. In ACM SIGGRAPH 2005 Sketches, ACM, New York, NY, USA, SIGGRAPH ’05.
2. Cao, X., Zhang, H., Liu, S., Guo, X., and Lin, L. 2013. Sym-fish: A symmetry-aware flip invariant sketch histogram shape descriptor. In IEEE International Conference on Computer Vision (ICCV).
3. Csurka, G., Dance, C. R., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In In Workshop on Statistical Learning in Computer Vision, ECCV, 1–22.
4. Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A sketching interface for articulated figure animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’03, 320–328.
5. Donmez, N., and Singh, K. 2012. Concepture: A regular language based framework for recognizing gestures with varying and repetitive patterns. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SBIM ’12, 29–37.
6. Eitz, M., Hildebrand, K., Boubekeur, T., and Alexa, M. 2011. Sketch-based image retrieval: Benchmark and bag-of-features descriptors. IEEE Transactions on Visualization and Computer Graphics 17, 11 (Nov.), 1624–1636.
7. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 44:1–44:10.
8. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. 31, 4 (July), 31:1–31:10.
9. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. 2010. The pascal visual object classes (voc) challenge. International Journal of Computer Vision 88, 2 (June), 303–338.
10. Hammond, T., and Davis, R. 2007. Ladder, a sketching language for user interface developers. In ACM SIGGRAPH 2007 Courses, ACM, New York, NY, USA, SIGGRAPH ’07.
11. Hearst, M. A., Dumais, S., Osman, E., Platt, J., and Scholkopf, B. 1998. Support vector machines. Intelligent Systems and their Applications, IEEE 13, 4, 18–28.
12. Hoiem, D., Chodpathumwan, Y., and Dai, Q. 2012. Diagnosing error in object detectors. In Proceedings of the 12th European Conference on Computer Vision – Volume Part III, Springer-Verlag, Berlin, Heidelberg, ECCV’12, 340–353.
13. Jaakkola, T., and Haussler, D. 1998. Exploiting generative models in discriminative classifiers. In In Advances in Neural Information Processing Systems 11, MIT Press, 487–493.
14. LaViola, Jr., J. J., and Zeleznik, R. C. 2004. Mathpad2: A system for the creation and exploration of mathematical sketches. ACM Trans. Graph. 23, 3 (Aug.), 432–440.
15. Lazebnik, S., Schmid, C., and Ponce, J. 2006. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, vol. 2, 2169–2178.
16. Li, Y., Song, Y.-Z., and Gong, S. 2013. Sketch recognition by ensemble matching of structured features. In In British Machine Vision Conference (BMVC).
17. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110.
18. Olsen, L., Samavati, F. F., Sousa, M. C., and Jorge, J. A. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1, 85–103.
19. Ouyang, T. Y., and Davis, R. 2011. Chemink: A natural real-time recognition system for chemical drawings. In Proceedings of the 16th International Conference on Intelligent User Interfaces, ACM, New York, NY, USA, IUI ’11, 267–276.
20. Perronnin, F., Liu, Y., Sanchez, J., and Poirier, H. 2010. Large-scale image retrieval with compressed fisher vectors. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, 3384–3391.
21. Rubine, D. 1991. Specifying gestures by example. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH ’91, 329–337.
22. Sánchez, J., Perronnin, F., Mensink, T., and Verbeek, J. 2013. Image classification with the fisher vector: Theory and practice. International Journal of Computer Vision 105, 3, 222–245.
23. Schmidt, R., Wyvill, B., Sousa, M. C., and Jorge, J. A. 2006. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2006 Courses, ACM, New York, NY, USA, SIGGRAPH ’06.
24. Sezgin, T. M. 2001. Sketch based interfaces: Early processing for sketch understanding. In Proceedings of PUI-2001. NY, ACM Press.
25. Shesh, A., and Chen, B. 2008. Efficient and dynamic simplification of line drawings. Comput. Graph. Forum 27, 2, 537–545.Cross Ref
26. Sivic, J., and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision, vol. 2, 1470–1477.
27. Sutherland, I. E. 1964. Sketch pad a man-machine graphical communication system. In Proceedings of the SHARE Design Automation Workshop, ACM, New York, NY, USA, DAC ’64, 6.329–6.346.
28. Torralba, A., and Efros, A. A. 2011. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Washington, DC, USA, CVPR ’11, 1521–1528.
29. Vedaldi, A., and Fulkerson, B., 2008. VLFeat: An open and portable library of computer vision algorithms. http://www.vlfeat.org/.
