“Sketch-based shape retrieval” by Eitz, Richter, Boubekeur, Hildebrand and Alexa

  • ©Mathias Eitz, Ronald Richter, Tamy Boubekeur, Kristian Hildebrand, and Marc Alexa




    Sketch-based shape retrieval



    We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.


    1. Bay, H., Tuytelaars, T., and Gool, L. J. V. 2006. SURF: Speeded up robust features. In ECCV, 404–417. Google ScholarDigital Library
    2. Bülthoff, H., and Edelman, S. 1992. Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc. National Academy of Sciences 89, 1, 60–64.Google ScholarCross Ref
    3. Candès, E. J., and Donoho, D. L. 1999. Curvelets — a surprisingly effective nonadaptive representation for objects with edges. In Int’l. Conf. Curves and Surfaces, 105–120.Google Scholar
    4. Canny, J. 1986. A computational approach to edge detection. IEEE TPAMI 8, 6, 679–698. Google ScholarDigital Library
    5. Chen, D.-Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On visual similarity based 3d model retrieval. Comput. Graph. Forum (Proc. Eurographics) 22, 3, 223–232.Google ScholarCross Ref
    6. Chen, T., Cheng, M., Tan, P., Shamir, A., and Hu, S. 2009. Sketch2Photo: internet image montage. ACM TOG (Proc. SIGGRAPH ASIA) 28, 5, 124:1–124:10. Google ScholarDigital Library
    7. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM TOG (Proc. SIGGRAPH) 27, 3, 88:1–88:11. Google ScholarDigital Library
    8. Daras, P., and Axenopoulos, A. 2010. A 3D shape retrieval framework supporting multimodal queries. IJCV 89, 2, 229–247. Google ScholarDigital Library
    9. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. ACM TOG (Proc. SIGGRAPH) 22, 3, 848–855. Google ScholarDigital Library
    10. Dutagaci, H., Cheung, C. P., and Godil, A. 2010. A benchmark for best view selection of 3D objects. In Proc. ACM workshop on 3D object retrieval, 45–50. Google ScholarDigital Library
    11. Eitz, M., Richter, R., Hildebrand, K., Boubekeur, T., and Alexa, M. 2011. Photosketcher: interactive sketch-based image synthesis. IEEE CG&A 31, 6, 56–66. Google ScholarDigital Library
    12. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. 2003. A search engine for 3D models. ACM TOG 22, 1, 83–105. Google ScholarDigital Library
    13. Hou, S., and Ramani, K. 2006. Sketch-based 3D engineering part class browsing and retrieval. In Sketch-Based Interfaces and Modeling, 131–138. Google ScholarDigital Library
    14. Jacobs, C. E., Finkelstein, A., and Salesin, D. H. 1995. Fast multiresolution image querying. In Proc. SIGGRAPH 95, 277–286. Google ScholarDigital Library
    15. Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Sketch-Based Interfaces and Modeling, 97–104. Google ScholarDigital Library
    16. Lee, Y., Zitnick, C., and Cohen, M. 2011. ShadowDraw: real-time user guidance for freehand drawing. ACM TOG (Proc. SIGGRAPH) 30, 4, 27:1–27:10. Google ScholarDigital Library
    17. Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Trans. Information Theory 28, 2, 129–137. Google ScholarDigital Library
    18. Löffler, J. 2000. Content-based retrieval of 3D models in distributed web databases by visual shape information. In Int’l. Conf. Information Visualization, 82–87. Google ScholarDigital Library
    19. Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV 60, 2, 91–110. Google ScholarDigital Library
    20. Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 3, 145–175. Google ScholarDigital Library
    21. Pu, J., Lou, K., and Ramani, K. 2005. A 2D sketch-based user interface for 3D CAD model retrieval. Computer-Aided Design and Applications 2, 6, 717–725.Google ScholarCross Ref
    22. Schölkopf, B., and Smola, A. 2002. Learning with Kernels. The MIT Press.Google Scholar
    23. Secord, A., Lu, J., Finkelstein, A., Singh, M., and Nealen, A. 2011. Perceptual models of viewpoint preference. ACM TOG 30, 5, 109:1–109:12. Google ScholarDigital Library
    24. Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., and Guo, B. 2011. Discriminative sketch-based 3D model retrieval via robust shape matching. Computer Graphics Forum (Proc. Pacific Graphics) 30, 7, 2011–2020.Google ScholarCross Ref
    25. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. In Proc. Shape Modeling International, 167–178. Google ScholarDigital Library
    26. Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In Proc. Graphics Interface, 63–70. Google ScholarDigital Library
    27. Sivic, J., and Zisserman, A. 2003. Video Google: a text retrieval approach to object matching in videos. In ICCV, 1470–1477. Google ScholarDigital Library
    28. Squire, D., Mueller, W., Mueller, H., and Raki, J. 1999. Content-based query of image databases. In Scand. Conf. on Image Analysis, 143–149.Google Scholar
    29. Tangelder, J., and Veltkamp, R. 2008. A survey of content based 3D shape retrieval methods. Multimedia Tools and Applications 39, 441–471. Google ScholarDigital Library
    30. Witten, I., Moffat, A., and Bell, T. 1999. Managing gigabytes: compressing and indexing documents and images. Morgan Kaufmann. Google ScholarDigital Library
    31. Yoon, S., Scherer, M., Schreck, T., and Kuijper, A. 2010. Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In Int’l. Conf. Multimedia, 193–200. Google ScholarDigital Library

ACM Digital Library Publication: