“Interpreting concept sketches” by Shao, Li, Zhou, Xu, Guo, et al. …

  • ©Tianjia Shao, Wilmot Li, Kun Zhou, Weiwei Xu, Baining Guo, and Niloy J. Mitra




    Interpreting concept sketches

Session/Category Title: Line Drawing




    Concept sketches are popularly used by designers to convey pose and function of products. Understanding such sketches, however, requires special skills to form a mental 3D representation of the product geometry by linking parts across the different sketches and imagining the intermediate object configurations. Hence, the sketches can remain inaccessible to many, especially non-designers. We present a system to facilitate easy interpretation and exploration of concept sketches. Starting from crudely specified incomplete geometry, often inconsistent across the different views, we propose a globally-coupled analysis to extract part correspondence and inter-part junction information that best explain the different sketch views. The user can then interactively explore the abstracted object to gain better understanding of the product functions. Our key technical contribution is performing shape analysis without access to any coherent 3D geometric model by reasoning in the space of inter-part relations. We evaluate our system on various concept sketches obtained from popular product design books and websites.


    1. Bae, S.-H., Balakrishnan, R., and Singh, K. 2008. ILoveSketch: as-natural-as-possible sketching system for creating 3D curve models. In UIST, 151–160. Google ScholarDigital Library
    2. Chen, X., Kang, S. B., Xu, Y.-Q., Dorsey, J., and Shum, H.-Y. 2008. Sketching reality: Realistic interpretation of architectural designs. ACM TOG 27, 2, 11:1–11:15. Google ScholarDigital Library
    3. Fidler, S., Dickinson, S., and Urtasun, R. 2012. 3d object detection and viewpoint estimation with a deformable 3d cuboid model. In NIPS, 620–628.Google Scholar
    4. Gal, R., Sorkine, O., Mitra, N. J., and Cohen-Or, D. 2009. iwires: an analyze-and-edit approach to shape manipulation. 33:1–33:10. Google ScholarDigital Library
    5. Gupta, A., Satkin, S., Efros, A. A., and Hebert, M. 2011. From 3D scene geometry to human workspace. In CVPR, 1961–1968. Google ScholarDigital Library
    6. Haller, L., and Cullen, C. D. 2003. Design Secrets: Products 1 and 2: 50 Real-Life Product Design Projects Uncovered. Rockport Publishers.Google Scholar
    7. Hartley, A., and Zisserman, A. 2006. Multiple view geometry in computer vision (2. Ed.). Cambridge University Press. Google ScholarDigital Library
    8. Heiser, J., Tversky, B., and Silverman, M. 2004. Sketches for and from collaboration. Sydney: Key Centre for Design.Google Scholar
    9. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: a sketching interface for 3d freeform design. In Proc. of SIGGRAPH, 409–416. Google ScholarDigital Library
    10. Karpenko, O. A., and Hughes, J. F. 2006. SmoothSketch: 3D free-form shapes from complex sketches. Proc. ACM SIGGRAPH 25, 3. Google ScholarDigital Library
    11. Lee, S., Feng, D., and Gooch, B. 2008. Automatic construction of 3D models from architectural line drawings. In Proc. I3D, 123–130. Google ScholarDigital Library
    12. Leordeanu, M., and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In ICCV, 1482–1489. Google ScholarDigital Library
    13. Mitra, N. J., Pauly, M., Wand, M., and Ceylan, D. 2012. Symmetry in 3D geometry: Extraction and applications. In EUROGRAPHICS State-of-the-art Report.Google Scholar
    14. Mitra, N. J., Wand, M., Zhang, H., Cohen-Or, D., and Bokeloh, M. 2013. Structure-aware shape processing. In EUROGRAPHICS State-of-the-art Report.Google Scholar
    15. Mitra, N. J., Yang, Y.-L., Yan, D.-M., Li, W., and Agrawala, M. 2013. Illustrating how mechanical assemblies work. Comm. of the ACM (Research Highlight) 56, 1, 106–114. Google ScholarDigital Library
    16. Olofsson, E., and Sjolen, K. 2005. Design Sketching. Keeos Design Books.Google Scholar
    17. Schmidt, R., Khan, A., Singh, K., and Kurtenbach, G. 2009. Analytic drawing of 3D scaffolds. ACM TOG (SIGGRAPH Asia) 28, 5, 149:1–149:10. Google ScholarDigital Library
    18. Steur, R., and Eissen, K. 2007. Sketching: Drawing techniques for product designers. BIS Publishers.Google Scholar
    19. Suwa, M., and Tversky, B. 2009. Thinking with sketches. Oxford University Press.Google Scholar
    20. Xu, W., Wang, J., Yin, K., Zhou, K., van de Panne, M., Chen, F., and Guo, B. 2009. Joint-aware manipulation of deformable models. Proc. ACM SIGGRAPH 28, 3, 35:1–35:9. Google ScholarDigital Library
    21. Zeleznik, R. C., Herndon, K. P., and Hughes, J. F. 1996. SKETCH: an interface for sketching 3D scenes. In Proc. ACM SIGGRAPH. Google ScholarDigital Library
    22. Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: cuboid proxies for smart image manipulation. 99:1–99:11.Google Scholar
    23. Zhu, B., Iwata, M., Haraguchi, R., Ashihara, T., Umetani, N., Igarashi, T., and Nakazawa, K. 2011. Sketch-based dynamic illustration of fluid systems. In ACM TOG (SIGGRAPH Asia), 134:1–134:8. Google ScholarDigital Library

ACM Digital Library Publication:

Overview Page: