“ShadowDraw: real-time user guidance for freehand drawing” by Lee, Zitnick and Cohen

  • ©Yong Jae Lee, Lawrence C. Zitnick, and Michael Cohen




    ShadowDraw: real-time user guidance for freehand drawing



    We present ShadowDraw, a system for guiding the freeform drawing of objects. As the user draws, ShadowDraw dynamically updates a shadow image underlying the user’s strokes. The shadows are suggestive of object contours that guide the user as they continue drawing. This paradigm is similar to tracing, with two major differences. First, we do not provide a single image from which the user can trace; rather ShadowDraw automatically blends relevant images from a large database to construct the shadows. Second, the system dynamically adapts to the user’s drawings in real-time and produces suggestions accordingly. ShadowDraw works by efficiently matching local edge patches between the query, constructed from the current drawing, and a database of images. A hashing technique enforces both local and global similarity and provides sufficient speed for interactive feedback. Shadows are created by aggregating the edge maps from the best database matches, spatially weighted by their match scores. We test our approach with human subjects and show comparisons between the drawings that were produced with and without the system. The results show that our system produces more realistically proportioned line drawings.


    1. Arvo, J., and Novins, K. 2000. Fluid sketches: Continuous recognition and morphing of simple hand-drawn shapes. ACM UIST. Google Scholar
    2. Ballard, D. 1981. Generalizing the hough transform to detect arbitray shapes. In Pattern Recognition, vol. 13, 111–122.Google ScholarCross Ref
    3. Beaudot, W., and Mullen, K. 2003. How long range is contour integration in human color vision. In Visual Neuroscience, vol. 15, 51–64.Google ScholarCross Ref
    4. Bhat, P., Zitnick, C. L., Cohen, M., and Curless, B. 2009. Gradientshop: A gradient-domain optimization framework for image and video filtering. TOG. Google Scholar
    5. Canny, J. 1986. A computational approach to edge detection. In TPAMI, vol. 8, 679–698. Google ScholarDigital Library
    6. Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., and Zhang, L. 2010. Mindfinder: Finding images by sketching. In ACM Multimedia International Conference.Google Scholar
    7. Carson, C., Belongie, S., Greenspan, H., and Malik, J. 2002. Blobworld: Image segmentation using expectation-maximization and its application to image querying. In TPAMI, vol. 24, 1026–1038. Google ScholarDigital Library
    8. Chalechale, A., Naghdy, G., and Mertins, A. 2005. Sketch-based image matching using angular partitioning. IEEE Trans. Systems, Man, and Cybernetics. Google Scholar
    9. Chaudhuri, S., and Koltun, V. 2010. Data-driven suggestions for creativity support in 3d modeling. ACM SIGGRAPH ASIA. Google Scholar
    10. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., and Hu, S.-M. 2009. Sketch2photo: Internet image montage. ACM SIGGRAPH ASIA. Google Scholar
    11. Chum, O., Philbin, J., Sivic, J., Isard, M., and Zisserman, A. 2007. Total recall: Automatic query expansion with a generative feature model for object retrieval. In CVPR.Google Scholar
    12. Chum, O., Philbin, J., and Zisserman, A. 2008. Near duplicate image detection: min-hash and tf-idf weighting. In BMVC.Google Scholar
    13. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? SIGGRAPH. Google Scholar
    14. Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. In ACM Computing Surveys, vol. 40, 1–60. Google ScholarDigital Library
    15. Dixon, D., Prasad, M., and Hammond, T. 2010. icandraw: Using sketch recognition and corrective feedback to assist a user in drawing human faces. ACM CHI. Google Scholar
    16. Eitz, M., Hildebrand, K., Boubekeur, T., and Alexa, M. 2009. Photosketch: A sketch based image query and compositing system. ACM SIGGRAPH – Talk Program. Google ScholarDigital Library
    17. Elder, J., and Goldberg, R. 2001. Image editing in the contour domain. In TPAMI, vol. 23, 291–296. Google ScholarDigital Library
    18. Fei-Fei, L., Fergus, R., and Perona, P. 2004. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In Workshop on Generative-Model Based Vision, CVPR. Google ScholarDigital Library
    19. Gavilan, D., Saito, S., and Nakajima, M. 2007. Sketch-to-collage. ACM SIGGRAPH – Posters. Google Scholar
    20. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM SIGGRAPH. Google Scholar
    21. Hu, R., Barnard, M., and Collomosse, J. 2010. Gradient field descriptor for sketch based retrieval and localization. ICIP.Google Scholar
    22. Igarashi, T., and Hughes, J. F. 2001. A suggestive interface for 3d drawing. ACM UIST. Google Scholar
    23. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3d freeform design. ACM SIGGRAPH. Google ScholarDigital Library
    24. Jacobs, C. E., Finkelstein, A., and Salesin, D. H. 1995. Fast multiresolution image querying. In SIGGRAPH. Google Scholar
    25. Lee, D. C., Ke, Q., and Isard, M. 2010. Partition min-hash for partial duplicate image discovery. In ECCV. Google Scholar
    26. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. IJCV. Google Scholar
    27. Nister, D., and Stewenius, H. 2006. Scalable recognition with a vocabulary tree. In CVPR. Google Scholar
    28. Sivic, J., Kaneva, B., Torralba, A., Avidan, S., and Freeman, W. 2008. Creating and exploring a large photorealistic virtual space. In Workshop on Internet Vision, CVPR.Google Scholar
    29. Winder, S., Hua, G., and Brown, M. 2009. Picking the best daisy. In CVPR.Google Scholar
    30. Witten, I. H., Moffat, A., and Bell, T. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann. Google ScholarDigital Library
    31. Zitnick, C. L. 2010. Binary coherent edge descriptors. In ECCV. Google Scholar

ACM Digital Library Publication: