“ShadowDraw: real-time user guidance for freehand drawing” by Lee, Zitnick and Cohen

We present ShadowDraw, a system for guiding the freeform drawing of objects. As the user draws, ShadowDraw dynamically updates a shadow image underlying the user’s strokes. The shadows are suggestive of object contours that guide the user as they continue drawing. This paradigm is similar to tracing, with two major differences. First, we do not provide a single image from which the user can trace; rather ShadowDraw automatically blends relevant images from a large database to construct the shadows. Second, the system dynamically adapts to the user’s drawings in real-time and produces suggestions accordingly. ShadowDraw works by efficiently matching local edge patches between the query, constructed from the current drawing, and a database of images. A hashing technique enforces both local and global similarity and provides sufficient speed for interactive feedback. Shadows are created by aggregating the edge maps from the best database matches, spatially weighted by their match scores. We test our approach with human subjects and show comparisons between the drawings that were produced with and without the system. The results show that our system produces more realistically proportioned line drawings.

1. Arvo, J., and Novins, K. 2000. Fluid sketches: Continuous recognition and morphing of simple hand-drawn shapes. ACM UIST. Google Scholar
2. Ballard, D. 1981. Generalizing the hough transform to detect arbitray shapes. In Pattern Recognition, vol. 13, 111–122.Google ScholarCross Ref
3. Beaudot, W., and Mullen, K. 2003. How long range is contour integration in human color vision. In Visual Neuroscience, vol. 15, 51–64.Google ScholarCross Ref
4. Bhat, P., Zitnick, C. L., Cohen, M., and Curless, B. 2009. Gradientshop: A gradient-domain optimization framework for image and video filtering. TOG. Google Scholar
5. Canny, J. 1986. A computational approach to edge detection. In TPAMI, vol. 8, 679–698. Google ScholarDigital Library
6. Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., and Zhang, L. 2010. Mindfinder: Finding images by sketching. In ACM Multimedia International Conference.Google Scholar
7. Carson, C., Belongie, S., Greenspan, H., and Malik, J. 2002. Blobworld: Image segmentation using expectation-maximization and its application to image querying. In TPAMI, vol. 24, 1026–1038. Google ScholarDigital Library
8. Chalechale, A., Naghdy, G., and Mertins, A. 2005. Sketch-based image matching using angular partitioning. IEEE Trans. Systems, Man, and Cybernetics. Google Scholar
9. Chaudhuri, S., and Koltun, V. 2010. Data-driven suggestions for creativity support in 3d modeling. ACM SIGGRAPH ASIA. Google Scholar
10. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., and Hu, S.-M. 2009. Sketch2photo: Internet image montage. ACM SIGGRAPH ASIA. Google Scholar
11. Chum, O., Philbin, J., Sivic, J., Isard, M., and Zisserman, A. 2007. Total recall: Automatic query expansion with a generative feature model for object retrieval. In CVPR.Google Scholar
12. Chum, O., Philbin, J., and Zisserman, A. 2008. Near duplicate image detection: min-hash and tf-idf weighting. In BMVC.Google Scholar
13. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? SIGGRAPH. Google Scholar
14. Datta, R., Joshi, D., Li, J., and Wang, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. In ACM Computing Surveys, vol. 40, 1–60. Google ScholarDigital Library
15. Dixon, D., Prasad, M., and Hammond, T. 2010. icandraw: Using sketch recognition and corrective feedback to assist a user in drawing human faces. ACM CHI. Google Scholar
16. Eitz, M., Hildebrand, K., Boubekeur, T., and Alexa, M. 2009. Photosketch: A sketch based image query and compositing system. ACM SIGGRAPH – Talk Program. Google ScholarDigital Library
17. Elder, J., and Goldberg, R. 2001. Image editing in the contour domain. In TPAMI, vol. 23, 291–296. Google ScholarDigital Library
18. Fei-Fei, L., Fergus, R., and Perona, P. 2004. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In Workshop on Generative-Model Based Vision, CVPR. Google ScholarDigital Library
19. Gavilan, D., Saito, S., and Nakajima, M. 2007. Sketch-to-collage. ACM SIGGRAPH – Posters. Google Scholar
20. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM SIGGRAPH. Google Scholar
21. Hu, R., Barnard, M., and Collomosse, J. 2010. Gradient field descriptor for sketch based retrieval and localization. ICIP.Google Scholar
22. Igarashi, T., and Hughes, J. F. 2001. A suggestive interface for 3d drawing. ACM UIST. Google Scholar
23. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3d freeform design. ACM SIGGRAPH. Google ScholarDigital Library
24. Jacobs, C. E., Finkelstein, A., and Salesin, D. H. 1995. Fast multiresolution image querying. In SIGGRAPH. Google Scholar
25. Lee, D. C., Ke, Q., and Isard, M. 2010. Partition min-hash for partial duplicate image discovery. In ECCV. Google Scholar
26. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. IJCV. Google Scholar
27. Nister, D., and Stewenius, H. 2006. Scalable recognition with a vocabulary tree. In CVPR. Google Scholar
28. Sivic, J., Kaneva, B., Torralba, A., Avidan, S., and Freeman, W. 2008. Creating and exploring a large photorealistic virtual space. In Workshop on Internet Vision, CVPR.Google Scholar
29. Winder, S., Hua, G., and Brown, M. 2009. Picking the best daisy. In CVPR.Google Scholar
30. Witten, I. H., Moffat, A., and Bell, T. 1999. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann. Google ScholarDigital Library
31. Zitnick, C. L. 2010. Binary coherent edge descriptors. In ECCV. Google Scholar