“Tracing versus freehand for evaluating computer-generated drawings” by Wang, Qju, Feng, Rushmeier, McMillan, et al. …

  • ©Zeyu Wang, Sherry Qju, Nicole Feng, Holly E. Rushmeier, Leonard McMillan, and Julie Dorsey

Conference:


Type:


Title:

    Tracing versus freehand for evaluating computer-generated drawings

Presenter(s)/Author(s):


Abstract:


    Non-photorealistic rendering (NPR) and image processing algorithms are widely assumed as a proxy for drawing. However, this assumption is not well assessed due to the difficulty in collecting and registering freehand drawings. Alternatively, tracings are easier to collect and register, but there is no quantitative evaluation of tracing as a proxy for freehand drawing. In this paper, we compare tracing, freehand drawing, and computer-generated drawing approximation (CGDA) to understand their similarities and differences. We collected a dataset of 1,498 tracings and freehand drawings by 110 participants for 100 image prompts. Our drawings are registered to the prompts and include vector-based timestamped strokes collected via stylus input. Comparing tracing and freehand drawing, we found a high degree of similarity in stroke placement and types of strokes used over time. We show that tracing can serve as a viable proxy for freehand drawing because of similar correlations between spatio-temporal stroke features and labeled stroke types. Comparing hand-drawn content and current CGDA output, we found that 60% of drawn pixels corresponded to computer-generated pixels on average. The overlap tended to be commonly drawn content, but people’s artistic choices and temporal tendencies remained largely uncaptured. We present an initial analysis to inform new CGDA algorithms and drawing applications, and provide the dataset for use by the community.

References:


    1. Pablo Arbeláez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 5 (May 2011), 898–916.Google ScholarDigital Library
    2. Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. 2013. Style and Abstraction in Portrait Sketching. ACM Trans. Graph. 32, 4, Article 55 (July 2013), 12 pages.Google ScholarDigital Library
    3. John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 6 (Nov 1986), 679–698.Google Scholar
    4. Forrester Cole, Aleksey Golovinskiy, Alex Limpaecher, Heather Stoddart Barros, Adam Finkelstein, Thomas Funkhouser, and Szymon Rusinkiewicz. 2008. Where Do People Draw Lines? ACM Trans. Graph. 27, 3, Article 88 (Aug 2008), 11 pages.Google ScholarDigital Library
    5. Richard D. De Veaux, Paul F. Velleman, and David E. Bock. 2020. Stats: Data and Models, 5th Edition. Pearson.Google Scholar
    6. Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive Contours for Conveying Shape. ACM Trans. Graph. 22, 3 (July 2003), 848–855.Google ScholarDigital Library
    7. Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, and Adrien Bousseau. 2018. 3D Sketching Using Multi-View Deep Volumetric Prediction. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 21 (July 2018), 22 pages.Google ScholarDigital Library
    8. Betty Edwards. 2012. Drawing on the Right Side of the Brain. Penguin Group.Google Scholar
    9. Mathias Eitz, James Hays, and Marc Alexa. 2012. How Do Humans Sketch Objects? ACM Trans. Graph. 31, 4, Article 44 (July 2012), 10 pages.Google ScholarDigital Library
    10. Lubin Fan, Ruimin Wang, Linlin Xu, Jiansong Deng, and Ligang Liu. 2013. Modeling by Drawing with Shadow Guidance. Computer Graphics Forum 32, 7 (2013), 157–166.Google ScholarCross Ref
    11. Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J. Mitra. 2011. Animated Construction of Line Drawings. ACM Trans. Graph. 30, 6, Article 133 (Dec 2011), 10 pages.Google ScholarDigital Library
    12. Emma Gowen and R. Chris Miall. 2006. Eye-Hand Interactions in Tracing and Drawing Tasks. Human Movement Science 25, 4(2006), 568–585.Google ScholarCross Ref
    13. Emma Gowen and R. Chris Miall. 2007. Differentiation Between External and Internal Cuing: An fMRI Study Comparing Tracing with Drawing. NeuroImage 36, 2 (2007), 396–410.Google ScholarCross Ref
    14. Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia Pont, Frédo Durand, and Adrien Bousseau. 2019. OpenSketch: A Richly-Annotated Dataset of Product Design Sketches. ACM Trans. Graph. 38, 6, Article 232 (Nov 2019), 16 pages.Google ScholarDigital Library
    15. David Ha and Douglas Eck. 2018. A Neural Representation of Sketch Drawings. In Proceedings of the International Conference on Learning Representations.Google Scholar
    16. Xiaoguang Han, Chang Gao, and Yizhou Yu. 2017. DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature Modeling. ACM Trans. Graph. 36, 4, Article 126 (July 2017), 12 pages.Google ScholarDigital Library
    17. Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2017. Shape Synthesis from Sketches via Procedural Models and Convolutional Networks. IEEE Transactions on Visualization and Computer Graphics 23, 8 (Aug 2017), 2003–2013.Google ScholarDigital Library
    18. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-To-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
    19. David Jamieson. 2019. Why Learn to Draw When You Can Trace? https://vitruvianstudio.com/why-learn-to-draw-when-you-can-trace/. Accessed Aug 9, 2020.Google Scholar
    20. Christian Mosbæk Johannessen and Theo Van Leeuwen. 2017. The Materiality of Writing: A Trace Making Perspective. Routledge.Google Scholar
    21. Tilke Judd, Frédo Durand, and Edward Adelson. 2007. Apparent Ridges for Line Drawing. ACM Trans. Graph. 26, 3, Article 19 (July 2007), 8 pages.Google ScholarDigital Library
    22. Evangelos Kalogerakis, Derek Nowrouzezahrai, Simon Breslav, and Aaron Hertzmann. 2012. Learning Hatching for Pen-and-Ink Illustration of Surfaces. ACM Trans. Graph. 31, 1, Article 1 (Feb 2012), 17 pages.Google ScholarDigital Library
    23. Kitware. 2020. Insight Toolkit. https://itk.org/. Accessed Oct 20, 2020.Google Scholar
    24. Yong Jae Lee, C. Lawrence Zitnick, and Michael F. Cohen. 2011. ShadowDraw: Real-Time User Guidance for Freehand Drawing. ACM Trans. Graph. 30, 4, Article 27 (July 2011), 10 pages.Google ScholarDigital Library
    25. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling. ACM Trans. Graph. 37, 6, Article 238 (Dec 2018), 12 pages.Google ScholarDigital Library
    26. Mengtian Li, Zhe Lin, Radomir Mech, Ersin Yumer, and Deva Ramanan. 2019. Photo-Sketching: Inferring Contour Drawings from Images. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. 1403–1412.Google ScholarCross Ref
    27. Alex Limpaecher, Nicolas Feltman, Adrien Treuille, and Michael Cohen. 2013. Real-Time Drawing Assistance through Crowdsourcing. ACM Trans. Graph. 32, 4, Article 54 (July 2013), 8 pages.Google ScholarDigital Library
    28. Difan Liu, Mohamed Nabail, Aaron Hertzmann, and Evangelos Kalogerakis. 2020. Neural Contours: Learning to Draw Lines from 3D Shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
    29. Jingbo Liu, Hongbo Fu, and Chiew-Lan Tai. 2014. Dynamic Sketching: Simulating the Process of Observational Drawing. In Proceedings of the Workshop on Computational Aesthetics. ACM, New York, NY, USA, 15–22.Google ScholarDigital Library
    30. Cewu Lu, Li Xu, and Jiaya Jia. 2012. Combining Sketch and Tone for Pencil Drawing Production. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering. Eurographics Association, Goslar, Germany, 65–73.Google Scholar
    31. Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks. In Proceedings of the International Conference on 3D Vision. 67–77.Google ScholarCross Ref
    32. James M Mahoney. 2018. The V-Sketch System: Machine Assisted Design Exploration in Virtual Reality. Master’s thesis. Cornell University.Google Scholar
    33. Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel. 2004. Ridge-Valley Lines on Meshes via Implicit Surface Fitting. ACM Trans. Graph. 23, 3 (Aug 2004), 609–612.Google ScholarDigital Library
    34. Justin Ostrofsky, Aaron Kozbelt, and Angelika Seidel. 2012. Perceptual Constancies and Visual Selection as Predictors of Realistic Drawing Skill. Psychology of Aesthetics, Creativity, and the Arts 6, 2 (2012), 124.Google ScholarCross Ref
    35. Emil Praun, Hugues Hoppe, Matthew Webb, and Adam Finkelstein. 2001. Real-time Hatching. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, USA, 579–584.Google Scholar
    36. Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. 2016. The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies. ACM Trans. Graph. 35, 4, Article 119 (July 2016), 12 pages.Google ScholarDigital Library
    37. Boxin Shi, Zhe Wu, Zhipeng Mo, Dinglong Duan, Sai-Kit Yeung, and Ping Tan. 2016. A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3707–3716.Google ScholarCross Ref
    38. Miroslava Slavcheva, Wadim Kehl, Nassir Navab, and Slobodan Ilic. 2018. SDF-2-SDF Registration for Real-Time 3D Reconstruction from RGB-D Data. International Journal of Computer Vision 126, 6 (June 2018), 615–636.Google ScholarDigital Library
    39. Qingkun Su, Wing Ho Andy Li, Jue Wang, and Hongbo Fu. 2014. EZ-Sketching: Three-Level Optimization for Error-Tolerant Image Tracing. ACM Trans. Graph. 33, 4, Article 54 (July 2014), 9 pages.Google ScholarDigital Library
    40. Wanchao Su, Dong Du, Xin Yang, Shizhe Zhou, and Hongbo Fu. 2018. Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 22 (July 2018), 17 pages.Google ScholarDigital Library
    41. Masaki Suwa and Barbara Tversky. 1997. What Do Architects and Students Perceive in Their Design Sketches? A Protocol Analysis. Design Studies 18, 4 (1997), 385–403.Google ScholarCross Ref
    42. Corey Toler-Franklin, Adam Finkelstein, and Szymon Rusinkiewicz. 2007. Illustration of Complex Real-World Objects Using Images with Normals. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering. ACM, New York, NY, USA, 111-119.Google ScholarDigital Library
    43. Holger Winnemöller, Jan Eric Kyprianidis, and Sven C. Olsen. 2012. XDoG: An Extended Difference-of-Gaussians Compendium Including Advanced Image Stylization. Computers & Graphics 36, 6 (2012), 740–753.Google ScholarDigital Library
    44. Saining Xie and Zhuowen Tu. 2015. Holistically-Nested Edge Detection. In Proceedings of the IEEE International Conference on Computer Vision. 1395–1403.Google ScholarDigital Library
    45. Ying Xiong, Ayan Chakrabarti, Ronen Basri, Steven J. Gortler, David W. Jacobs, and Todd Zickler. 2015. From Shading to Local Shape. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1 (Jan 2015), 67–79.Google ScholarCross Ref
    46. Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A Benchmark for Rough Sketch Cleanup. ACM Trans. Graph. 39, 6, Article 163 (Nov 2020), 14 pages.Google ScholarDigital Library
    47. Evan You. 2020. Vue.js. https://vuejs.org/. Accessed Oct 20, 2020.Google Scholar
    48. Li Zhang. 2012. Photometric Stereo. http://pages.cs.wisc.edu/~lizhang/courses/cs766-2012f/projects/phs/index.htm. Accessed Apr 12, 2019.Google Scholar

ACM Digital Library Publication:


Overview Page: