“StrokeAggregator: consolidating raw sketches into artist-intended curve drawings” by Liu, Rosales and Sheffer

  • ©Chenxi Liu, Enrique Rosales, and Alla Sheffer



Entry Number: 97


    StrokeAggregator: consolidating raw sketches into artist-intended curve drawings


Session Title: Sketching



    When creating line drawings, artists frequently depict intended curves using multiple, tightly clustered, or overdrawn, strokes. Given such sketches, human observers can readily envision these intended, aggregate, curves, and mentally assemble the artist’s envisioned 2D imagery. Algorithmic stroke consolidation—replacement of overdrawn stroke clusters by corresponding aggregate curves—can benefit a range of sketch processing and sketch-based modeling applications which are designed to operate on consolidated, intended curves. We propose StrokeAggregator, a novel stroke consolidation method that significantly improves on the state of the art, and produces aggregate curve drawings validated to be consistent with viewer expectations. Our framework clusters strokes into groups that jointly define intended aggregate curves by leveraging principles derived from human perception research and observation of artistic practices. We employ these principles within a coarse-to-fine clustering method that starts with an initial clustering based on pairwise stroke compatibility analysis, and then refines it by analyzing interactions both within and in-between clusters of strokes. We facilitate this analysis by computing a common 1D parameterization for groups of strokes via common aggregate curve fitting. We demonstrate our method on a large range of line drawings, and validate its ability to generate consolidated drawings that are consistent with viewer perception via qualitative user evaluation, and comparisons to manually consolidated drawings and algorithmic alternatives.


    1. Rahul Arora, Ishan Darolia, Vinay P. Namboodiri, Karan Singh, and Adrien Bousseau. 2017. SketchSoup: Exploratory Ideation Using Design Sketches. Computer Graphics Forum (2017).Google Scholar
    2. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In Proc. UIST. 151–160. Google ScholarDigital Library
    3. Nikhil Bansal, Avrim Blum, and Shuchi Chawla. 2004. Correlation clustering. Machine Learning 56, 1–3 (2004), 89–113. Google ScholarDigital Library
    4. Bin Bao and Hongbo Fu. 2012. Vectorizing line drawings with near-constant line width. In Proc. International Conference on Image Processing. 805–808.Google ScholarCross Ref
    5. Ilya Baran, Jaakko Lehtinen, and Jovan Popović. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, Vol. 29. 655–664.Google ScholarCross Ref
    6. Pascal Barla, Joëlle Thollot, and François Sillion. 2005. Geometric Clustering for Line Drawing Simplification. In Proc. EGSR. Google ScholarDigital Library
    7. A. Bartolo, K. P. Camilleri, S. G. Fabri, J. C. Borg, and P. J. Farrugia. 2007. Scribbles to Vectors: Preparation of Scribble Drawings for CAD Interpretation. In Sketch-Based Interfaces and Modeling. Google ScholarDigital Library
    8. Thomas Baudel. 1994. A Mark-based Interaction Paradigm for Free-hand Drawing. In Proc. UIST. 185–192. Google ScholarDigital Library
    9. Mikhail Bessmeltsev, Will Chang, Nicholas Vining, Alla Sheffer, and Karan Singh. 2015. Modeling character canvases from cartoon drawings. ACM Trans. Graph. 34, 5 (2015), 162. Google ScholarDigital Library
    10. Mikhail Bessmeltsev, Nicholas Vining, and Alla Sheffer. 2016. Gesture3D: Posing 3D Characters via Gesture Drawings. ACM Trans. Graph. 35, 6 (2016). Google ScholarDigital Library
    11. Pengbo Bo, Gongning Luo, and Kuanquan Wang. 2016. A graph-based method for fitting planar B-spline curves with intersections. Journal of Computational Design and Engineering 3, 1 (2016), 14–23.Google ScholarCross Ref
    12. Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. 2015. Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection. ACM Trans. Knowl. Discov. Data 10, 1 (2015), 5:1–5:51. Google ScholarDigital Library
    13. Jiazhou Chen, Gael Guennebaud, Pascal Barla, and Xavier Granier. 2013. Non-Oriented MLS Gradient Fields. In Computer Graphics Forum, Vol. 32. 98–109.Google ScholarCross Ref
    14. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Science Sinica 14 (1965), 1396–1400.Google Scholar
    15. Jack Edmonds. 1967. Optimum branchings. J. Res. Nat. Bur. Standards 71B, 4 (1967), 233–240.Google ScholarCross Ref
    16. Koos Eissen and Roselien Steur. 2008. Sketching: Drawing Techniques for Product Designers. Bis Publishers.Google Scholar
    17. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph. 35, 4 (2016), 120. Google ScholarDigital Library
    18. Mark Finch, John Snyder, and Hugues Hoppe. 2011. Freeform Vector Graphics with Controlled Thin-plate Splines. ACM Trans. Graph. 30, 6 (2011), 166:1–166:10. Google ScholarDigital Library
    19. Stéphane Grabli, Frédo Durand, and François Sillion. 2004. Density Measure for Line-Drawing Simplification. In Proc. Pacific Graphics. Google ScholarDigital Library
    20. Cindy Grimm and Pushkar Joshi. 2012. Just DrawIt: A 3D Sketching System. In Proc. SBIM. 121–130. Google ScholarDigital Library
    21. Robert Hess and David Field. 1999. Integration of contours: new insights. Trends in Cognitive Sciences 3, 12 (1999), 480–486.Google ScholarCross Ref
    22. Xavier Hilaire and Karl Tombre. 2006. Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Mach. Intell 28, 6 (2006), 890–904. Google ScholarDigital Library
    23. Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-rigid-as-possible Shape Manipulation. ACM Trans. Graph. 24, 3 (2005), 1134–1141. Google ScholarDigital Library
    24. Pradeep Kumar Jayaraman, Chi-Wing Fu, Jianmin Zheng, Xueting Liu, and Tien-Tsin Wong. 2017. Globally Consistent Wrinkle-Aware Shading of Line Drawings. IEEE Trans. Vis. Comput. Graph (2017).Google Scholar
    25. Robert D. Kalnins, Philip L. Davidson, Lee Markosian, and Adam Finkelstein. 2003. Coherent Stylized Silhouettes. ACM Trans. Graph. 22, 3 (2003), 856–861. Google ScholarDigital Library
    26. Margret Keuper, Evgeny Levinkov, Nicolas Bonneel, Guillaume Lavoué, Thomas Brox, and Bjorn Andres. 2015. Efficient decomposition of image and mesh graphs by lifted multicuts. In Proc. ICCV. 1751–1759. Google ScholarDigital Library
    27. In-Kwon Lee. 2000. Curve reconstruction from unorganized points. Computer aided geometric design 17, 2 (2000), 161–177.Google Scholar
    28. David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. 37–49.Google Scholar
    29. H Lipson and M Shpitalni. 1996. Optimization-based reconstruction of a 3D object from a single freehand line drawing. Computer-Aided Design 28, 8 (1996), 651 — 663.Google ScholarCross Ref
    30. Xueting Liu, Tien-Tsin Wong, and Pheng-Ann Heng. 2015. Closure-aware sketch simplification. ACM Trans. Graph. 34, 6 (2015), 168. Google ScholarDigital Library
    31. James McCrae and Karan Singh. 2009. Sketching piecewise clothoid curves. Computers & Graphics 33, 4 (2009), 452–461. Google ScholarDigital Library
    32. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011a. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011). Google ScholarDigital Library
    33. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011b. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011). 185:1–185:10. Google ScholarDigital Library
    34. Gioacchino Noris, Alexander Hornung, Robert W Sumner, Maryann Simmons, and Markus Gross. 2013. Topology-driven vectorization of clean line drawings. ACM Trans. Graph. 32, 1 (2013), 4. Google ScholarDigital Library
    35. Gioacchino Noris, Daniel Sỳkora, A Shamir, Stelian Coros, Brian Whited, Maryann Simmons, Alexander Hornung, M Gross, and R Sumner. 2012. Smart scribbles for sketch segmentation. In Computer Graphics Forum, Vol. 31. 2516–2527. Google ScholarDigital Library
    36. Gunay Orbay and Levent Burak Kara. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans. Vis. Comput. Graph 17, 5 (2011), 694–708. Google ScholarDigital Library
    37. Paul L. Rosin. 1994. Grouping Curved Lines. In Machine Graphics and Vision 7. 625–644.Google Scholar
    38. Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: Shading Concept Sketches Using Cross-section Curves. ACM Trans. Graph. 31, 4 (2012), 45:1–45:11. Google ScholarDigital Library
    39. Amit Shesh and Baoquan Chen. 2008. Efficient and dynamic simplification of line drawings. In Computer Graphics Forum, Vol. 27. 537–545.Google ScholarCross Ref
    40. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35, 4 (2016), 121. Google ScholarDigital Library
    41. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2017. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Trans. Graph. (2017). Google ScholarDigital Library
    42. O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rössl, and H.-P. Seidel. 2004. Laplacian Surface Editing. In Proc. Symposium on Geometry Processing. 175–184. Google ScholarDigital Library
    43. Johan Wagemans, James H Elder, Michael Kubovy, Stephen E Palmer, Mary A Peterson, Manish Singh, and Rüdiger von der Heydt. 2012. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological bulletin 138, 6 (2012), 1172.Google Scholar
    44. Brett Wilson and Kwan-Liu Ma. 2004. Rendering complexity in computer-generated pen-and-ink illustrations. In Proc. NPAR. 129–137. Google ScholarDigital Library
    45. Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization. ACM Trans. Graph. 33, 4 (2014), 131:1–131:13. Google ScholarDigital Library
    46. Pengfei Xu, Hongbo Fu, Oscar Kin-Chung Au, and Chiew-Lan Tai. 2012. Lazy selection: a scribble-based tool for smart shape elements selection. ACM Trans. Graph. 31, 6 (2012), 142. Google ScholarDigital Library

ACM Digital Library Publication: