“StrokeAggregator: consolidating raw sketches into artist-intended curve drawings” by Liu, Rosales and Sheffer
Conference:
Type:
Entry Number: 97
Title:
- StrokeAggregator: consolidating raw sketches into artist-intended curve drawings
Session/Category Title: Sketching
Presenter(s)/Author(s):
Moderator(s):
Abstract:
When creating line drawings, artists frequently depict intended curves using multiple, tightly clustered, or overdrawn, strokes. Given such sketches, human observers can readily envision these intended, aggregate, curves, and mentally assemble the artist’s envisioned 2D imagery. Algorithmic stroke consolidation—replacement of overdrawn stroke clusters by corresponding aggregate curves—can benefit a range of sketch processing and sketch-based modeling applications which are designed to operate on consolidated, intended curves. We propose StrokeAggregator, a novel stroke consolidation method that significantly improves on the state of the art, and produces aggregate curve drawings validated to be consistent with viewer expectations. Our framework clusters strokes into groups that jointly define intended aggregate curves by leveraging principles derived from human perception research and observation of artistic practices. We employ these principles within a coarse-to-fine clustering method that starts with an initial clustering based on pairwise stroke compatibility analysis, and then refines it by analyzing interactions both within and in-between clusters of strokes. We facilitate this analysis by computing a common 1D parameterization for groups of strokes via common aggregate curve fitting. We demonstrate our method on a large range of line drawings, and validate its ability to generate consolidated drawings that are consistent with viewer perception via qualitative user evaluation, and comparisons to manually consolidated drawings and algorithmic alternatives.
References:
1. Rahul Arora, Ishan Darolia, Vinay P. Namboodiri, Karan Singh, and Adrien Bousseau. 2017. SketchSoup: Exploratory Ideation Using Design Sketches. Computer Graphics Forum (2017).Google Scholar
2. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In Proc. UIST. 151–160. Google ScholarDigital Library
3. Nikhil Bansal, Avrim Blum, and Shuchi Chawla. 2004. Correlation clustering. Machine Learning 56, 1–3 (2004), 89–113. Google ScholarDigital Library
4. Bin Bao and Hongbo Fu. 2012. Vectorizing line drawings with near-constant line width. In Proc. International Conference on Image Processing. 805–808.Google ScholarCross Ref
5. Ilya Baran, Jaakko Lehtinen, and Jovan Popović. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, Vol. 29. 655–664.Google ScholarCross Ref
6. Pascal Barla, Joëlle Thollot, and François Sillion. 2005. Geometric Clustering for Line Drawing Simplification. In Proc. EGSR. Google ScholarDigital Library
7. A. Bartolo, K. P. Camilleri, S. G. Fabri, J. C. Borg, and P. J. Farrugia. 2007. Scribbles to Vectors: Preparation of Scribble Drawings for CAD Interpretation. In Sketch-Based Interfaces and Modeling. Google ScholarDigital Library
8. Thomas Baudel. 1994. A Mark-based Interaction Paradigm for Free-hand Drawing. In Proc. UIST. 185–192. Google ScholarDigital Library
9. Mikhail Bessmeltsev, Will Chang, Nicholas Vining, Alla Sheffer, and Karan Singh. 2015. Modeling character canvases from cartoon drawings. ACM Trans. Graph. 34, 5 (2015), 162. Google ScholarDigital Library
10. Mikhail Bessmeltsev, Nicholas Vining, and Alla Sheffer. 2016. Gesture3D: Posing 3D Characters via Gesture Drawings. ACM Trans. Graph. 35, 6 (2016). Google ScholarDigital Library
11. Pengbo Bo, Gongning Luo, and Kuanquan Wang. 2016. A graph-based method for fitting planar B-spline curves with intersections. Journal of Computational Design and Engineering 3, 1 (2016), 14–23.Google ScholarCross Ref
12. Ricardo J. G. B. Campello, Davoud Moulavi, Arthur Zimek, and Jörg Sander. 2015. Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection. ACM Trans. Knowl. Discov. Data 10, 1 (2015), 5:1–5:51. Google ScholarDigital Library
13. Jiazhou Chen, Gael Guennebaud, Pascal Barla, and Xavier Granier. 2013. Non-Oriented MLS Gradient Fields. In Computer Graphics Forum, Vol. 32. 98–109.Google ScholarCross Ref
14. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Science Sinica 14 (1965), 1396–1400.Google Scholar
15. Jack Edmonds. 1967. Optimum branchings. J. Res. Nat. Bur. Standards 71B, 4 (1967), 233–240.Google ScholarCross Ref
16. Koos Eissen and Roselien Steur. 2008. Sketching: Drawing Techniques for Product Designers. Bis Publishers.Google Scholar
17. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph. 35, 4 (2016), 120. Google ScholarDigital Library
18. Mark Finch, John Snyder, and Hugues Hoppe. 2011. Freeform Vector Graphics with Controlled Thin-plate Splines. ACM Trans. Graph. 30, 6 (2011), 166:1–166:10. Google ScholarDigital Library
19. Stéphane Grabli, Frédo Durand, and François Sillion. 2004. Density Measure for Line-Drawing Simplification. In Proc. Pacific Graphics. Google ScholarDigital Library
20. Cindy Grimm and Pushkar Joshi. 2012. Just DrawIt: A 3D Sketching System. In Proc. SBIM. 121–130. Google ScholarDigital Library
21. Robert Hess and David Field. 1999. Integration of contours: new insights. Trends in Cognitive Sciences 3, 12 (1999), 480–486.Google ScholarCross Ref
22. Xavier Hilaire and Karl Tombre. 2006. Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Mach. Intell 28, 6 (2006), 890–904. Google ScholarDigital Library
23. Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-rigid-as-possible Shape Manipulation. ACM Trans. Graph. 24, 3 (2005), 1134–1141. Google ScholarDigital Library
24. Pradeep Kumar Jayaraman, Chi-Wing Fu, Jianmin Zheng, Xueting Liu, and Tien-Tsin Wong. 2017. Globally Consistent Wrinkle-Aware Shading of Line Drawings. IEEE Trans. Vis. Comput. Graph (2017).Google Scholar
25. Robert D. Kalnins, Philip L. Davidson, Lee Markosian, and Adam Finkelstein. 2003. Coherent Stylized Silhouettes. ACM Trans. Graph. 22, 3 (2003), 856–861. Google ScholarDigital Library
26. Margret Keuper, Evgeny Levinkov, Nicolas Bonneel, Guillaume Lavoué, Thomas Brox, and Bjorn Andres. 2015. Efficient decomposition of image and mesh graphs by lifted multicuts. In Proc. ICCV. 1751–1759. Google ScholarDigital Library
27. In-Kwon Lee. 2000. Curve reconstruction from unorganized points. Computer aided geometric design 17, 2 (2000), 161–177.Google Scholar
28. David Levin. 2004. Mesh-independent surface interpolation. In Geometric modeling for scientific visualization. 37–49.Google Scholar
29. H Lipson and M Shpitalni. 1996. Optimization-based reconstruction of a 3D object from a single freehand line drawing. Computer-Aided Design 28, 8 (1996), 651 — 663.Google ScholarCross Ref
30. Xueting Liu, Tien-Tsin Wong, and Pheng-Ann Heng. 2015. Closure-aware sketch simplification. ACM Trans. Graph. 34, 6 (2015), 168. Google ScholarDigital Library
31. James McCrae and Karan Singh. 2009. Sketching piecewise clothoid curves. Computers & Graphics 33, 4 (2009), 452–461. Google ScholarDigital Library
32. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011a. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011). Google ScholarDigital Library
33. Liangliang Nan, Andrei Sharf, Ke Xie, Tien-Tsin Wong, Oliver Deussen, Daniel Cohen-Or, and Baoquan Chen. 2011b. Conjoining Gestalt Rules for Abstraction of Architectural Drawings. ACM Trans. Graph. 30, 6 (2011). 185:1–185:10. Google ScholarDigital Library
34. Gioacchino Noris, Alexander Hornung, Robert W Sumner, Maryann Simmons, and Markus Gross. 2013. Topology-driven vectorization of clean line drawings. ACM Trans. Graph. 32, 1 (2013), 4. Google ScholarDigital Library
35. Gioacchino Noris, Daniel Sỳkora, A Shamir, Stelian Coros, Brian Whited, Maryann Simmons, Alexander Hornung, M Gross, and R Sumner. 2012. Smart scribbles for sketch segmentation. In Computer Graphics Forum, Vol. 31. 2516–2527. Google ScholarDigital Library
36. Gunay Orbay and Levent Burak Kara. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans. Vis. Comput. Graph 17, 5 (2011), 694–708. Google ScholarDigital Library
37. Paul L. Rosin. 1994. Grouping Curved Lines. In Machine Graphics and Vision 7. 625–644.Google Scholar
38. Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh. 2012. CrossShade: Shading Concept Sketches Using Cross-section Curves. ACM Trans. Graph. 31, 4 (2012), 45:1–45:11. Google ScholarDigital Library
39. Amit Shesh and Baoquan Chen. 2008. Efficient and dynamic simplification of line drawings. In Computer Graphics Forum, Vol. 27. 537–545.Google ScholarCross Ref
40. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to simplify: fully convolutional networks for rough sketch cleanup. ACM Trans. Graph. 35, 4 (2016), 121. Google ScholarDigital Library
41. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2017. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Trans. Graph. (2017). Google ScholarDigital Library
42. O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rössl, and H.-P. Seidel. 2004. Laplacian Surface Editing. In Proc. Symposium on Geometry Processing. 175–184. Google ScholarDigital Library
43. Johan Wagemans, James H Elder, Michael Kubovy, Stephen E Palmer, Mary A Peterson, Manish Singh, and Rüdiger von der Heydt. 2012. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological bulletin 138, 6 (2012), 1172.Google Scholar
44. Brett Wilson and Kwan-Liu Ma. 2004. Rendering complexity in computer-generated pen-and-ink illustrations. In Proc. NPAR. 129–137. Google ScholarDigital Library
45. Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization. ACM Trans. Graph. 33, 4 (2014), 131:1–131:13. Google ScholarDigital Library
46. Pengfei Xu, Hongbo Fu, Oscar Kin-Chung Au, and Chiew-Lan Tai. 2012. Lazy selection: a scribble-based tool for smart shape elements selection. ACM Trans. Graph. 31, 6 (2012), 142. Google ScholarDigital Library
