“CAD2Sketch: Generating Concept Sketches from CAD Sequences” by Hähnlein, Li, Mitra and Bousseau
Conference:
Type(s):
Title:
- CAD2Sketch: Generating Concept Sketches from CAD Sequences
Session/Category Title: Computer-Aided Design
Presenter(s)/Author(s):
Abstract:
Concept sketches are ubiquitous in industrial design, as they allow designers to quickly depict imaginary 3D objects. To construct their sketches with accurate perspective, designers rely on longstanding drawing techniques, including the use of auxiliary construction lines to identify midpoints of perspective planes, to align points vertically and horizontally, and to project planar curves from one perspective plane to another. We present a method to synthesize such construction lines from CAD sequences. Importantly, our method balances the presence of construction lines with overall clutter, such that the resulting sketch is both well-constructed and readable, as professional designers are trained to do. In addition to generating sketches that are visually similar to real ones, we apply our method to synthesize a large quantity of paired sketches and normal maps, and show that the resulting dataset can be used to train a neural network to infer normals from concept sketches.1
References:
1. Maneesh Agrawala, Wilmot Li, and Floraine Berthouzoz. 2011. Design Principles for Visual Communication. Commun. ACM 54, 4 (2011).
2. Pierre Bénard and Aaron Hertzmann. 2019. Line drawings from 3D models: a tutorial. Foundations and Trends in Computer Graphics and Vision 11, 1–2 (2019).
3. Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. 2013. Style and abstraction in portrait sketching. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–12.
4. Alexandra Bonnici, Alican Akman, Gabriel Calleja, Kenneth P Camilleri, Patrick Fehling, Alfredo Ferreira, Florian Hermuth, Johann Habakuk Israel, Tom Landwehr, Juncheng Liu, et al. 2019. Sketch-based interaction and modeling: where do we stand? Artificial intelligence for engineering design analysis and manufacturing (2019), 1–19.
5. Xiang Cao and Shumin Zhai. 2007. Modeling Human Performance of Pen Stroke Gestures (ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)).
6. Caroline Chan, Frédo Durand, and Phillip Isola. 2022. Learning to generate line drawings that convey geometry and semantics. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
7. Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive Contours for Conveying Shape. ACM Transactions on Graphics (Proc. SIGGRAPH) 22, 3 (July 2003), 848–855.
8. Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei Efros, and Adrien Bousseau. 2018. 3D Sketching using Multi-View Deep Volumetric Prediction. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 21 (2018).
9. Betty Edwards. 1979. Drawing on the right side of the brain : a course in enhancing creativity and artistic confidence.
10. Koos Eissen and Roselien Steur. 2008. Sketching: Drawing Techniques for Product Designers. Bis Publishers.
11. Koos Eissen and Roselien Steur. 2011. Sketching: The Basics. Bis Publishers.
12. Giorgio Gori, Alla Sheffer, Nicholas Vining, Enrique Rosales, Nathan Carr, and Tao Ju. 2017. FlowRep: Descriptive Curve Networks for Free-Form Design Shapes. ACM Transaction on Graphics (proc. SIGGRAPH) 36, 4 (2017).
13. Stéphane Grabli, Emmanuel Turquin, Frédo Durand, and François X. Sillion. 2004. Programmable Style for NPR Line Drawing. In Proc. Eurographics Conference on Rendering Techniques (EGSR).
14. Stéphane Grabli, Emmanuel Turquin, Frédo Durand, and François X. Sillion. 2010. Programmable rendering of line drawing from 3D scenes. ACM Transactions on Graphics 29, 2 (2010).
15. Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti (Derek) Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wu, Kwang Moo Yi, Fangcheng Zhong, and Andrea Tagliasacchi. 2022. Kubric: a scalable dataset generator. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
16. Igor Griva, Stephen G. Nash, and Ariela Sofer. 2008. Linear and Nonlinear Optimization (2. ed.). SIAM.
17. Yulia Gryaditskaya, Felix Hähnlein, Chenxi Liu, Alla Sheffer, and Adrien Bousseau. 2020. Lifting Freehand Concept Sketches into 3D. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) (2020).
18. Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia Pont, Fredo Durand, and Adrien Bousseau. 2019. OpenSketch: A Richly-Annotated Dataset of Product Design Sketches. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) (2019).
19. Benoit Guillard, Edoardo Remelli, Pierre Yvernay, and Pascal Fua. 2021. Sketch2Mesh: Reconstructing and Editing 3D Shapes From Sketches. In Proc. IEEE International Conference on Computer Vision (ICCV).
20. Gurobi Optimization, LLC. 2021. Gurobi Optimizer Reference Manual. https://www.gurobi.com
21. Felix Hähnlein, Yulia Gryaditskaya, Alla Sheffer, and Adrien Bousseau. 2022. Symmetry-driven 3D Reconstruction from Concept Sketches. In SIGGRAPH Conference Proceedings. 1–8.
22. James W Hennessey, Han Liu, Holger Winnemöller, Mira Dontcheva, and Niloy J Mitra. 2016. How2Sketch: generating easy-to-follow tutorials for sketching 3D objects. arXiv preprint arXiv:1607.07980 (2016).
23. Kevin Henry. 2012. Drawing for Product Designers. Laurence King Publishing.
24. Aaron Hertzmann and Denis Zorin. 2000. Illustrating Smooth Surfaces. In SIGGRAPH.
25. George Hlavács. 2014. The Exceptionally Simple Theory of Sketching. Laurence King Publishing.
26. Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2016. Shape Synthesis from Sketches via Procedural Models and Convolutional Networks. IEEE Transactions on Visualization and Computer Graphics (TVCG) 22, 10 (2016), 1.
27. Tilke Judd, Frédo Durand, and Edward H. Adelson. 2007. Apparent ridges for line drawing. ACM Transactions on Graphics 26, 3 (2007).
28. Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. Abc: A big cad model dataset for geometric deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9601–9611.
29. Yong Jae Lee, C. Lawrence Zitnick, and Michael F. Cohen. 2011. ShadowDraw: Real-Time User Guidance for Freehand Drawing. ACM Transaction on Graphics (proc. SIGGRAPH) (2011).
30. Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. 2020. Sketch2cad: Sequential cad modeling by sketching in context. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.
31. Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. 2022. Free2CAD: Parsing Freehand Drawings into CAD Commands. ACM Transaction on Graphics (proc. SIGGRAPH) (2022).
32. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust flow-guided neural prediction for sketch-based freeform surface modeling. In ACM Trans. Graph. (Proc. SIGGRAPH Asia). ACM, 238.
33. Wilmot Li, Lincoln Ritter, Maneesh Agrawala, Brian Curless, and David Salesin. 2007. Interactive Cutaway Illustrations of Complex 3D Models. ACM Transactions on Graphics (proc. SIGGRAPH) 26, 3 (July 2007).
34. Chenxi Liu, Enrique Rosales, and Alla Sheffer. 2018. StrokeAggregator: consolidating raw sketches into artist-intended curve drawings. ACM Transactions on Graphics (TOG) 37, 4 (2018), 97.
35. Difan Liu, Matthew Fisher, Aaron Hertzmann, and Evangelos Kalogerakis. 2021. Neural Strokes: Stylized Line Drawing of 3D Shapes. In Proc. IEEE International Conference on Computer Vision (ICCV). 14204–14213.
36. Difan Liu, Mohamed Nabail, Aaron Hertzmann, and Evangelos Kalogerakis. 2020. Neural contours: Learning to draw lines from 3d shapes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5428–5436.
37. Jingbo Liu, Hongbo Fu, and Chiew-Lan Tai. 2014. Dynamic Sketching: Simulating the Process of Observational Drawing. In Proceedings of the Workshop on Computational Aesthetics.
38. Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D shape reconstruction from sketches via multi-view convolutional networks. In IEEE International Conference on 3D Vision (3DV). 67–77.
39. Thomas L Magnanti and Prakash Mirchandani. 1993. Shortest paths, single origin-destination network design, and associated polyhedra. Networks 23, 2 (1993), 103–121.
40. Niloy J. Mitra, Yong-Liang Yang, Dong-Ming Yan, Wilmot Li, and Maneesh Agrawala. 2010. Illustrating How Mechanical Assemblies Work. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 3 (2010).
41. Gen Nishida, Ignacio Garcia-Dorado, Daniel G. Aliaga, Bedrich Benes, and Adrien Bousseau. 2016. Interactive Sketching of Urban Procedural Models. ACM Trans. Graph. (SIGGRAPH) 35, 4, Article 130 (July 2016), 11 pages.
42. Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel. 2004. Ridge-Valley Lines on Meshes via Implicit Surface Fitting. ACM Transactions on Graphics (Proc. SIGGRAPH) 23, 3 (2004).
43. Wamiq Para, Shariq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas J Guibas, and Peter Wonka. 2021. SketchGen: Generating Constrained CAD Sketches. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 34.
44. PTC. 2019. OnShape. www.onshape.com
45. Scott Robertson and Thomas Bertling. 2013. How to draw. Design Studio Press.
46. Ryan Schmidt, Tobias Isenberg, Pauline Jepp, Karan Singh, and Brian Wyvill. 2007. Sketching, Scaffolding, and Inking: A Visual History for Interactive 3D Modeling. In International Symposium on Non-Photorealistic Animation and Rendering (NPAR).
47. Ryan Schmidt, Azam Khan, Gord Kurtenbach, and Karan Singh. 2009a. On Expert Performance in 3D Curve-drawing Tasks. In Sketch-Based Interfaces and Modeling (SBIM). 8.
48. Ryan Schmidt, Azam Khan, Karan Singh, and Gord Kurtenbach. 2009b. Analytic drawing of 3D scaffolds. In ACM transactions on graphics (TOG), Vol. 28. ACM, 149.
49. Ari Seff, Wenda Zhou, Nick Richardson, and Ryan P Adams. 2022. Vitruvion: A Generative Model of Parametric CAD Sketches. In International Conference on Learning Representations (ICLR).
50. Wanchao Su, Dong Du, Xin Yang, Shizhe Zhou, and Hongbo Fu. 2018. Interactive sketch-based normal map generation with deep neural networks. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 1 (2018), 22.
51. Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMillan, and Julie Dorsey. 2021. Tracing versus Freehand for Evaluating Computer-Generated Drawings. ACM Transactions on Graphics (Proc. SIGGRAPH) 40, 4 (2021).
52. Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik. 2021. Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Construction from Human Design Sequences. ACM Transactions on Graphics (TOG) 40, 4 (2021).
53. Georges Winkenbach and David H. Salesin. 1994. Computer-Generated Pen-and-Ink Illustration. In SIGGRAPH.
54. Erroll Wood, Tadas Baltrušaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J. Cashman, and Jamie Shotton. 2021. Fake It Till You Make It: Face Analysis in the Wild Using Synthetic Data Alone. In IEEE International Conference on Computer Vision (ICCV).
55. Rundi Wu, Chang Xiao, and Changxi Zheng. 2021. DeepCAD: A Deep Generative Network for Computer-Aided Design Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 6772–6782.
56. Song-Hai Zhang, Yuan-Chen Guo, and Qing-Wen Gu. 2021. Sketch2Model: View-Aware 3D Modeling From Single Free-Hand Sketches. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
57. Yue Zhong, Yulia Gryaditskaya, Honggang Zhang, and Yi-Zhe Song. 2020a. Deep Sketch-Based Modeling: Tips and Tricks. In Proceedings of International Conference on 3D Vision (3DV).
58. Yue Zhong, Yonggang Qi, Yulia Gryaditskaya, Honggang Zhang, and Yi-Zhe Song. 2020b. Towards practical sketch-based 3d shape generation: The role of professional sketches. IEEE Transactions on Circuits and Systems for Video Technology 31, 9 (2020), 3518–3528.
