“Toonsynth: example-based synthesis of hand-colored cartoon animations” by Dvorožňák, Li, Kim and Sýkora
Conference:
Type:
Entry Number: 167
Title:
- Toonsynth: example-based synthesis of hand-colored cartoon animations
Session/Category Title: Bodies in Motion Human Performance Capture
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a new example-based approach for synthesizing hand-colored cartoon animations. Our method produces results that preserve the specific visual appearance and stylized motion of manually authored animations without requiring artists to draw every frame from scratch. In our framework, the artist first stylizes a limited set of known source skeletal animations from which we extract a style-aware puppet that encodes the appearance and motion characteristics of the artwork. Given a new target skeletal motion, our method automatically transfers the style from the source examples to create a hand-colored target animation. Compared to previous work, our technique is the first to preserve both the detailed visual appearance and stylized motion of the original hand-drawn content. Our approach has numerous practical applications including traditional animation production and content creation for games.
References:
1. Rahul Arora, Ishan Darolia, Vinay Namboodiri, Karan Singh, and Adrien Bousseau. 2017. SketchSoup: Exploratory Ideation Using Design Sketches. Computer Graphics Forum 36, 8 (2017), 302–312.Google ScholarCross Ref
2. Yunfei Bai, Danny M Kaufman, Karen Liu, and Jovan Popović. 2016. Artist-directed dynamics for 2D animation. ACM Transactions on Graphics 35, 4 (2016), 145. Google ScholarDigital Library
3. William Baxter and Ken-ichi Anjyo. 2006. Latent Doodle Space. Computer Graphics Forum 25, 3 (2006), 477–485.Google ScholarCross Ref
4. William Baxter, Pascal Barla, and Ken Anjyo. 2009. N-way morphing for 2D animation. Journal of Visualization and Computer Animation 20, 2–3 (2009), 79–87. Google ScholarDigital Library
5. Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden. 2013. Stylizing animation by example. ACM Transactions on Graphics 32, 4 (2013), 119. Google ScholarDigital Library
6. Mikhail Bessmeltsev, Nicholas Vining, and Alla Sheffer. 2016. Gesture3D: posing 3D characters via gesture drawings. ACM Transactions on Graphics 35, 6 (2016), 165. Google ScholarDigital Library
7. Christoph Bregler, Lorie Loeb, Erika Chuang, and Hrishi Deshpande. 2002. Turning to the Masters: Motion Capturing Cartoons. ACM Transactions on Graphics 21, 3 (2002), 399–407. Google ScholarDigital Library
8. Ian Buck, Adam Finkelstein, Charles Jacobs, Allison Klein, David Salesin, Joshua Seims, Richard Szeliski, and Kentaro Toyama. 2000. Performance-Driven Hand-Drawn Animation. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 101–108. Google ScholarDigital Library
9. Nestor Burtnyk and Marceli Wein. 1976. Interactive Skeleton Techniques for Enhancing Motion Dynamics in Key Frame Animation. Commun. ACM 19, 10 (1976), 564–569. Google ScholarDigital Library
10. Edwin Catmull. 1978. The Problems of Computer-Assisted Animation. 12, 3 (1978), 348–353. Google ScholarDigital Library
11. James Davis, Maneesh Agrawala, Erika Chuang, Zoran Popovic, and David Salesin. 2003. A sketching interface for articulated figure animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 320–328. Google ScholarDigital Library
12. Marek Dvorožňák, Pierre Bénard, Pascal Barla, Oliver Wang, and Daniel Sýkora. 2017. Example-Based Expressive Animation of 2D Rigid Bodies. ACM Transactions on Graphics 36, 4, Article 127 (2017). Google ScholarDigital Library
13. Elmar Eisemann and Frédo Durand. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (2004), 673–678. Google ScholarDigital Library
14. Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Eli Shechtman, Paul Asente, Jingwan Lu, and Daniel Sýkora. 2016. Stylit: Illumination-guided Example-based Stylization of 3D Renderings. ACM Transactions on Graphics 35, 4 (2016), 92. Google ScholarDigital Library
15. Jakub Fišer, Ondřej Jamriška, David Simons, Eli Shechtman, Jingwan Lu, Paul Asente, Michal Lukáč, and Daniel Sýkora. 2017. Example-Based Synthesis of Stylized Facial Animations. ACM Transactions on Graphics 36, 4, Article 155 (2017). Google ScholarDigital Library
16. Jakub Fišer, Michal Lukáč, Ondřej Jamriška, Martin Čadík, Yotam Gingold, Paul Asente, and Daniel Sýkora. 2014. Color Me Noisy: Example-Based Rendering of Hand-Colored Animations with Temporal Noise Control. Computer Graphics Forum 33, 4 (2014), 1–10.Google ScholarCross Ref
17. Ben Glocker, Nikos Komodakis, Georgios Tziritas, Nassir Navab, and Nikos Paragios. 2008. Dense Image Registration Through MRFs And Efficient Linear Programming. Medical Image Analysis 12, 6 (2008), 731–741.Google ScholarCross Ref
18. William van Haevre, Fabian di Fiore, and Frank van Reeth. 2005. Uniting Cartoon Textures with Computer Assisted Animation. In Proceedings of International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia. 245–253. Google ScholarDigital Library
19. Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. 2001. Image analogies. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 327–340. Google ScholarDigital Library
20. Alexander Hornung, Ellen Dekkers, and Leif Kobbelt. 2007. Character Animation from 2D Pictures and 3D Motion Data. ACM Transactions on Graphics 26, 1 (2007). Google ScholarDigital Library
21. Eakta Jain, Yaser Sheikh, and Jessica Hodgins. 2009. Leveraging the talent of hand animators to create three-dimensional animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 93–102. Google ScholarDigital Library
22. Ben Jones Jovan Popovic, James McCann, Wilmot Li, and Adam Bargteil. 2015. Dynamic sprites: Artistic authoring of interactive animations. Journal of Visualization and Computer Animation 26, 2 (2015), 97–108. Google ScholarDigital Library
23. Christina de Juan and Bobby Bodenheimer. 2004. Cartoon Textures. In Proceedings of Eurographics Symposium on Computer Animation. 267–276. Google ScholarDigital Library
24. Christina de Juan and Bobby Bodenheimer. 2006. Re-using traditional animation: Methods for semi-automatic segmentation and inbetweening. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 223–232. Google ScholarDigital Library
25. Rubaiat Habib Kazi, Tovi Grossman, Nobuyuki Umetani, and George Fitzmaurice. 2016. Motion Amplifiers: Sketching Dynamic Illustrations Using the Principles of 2D Animation. In Proceedings of ACM Conference on Human Factors in Computing Systems. 4599–4609. Google ScholarDigital Library
26. Alexander Kort. 2002. Computer Aided Inbetweening. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 125–132. Google ScholarDigital Library
27. Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3 (2002), 473–482. Google ScholarDigital Library
28. Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics 24, 3 (2005), 795–802. Google ScholarDigital Library
29. John Lasseter. 1987. Principles of Traditional Animation Applied to 3D Computer Animation. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. 35–44. Google ScholarDigital Library
30. Seungyong Lee, George Wolberg, and Sung Yong Shin. 1998. Polymorph: Morphing Among Multiple Images. IEEE Computer Graphics and Applications 18, 1 (1998), 58–71. Google ScholarDigital Library
31. Sun-Young Lee, Jong-Chul Yoon, Ji-Yong Kwon, and In-Kwon Lee. 2012. CartoonModes: Cartoon Stylization of Video Objects Through Modal Analysis. Graphical Models 74, 2 (2012), 51–60. Google ScholarDigital Library
32. Dushyant Mehta, Srinath Sridhar, Oleksandr Sotnychenko, Helge Rhodin, Mohammad Shafiei, Hans-Peter Seidel, Weipeng Xu, Dan Casas, and Christian Theobalt. 2017. VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics 36, 4 (2017), 44:1–44:14. Google ScholarDigital Library
33. Johannes Schmid, Robert Sumner, Huw Bowles, and Markus Gross. 2010. Programmable Motion Effects. ACM Transactions on Graphics 29, 4 (2010), 57. Google ScholarDigital Library
34. Daniel Sýkora, Mirela Ben-Chen, Martin Čadík, Brian Whited, and Maryann Simmons. 2011. TexToons: Practical Texture Mapping for Hand-drawn Cartoon Animations. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 75–83. Google ScholarDigital Library
35. Daniel Sýkora, Jan Buriánek, and Jiří Žára. 2005. Sketching Cartoons by Example. In Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling. 27–34.Google Scholar
36. Daniel Sýkora, John Dingliana, and Steven Collins. 2009. As-Rigid-As-Possible Image Registration for Hand-Drawn Cartoon Animations. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering. 25–33. Google ScholarDigital Library
37. Daniel Sýkora, Ladislav Kavan, Martin Čadík, Ondřej Jamriška, Alec Jacobson, Brian Whited, Maryann Simmons, and Olga Sorkine-Hornung. 2014. Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters. ACM Transactions on Graphics 33, 2 (2014), 16. Google ScholarDigital Library
38. Daniel Sýkora, David Sedlacek, Sun Jinchao, John Dingliana, and Steven Collins. 2010. Adding Depth to Cartoons Using Sparse Depth (In)equalities. Computer Graphics Forum 29, 2 (2010), 615–623.Google ScholarCross Ref
39. Cedric Vanaken, Chris Hermans, Tom Mertens, Fabian Di Fiore, Philippe Bekaert, and Frank Van Reeth. 2008. Strike a Pose: Image-Based Pose Synthesis. In Proceedings of the Conference on Vision, Modeling and Visualization. 131–138.Google Scholar
40. Jue Wang, Steven Drucker, Maneesh Agrawala, and Michael Cohen. 2006. The Cartoon Animation Filter. ACM Transactions on Graphics 25, 3 (2006), 1169–1173. Google ScholarDigital Library
41. Xun Wang, Wenwu Yang, Haoyu Peng, and Guozheng Wang. 2013. Shape-aware skeletal deformation for 2D characters. The Visual Computer 29, 6–8 (2013), 545–553. Google ScholarDigital Library
42. Yonatan Wexler, Eli Shechtman, and Michal Irani. 2007. Space-Time Completion of Video. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 3 (2007), 463–476. Google ScholarDigital Library
43. Brian Whited, Gioacchino Noris, Maryann Simmons, Robert Sumner, Markus Gross, and Jarek Rossignac. 2010. BetweenIT: An Interactive Tool for Tight Inbetweening. Computer Graphics Forum 29, 2 (2010), 605–614.Google ScholarCross Ref
44. Nora Willett, Wilmot Li, Jovan Popovic, Floraine Berthouzoz, and Adam Finkelstein. 2017. Secondary Motion for Performed 2D Animation. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST ’17). ACM, New York, NY, USA, 97–108. Google ScholarDigital Library
45. Wenwu Yang. 2017. Context-Aware Computer Aided Inbetweening. IEEE Transactions on Visualizaton and Computer Graphics (2017).Google Scholar
46. Chih-Kuo Yeh, Shi-Yang Huang, Pradeep Kumar Jayaraman, Chi-Wing Fu, and Tong-Yee Lee. 2017. Interactive High-Relief Reconstruction for Organic and Double-Sided Objects from a Photo. IEEE Transactions on Visualization and Computer Graphics 23, 7 (2017), 1796–1808.Google ScholarDigital Library
47. Lei Zhang, Hua Huang, and Hongbo Fu. 2012. EXCOL: An EXtract-and-COmplete Layering Approach to Cartoon Animation Reusing. IEEE Transactions on Visualization and Computer Graphics 18, 7 (2012), 1156–1169. Google ScholarDigital Library
48. Yufeng Zhu, Jovan Popović, Robert Bridson, and Danny Kaufman. 2017. Planar Interpolation with Extreme Deformation, Topology Change and Dynamics. ACM Transactions on Graphics 36, 6 (2017), 213. Google ScholarDigital Library
