“Stylizing animation by example” by Bénard, Cole, Kass, Mordatch, Hegarty, et al. …

  • ©Pierre Bénard, Forrester Cole, Michael Kass, Igor Mordatch, James Hegarty, Martin-Sebastian Senn, Kurt Fleischer, Davide Pesare, and Katherine Breeden

Conference:


Type:


Title:

    Stylizing animation by example

Session/Category Title: Artistic Rendering & Stylization

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Skilled artists, using traditional media or modern computer painting tools, can create a variety of expressive styles that are very appealing in still images, but have been unsuitable for animation. The key difficulty is that existing techniques lack adequate temporal coherence to animate these styles effectively. Here we augment the range of practical animation styles by extending the guided texture synthesis method of Image Analogies [Hertzmann et al. 2001] to create temporally coherent animation sequences. To make the method art directable, we allow artists to paint portions of keyframes that are used as constraints. The in-betweens calculated by our method maintain stylistic continuity and yet change no more than necessary over time.

References:


    1. Agarwala, A., Hertzmann, A., Salesin, D. H., and Seitz, S. M. 2004. Keyframe-based tracking for rotoscoping and animation. ACM Transactions on Graphics 23, 3, 584–591. Google ScholarDigital Library
    2. Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of the 7th symposium on Interactive 3D graphics, 217–226. Google ScholarDigital Library
    3. Bargteil, A. W., Sin, F., Michaels, J., Goktekin, T., and O’Brien, J. 2006. A texture synthesis method for liquid animations. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 345–351. Google ScholarDigital Library
    4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics 28, 3, 24:1–24:11. Google ScholarDigital Library
    5. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized PatchMatch correspondence algorithm. In Proceedings of the 11th European conference on computer vision conference on Computer vision, 29–43. Google ScholarDigital Library
    6. Bénard, P., Bousseau, A., and Thollot, J. 2009. Dynamic solid textures for real-time coherent stylization. Proceedings of the 2009 symposium on Interactive 3D graphics and games, 121–127. Google ScholarDigital Library
    7. Bénard, P., Lagae, A., Vangorp, P., Lefebvre, S., Drettakis, G., and Thollot, J. 2010. A dynamic noise primitive for coherent stylization. Computer Graphics Forum 4, 29, 1497–1506. Google ScholarDigital Library
    8. Bénard, P., Bousseau, A., and Thollot, J. 2011. State-of-the-Art Report on Temporal Coherence for Stylized Animations. Computer Graphics Forum 30, 8, 2367–2386.Google ScholarCross Ref
    9. Bénard, P., Jingwan, L., Cole, F., Finkelstein, A., and Thollot, J. 2012. Active Strokes: Coherent Line Stylization for Animated 3D Models. In NPAR 2012 – 10th International Symposium on Non-photorealistic Animation and Rendering, ACM, 37–46. Google ScholarDigital Library
    10. Bonneel, N., van de Panne, M., Lefebvre, S., and Drettakis, G. 2010. Proxy-guided texture synthesis for rendering natural scenes. In 15th International Workshop on Vision, Modeling and Visualization, 87–95.Google Scholar
    11. Bousseau, A., Neyret, F., Thollot, J., and Salesin, D. 2007. Video watercolorization using bidirectional texture advection. ACM Transactions on Graphics 26, 3, 104. Google ScholarDigital Library
    12. Cao, C., Chen, S., Zhang, W., and Tang, X. 2011. Automatic motion-guided video stylization and personalization. In Proceedings of the 19th ACM international conference on Multimedia, 1041–1044. Google ScholarDigital Library
    13. Chen, J., and Wang, B. 2009. High quality solid texture synthesis using position and index histogram matching. The Visual Computer 26, 4, 253–262. Google ScholarDigital Library
    14. Daniels, E. 1999. Deep canvas in Disney’s Tarzan. In ACM SIGGRAPH 99 Conference abstracts and applications, 200. Google ScholarDigital Library
    15. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image melding. ACM Transactions on Graphics 31, 4, 82:1–82:10. Google ScholarDigital Library
    16. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In Proceedings of SIGGRAPH ’01, 341–346. Google ScholarDigital Library
    17. Granlund, G. H., and Knutsson, H. 1995. Signal Processing for Computer Vision. Springer. Google ScholarDigital Library
    18. Han, J., Zhou, K., Wei, L.-Y., Gong, M., Bao, H., Zhang, X., and Guo, B. 2006. Fast example-based surface texture synthesis via discrete optimization. The Visual Computer 22, 9–11. Google ScholarDigital Library
    19. Haro, A. 2003. Example based processing for image and video synthesis. PhD thesis. AAI3117934. Google ScholarDigital Library
    20. Hashimoto, R., Johan, H., and Nishita, T. 2003. Creating various styles of animations using example-based filtering. In Computer Graphics International, 312–317.Google Scholar
    21. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of SIGGRAPH ’01, 327–340. Google ScholarDigital Library
    22. Kabul, I., Pizer, S. M., Rosenman, J., and Niethammer, M. 2011. An Optimal Control Approach for Texture Metamorphosis. Computer Graphics Forum 30, 8, 2341–2353.Google ScholarCross Ref
    23. Kagaya, M., Brendel, W., Deng, Q., Kesterson, T., Todorovic, S., Neill, P. J., and Zhang, E. 2011. Video painting with space-time-varying style parameters. IEEE Transactions on Visualization and Computer Graphics 17, 1, 74–87. Google ScholarDigital Library
    24. Kalnins, R. D., Markosian, L., Meier, B. J., Kowalski, M. A., Lee, J. C., Davidson, P. L., Webb, M., Hughes, J. F., and Finkelstein, A. 2002. WYSIWYG NPR: drawing strokes directly on 3D models. In ACM Transactions on Graphics, vol. 21, 755–762. Google ScholarDigital Library
    25. Kass, M., and Pesare, D. 2011. Coherent noise for non-photorealistic rendering. ACM Transactions on Graphics 30, 4, 30:1–30:6. Google ScholarDigital Library
    26. Kulla, C., Tucek, J., Bailey, R., and Grimm, C. 2003. Using texture synthesis for non-photorealistic shading from paint samples. In Proceedings of the 11th Pacific Conference on Computer Graphics and Applications, 477–481. Google ScholarDigital Library
    27. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Trans. Graph. 24, 3, 795–802. Google ScholarDigital Library
    28. Kwatra, V., Adalsteinsson, D., Kim, T., Kwatra, N., Carlson, M., and Lin, M. C. 2007. Texturing fluids. IEEE Transactions on Visualization and Computer Graphics 13, 5, 939–952. Google ScholarDigital Library
    29. Kyprianidis, J. E., Collomosse, J., Wang, T., and Isenberg, T. 2013. State of the “art”: A taxonomy of artistic stylization techniques for images and video. Visualization and Computer Graphics, IEEE Transactions on 19, 5, 866–885. Google ScholarDigital Library
    30. Lai, C.-H., and Wu, J.-L. 2007. Temporal texture synthesis by patch-based sampling and morphing interpolation. Computer Animation and Virtual Worlds 18, 4–5, 415–428. Google ScholarDigital Library
    31. Lasram, A., and Lefebvre, S. 2012. Parallel patch-based texture synthesis. In Proceedings of the 4th ACM SIGGRAPH/Eurographics conference on High-Performance Graphics, 115–124. Google ScholarDigital Library
    32. Lee, H., Seo, S., and Yoon, K. 2011. Directional texture transfer with edge enhancement. Computers & Graphics 35, 1, 81–91. Google ScholarDigital Library
    33. Lefebvre, S., and Hoppe, H. 2006. Appearance-space texture synthesis. ACM Transactions on Graphics 25, 3, 541–548. Google ScholarDigital Library
    34. Lin, L., Zeng, K., Lv, H., Wang, Y., Xu, Y., and Zhu, S.-C. 2010. Painterly animation using video semantics and feature correspondence. In Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, no. 212, 73–80. Google ScholarDigital Library
    35. Liu, Z., Liu, C., Shum, H.-Y., and Yu, Y. 2002. Pattern-based texture metamorphosis. Proceedings of the 10th Pacific Conference on Computer Graphics and Applications 1, 184–191. Google ScholarDigital Library
    36. Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. ACM Transactions on Graphics 24, 3, 787–794. Google ScholarDigital Library
    37. O’Donovan, P., and Hertzmann, A. 2011. AniPaint: Interactive Painterly Animation from Video. IEEE Transactions on Visualization and Computer Graphics 18, 3, 475–487. Google ScholarDigital Library
    38. Praun, E., Hoppe, H., Webb, M., and Finkelstein, A. 2001. Real-time hatching. Proceedings of SIGGRAPH ’01, 581. Google ScholarDigital Library
    39. Ray, N., Lévy, B., Wang, H., Turk, G., and Vallet, B. 2009. Material Space Texturing. Computer Graphics Forum 28, 6, 1659–1669.Google ScholarCross Ref
    40. Ruiters, R., Schnabel, R., and Klein, R. 2010. Patch-based Texture Interpolation. Computer Graphics Forum 29, 4, 1421–1429. Google ScholarDigital Library
    41. Schmid, J., Senn, M. S., Gross, M., and Sumner, R. W. 2011. OverCoat: an implicit canvas for 3D painting. In ACM Transactions on Graphics, vol. 30, 28:1–28:10. Google ScholarDigital Library
    42. Shechtman, E., Rav-Acha, A., Irani, M., and Seitz, S. 2010. Regenerative morphing. In IEEE Conference on Computer VIsion and Pattern Recognition (CVPR), 615–622.Google Scholar
    43. Sýkora, D., Ben-Chen, M., Čadík, M., Whited, B., and Simmons, M. 2011. TexToons. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering, 75–84. Google ScholarDigital Library
    44. Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In Proceedings of SIGGRAPH 2000, 479–488. Google ScholarDigital Library
    45. Wei, L.-Y., Lefebvre, S., Kwatra, V., and Turk, G. 2009. State of the Art in Example-based Texture Synthesis. In Eurographics 2009 – State of the Art Report, 93–117.Google Scholar
    46. Whited, B., Daniels, E., Kaschalk, M., Osborne, P., and Odermatt, K. 2012. Computer-assisted animation of line and paint in Disney’s Paperman. In ACM SIGGRAPH 2012 Talks, SIGGRAPH ’12, 19:1. Google ScholarDigital Library
    47. Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Transactions on Graphics 25, 3, 1221–1226. Google ScholarDigital Library
    48. Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. ACM Transactions on Graphics 22, 3, 295–302. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: