“Three-Dimensional Proxies for Hand-Drawn Characters” by Jain, Sheikh, Mahler and Hodgins

Drawing shapes by hand and manipulating computer-generated objects are the two dominant forms of animation. Though each medium has its own advantages, the techniques developed for one medium are not easily leveraged in the other medium because hand animation is two-dimensional, and inferring the third dimension is mathematically ambiguous. A second challenge is that the character is a consistent three-dimensional (3D) object in computer animation while hand animators introduce geometric inconsistencies in the two-dimensional (2D) shapes to better convey a character’s emotional state and personality. In this work, we identify 3D proxies to connect hand-drawn animation and 3D computer animation. We present an integrated approach to generate three levels of 3D proxies: single-points, polygonal shapes, and a full joint hierarchy. We demonstrate how this approach enables one medium to take advantage of techniques developed for the other; for example, 3D physical simulation is used to create clothes for a hand-animated character, and a traditionally trained animator is able to influence the performance of a 3D character while drawing with paper and pencil.

Agarwal, A. and Triggs, B. 2006. Recovering 3d human pose from monocular images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1, 44–58. Google ScholarDigital Library
Baran, I., Vlasic, D., Grinspun, E., and Popović, J. 2009. Semantic deformation transfer. ACM Trans. Graph. 28, 3, 1–6. Google ScholarDigital Library
Bourdev, L. and Malik, J. 2009. Poselets: Body part detectors trained using 3d human pose annotations. In Proceedings of the IEEE International Conference on Computer Vision.Google Scholar
Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
Cooper, D. 2002. 2D/3D Hybrid character animation on Spirit. In Proceedings of the ACM SIGGRAPH ’02 Conference Abstracts and Applications Conference. Google ScholarDigital Library
Corrêa, W. T., Jensen, R. J., Thayer, C. E., and Finkelstein, A. 1998. Texture mapping for cel animation. In Proceedings of the ACM SIGGRAPH ’98. 435–446. Google ScholarDigital Library
Culhane, S. 1990. Animation From Script to Screen. St. Martin’s Press, New York.Google Scholar
Daniels, E. 1999. Deep canvas in Disney’s Tarzan. In Proceedings of the ACM SIGGRAPH ’99 Conference Abstracts and Applications. Google ScholarDigital Library
Davis, J., Agrawala, M., Chuang, E., Popovic, Z., and Salesin, D. H. 2003. A sketching interface for articulated figure animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 320–328. Google ScholarDigital Library
Demirdjian, D., Ko, T., and Darrel, T. 2003. Constraining human body tracking. In Proceedings of the IEEE International Conference on Computer Vision. Google ScholarDigital Library
Elgammal, A. and Lee, C. 2004. Inferring 3d body pose from silhouettes using activity manifold learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
Ellis, D. 2003. Dynamic time warp (DTW) in Matlab. www.ee.columbia.edu/~dpwe/resources/matlab/dtw/.Google Scholar
Forsyth, D. A., Arikan, O., Ikemoto, L., O’Brien, J., and Ramanan, D. 2005. Computational studies of human motion: part 1, tracking and motion synthesis. Found. Trends Comput. Graph. Vis. 1, 2-3, 77–254. Google ScholarDigital Library
Gleicher, M. 1998. Retargetting motion to new characters. In Proceedings of the ACM SIGGRAPH ’98. 33–42. Google ScholarDigital Library
Grochow, K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Implicit surface joint limits to constrain video-based motion capture. ACM Trans. Graph. 23, 3, 522–531.Google ScholarDigital Library
Hartley, R. and Zisserman, A. 2003. Multiple View Geometry, 2 ed. Cambridge University Press. Google ScholarDigital Library
Herda, L., Urtasun, R., and Fua, P. 2004. Implicit surface joint limits to constrain video-based motion capture. In Proceedings of the European Conference on Computer Vision. 405–418.Google Scholar
Hornung, A., Dekkers, E., and Kobbelt, L. 2007. Character animation from 2d pictures and 3d motion data. ACM Trans. Graph. 26, 1, 1–9. Google ScholarDigital Library
Ikemoto, L., Arikan, O., and Forsyth, D. 2009. Generalizing motion edits with gaussian processes. ACM Trans. Graph. 28, 1, 1–12. Google ScholarDigital Library
Jain, E., Sheikh, Y., and Hodgins, J. K. 2009. Leveraging the talent of hand animators to create three-dimensional animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Google ScholarDigital Library
Jain, E., Sheikh, Y., Mahler, M., and Hodgins, J. 2010. Augmenting hand animation with three-dimensional secondary motion. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Google ScholarDigital Library
Johnston, O. and Thomas, F. 1995. The Illusion of Life: Disney Animation. Disney Editions.Google Scholar
Johnston, S. F. 2002. Lumo: Illumination for cel animation. In Proceedings of the (NPAR’02) Symposium on Non-Photorealistic Animation and Rendering. 45–52. Google ScholarDigital Library
Lasseter, J. 1994. Tricks to animating characters with a computer. In ACM SIGGRAPH Course Notes. Google ScholarDigital Library
Lee, H. J. and Chen, Z. 1985. Determination of 3d human body postures from a single view. Comput. Vis. Graph. Image Process. 30, 148–168.Google ScholarCross Ref
Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491–500. Google ScholarDigital Library
Li, Y., Gleicher, M., Xu, Y.-Q., and Shum, H.-Y. 2003. Stylizing motion with drawings. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 309–319. Google ScholarDigital Library
Moeslund, T. B. and Granum, E. 2006. A survey of computer vision-based human motion capture. Comput. Vis. Image Understand. 81, 3, 231–268. Google ScholarDigital Library
Petrović, L., Fujito, B., Williams, L., and Finkelstein, A. 2000. Shadows for cel animation. In Proceedings of the ACM SIGGRAPH’00. 511–516. Google ScholarDigital Library
Rademacher, P. 1999. View-dependent geometry. In Proceedings of the ACM SIGGRAPH’99 Conference. 439–446. Google ScholarDigital Library
Ramanan, D., Forsyth, D., and Zisserman, A. 2004. Tracking people by learning their appearance. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1, 65–81. Google ScholarDigital Library
Rivers, A., Igarashi, T., and Durand, F. 2010. 2.5d cartoon models. ACM Trans. Graph. 29, 4. Google ScholarDigital Library
Robertson, B. 1998. Mixed media. Comput. Graph. World, 32–35.Google Scholar
Rosenhahn, B., Brox, T., Cremers, D., and Seidel, H.-P. 2007a. Online smoothing for markerless motion capture. In Proceedings of the DAGM Symposium on Pattern Recognition 4713, 163–172. Google ScholarDigital Library
Rosenhahn, B., Brox, T., Cremers, D., and Seidel, H.-P. 2008. Staying well grounded in markerless motion capture. In Proceedings of the DAGM Symposium on Pattern Recognition 5096, 385–395. Google ScholarDigital Library
Rosenhahn, B., Brox, T., and Seidel, H.-P. 2007b. Scaled motion dynamics for markerless motion capture. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
Safonova, A., Hodgins, J. K., and Pollard, N. S. 2004. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23, 3. Google ScholarDigital Library
Sakoe, H. and Chiba, S. 1990. Dynamic programming algorithm optimization for spoken word recognition. Read. Speech Recogn. 159–165. Google ScholarDigital Library
Sidenbladh, H., Black, M., and Sigal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In Proceedings of the European Conference on Computer Vision. Google ScholarDigital Library
Sidenbladh, H., Black, M. J., and Fleet, D. J. 2000. Stochastic tracking of 3d human figures using 2d image motion. In Proceedings of the European Conference on Computer Vision, 702–718. Google ScholarDigital Library
Sminchisescu, C., Kanaujia, A., and Metaxas, D. 2005. Discriminative density propagation for 3d human motion estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
Sminchisescu, C. and Triggs, B. 2003. Estimating articulated human motion with covariance scaled sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
Stam, J. 2009. Nucleus: Towards a unified dynamics solver for computer graphics. In Proceedings of the IEEE International Conference on Computer-Aided Design and Computer Graphics, 1–11.Google ScholarCross Ref
Sýkora, D., Sedláček, D., Jinchao, S., Dingliana, J., and Collins, S. 2010. Adding depth to cartoons using sparse depth (in)equalities. Comput. Graph. Forum 29, 2, 615–623.Google ScholarCross Ref
Taylor, C. J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Comput. Vis. Image Understand. 80, 349–363. Google ScholarDigital Library
Urtasun, R., Fleet, D. J., and Fua, P. 2006. Temporal motion models for monocular and multiview 3d human body tracking. Comput. Vis. Image Understand. 104, 2, 157–177. Google ScholarDigital Library
Wei, X. and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Trans. Graph. 29, 4, 1–10. Google ScholarDigital Library
Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. 1997. Multiperspective panoramas for cel animation. In Proceedings of the ACM SIGGRAPH’97 Conference, 243–250. Google ScholarDigital Library
Wu, Y., Hua, G., and Yu, T. 2003. Tracking articulated body by dynamic markov network. In Proceedings of the IEEE International Conference on Computer Vision. Google ScholarDigital Library
Zhao, L. and Safonova, A. 2009. Achieving good connectivity in motion graphs. Graph. Models 71, 4, 139–152. Google ScholarDigital Library