“Hybrid Skeletal-Surface Motion Graphs for Character Animation From 4D Performance Capture” by Huang, Tejera, Collomosse and Hilton

  • ©Peng Huang, Margara Tejera, John Collomosse, and Adrian Hilton

Conference:


Type:


Title:

    Hybrid Skeletal-Surface Motion Graphs for Character Animation From 4D Performance Capture

Session/Category Title: Taking Control

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We present a novel hybrid representation for character animation from 4D Performance Capture (4DPC) data which combines skeletal control with surface motion graphs. 4DPC data are temporally aligned 3D mesh sequence reconstructions of the dynamic surface shape and associated appearance from multiple-view video. The hybrid representation supports the production of novel surface sequences which satisfy constraints from user-specified key-frames or a target skeletal motion. Motion graph path optimisation concatenates fragments of 4DPC data to satisfy the constraints while maintaining plausible surface motion at transitions between sequences. Space-time editing of the mesh sequence using a learned part-based Laplacian surface deformation model is performed to match the target skeletal motion and transition between sequences. The approach is quantitatively evaluated for three 4DPC datasets with a variety of clothing styles. Results for key-frame animation demonstrate production of novel sequences that satisfy constraints on timing and position of less than 1% of the sequence duration and path length. Evaluation of motion-capture-driven animation over a corpus of 130 sequences shows that the synthesised motion accurately matches the target skeletal motion. The combination of skeletal control with the surface motion graph extends the range and style of motion which can be produced while maintaining the natural dynamics of shape and appearance from the captured performance.

References:


    1. O. Arikan and D. Forsyth. 2002. Interactive motion generation from examples. ACM Trans. Graph. 21, 3, 483–490.
    2. O. Arikan, D. Forsyth, and J. O’Brien. 2003. Motion synthesis from annotations. ACM Trans. Graph. 22, 3, 402–408.
    3. M. Botsch and O. Sorkine. 2008. On linear variational surface deformation methods. IEEE Trans. Vis. Comput. Graph. 14, 1, 213–230.
    4. C. Bregler, M. Covell, and M. Slaney. 1997. Video rewrite: Driving visual speech with audio. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’97). ACM, New York, NY, USA, 353–360.
    5. C. Budd, P. Huang, M. Klaudiny, and A. Hilton. 2012. Global non-rigid alignment of surface sequences. Int. J. Comput. Vis. 102, 1–3, 256–270.
    6. D. Casas, M. Tejera, J.-Y. Guillemaut, and A. Hilton. 2013. Interactive animation of 4D performance capture. IEEE Trans. Vis. Comput. Graph. 19, 5, 762–73.
    7. D. Casas, M. Volino, J. Collomosse, and A. Hilton. 2014. 4D video textures for interactive character appearance. Comput. Graph. Forum 33, 2, 371–380.
    8. G. L. Cmu. Carnegie mellon university motion capture database. http://mocap.cs.cmu.edu/.
    9. T. Ezzat and G. Geiger. 2002. Trainable videorealistic speech animation. ACM Trans. Graph. 21, 3, 388–398.
    10. M. Flagg, A. Nakazawa, Q. Zhang, S. B. Kang, Y. K. Ryu, I. Essa, and J. M. Rehg. 2009. Human video textures. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D’09). ACM, New York, NY, USA, 199–206.
    11. J. Gall, C. Stoll, and E. D. Aguiar. 2009. Motion capture using joint skeleton tracking and surface estimation. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). 1746–1753.
    12. M. Gleicher, H. J. Shin, L. Kovar, and A. Jepsen. 2003. Snap-together motion: Assembling run-time animations. ACM Trans. Graph. 22, 3, 181–188.
    13. R. Heck and M. Gleicher. 2007. Parametric motion graphs. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D’07). ACM Press, New York, 129–136.
    14. A. Hornung and E. Dekkers. 2007. Character animation from 2D pictures and 3D motion data. ACM Trans. Graph. 26, 1, 1–9.
    15. P. Huang, A. Hilton, and J. Starck. 2009. Human motion synthesis from 3D video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). 1478–1485.
    16. P. Huang, A. Hilton, and J. Starck. 2010. Shape +. Int. J. Comput. Vis. 89, 2–3, 362–381.
    17. A. J. Hunt and A. W. Black. 1996. Unit selection in a concatenative speech synthesis system. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’96). 373–376.
    18. D. L. James, C. D. Twigg, A. Cove, and R. Y. Wang. 2007. Mesh ensemble motion graphs. ACM Trans. Graph. 26, 4.
    19. S. Kircher and M. Garland. 2008. Free-form motion processing. ACM Trans. Graph. 27, 2, 1–13.
    20. L. Kovar, M. Gleicher, and F. Pighin. 2002. Motion graphs. ACM Trans. Graph. 21, 3, 473–482.
    21. A. Laurentini. 1994. The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 2, 150–162.
    22. J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491–500.
    23. Y. Rubner, C. Tomasi, and L. Guibas. 1998. A metric for distributions with applications to image databases. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’98). 59–66.
    24. A. Schodl and I. Essa. 2002. Controlled animation of video sprites. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’02). ACM Press, New York, 121–127.
    25. A. Schodl, R. Szeliski, D. Salesin, and I. Essa. 2000. Video textures. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’00). ACM Press, 489–498.
    26. O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel. 2004. Laplacian surface editing. In Proceedings of the Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP’04). ACM Press, New York, 175–184.
    27. J. Starck and A. Hilton. 2007. Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27, 3, 21–31.
    28. J. Starck, G. Miller, and A. Hilton. 2005. Video-based character animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’05). ACM Press, New York, 49–58.
    29. R. W. Sumner, M. Zwicker, C. Gotsman, and J. Popovic. 2005. Mesh-based inverse kinematics. ACM Trans. Graph. 24, 3, 488–495.
    30. M. Tejera, D. Casas, and A. Hilton. 2013. Animation control of surface motion capture. IEEE Trans. Cybernet. 43, 1532–1545.
    31. M. Tejera and A. Hilton. 2013. Learning part-based models for animation from surface motion capture. In Proceedings of the International Conference on 3D Vision (3DV’13). 159–166.
    32. F. Xu, Y. Liu, J. Tompkin, G. Bharaj, J. Kautz, C. Theobalt, C. Stoll, Q. Dai, and H.-P. Seidel. 2011. Video-based characters creating new human performances from a multi-view video database. ACM Trans. Graph. 30, 4, 32:1–32:10.
    33. W. Xu, K. Zhou, Y. Yu, Q. Tan, Q. Peng, and B. Guo. 2007. Gradient domain editing of deforming mesh sequences. ACM Trans. Graph. 26, 3.

ACM Digital Library Publication:


Overview Page: