“Data-driven finger motion synthesis for gesturing characters”
Conference:
Type(s):
Title:
- Data-driven finger motion synthesis for gesturing characters
Session/Category Title: Motion Capture and Synthesis
Presenter(s)/Author(s):
Abstract:
Capturing the body movements of actors to create animations for movies, games, and VR applications has become standard practice, but finger motions are usually added manually as a tedious post-processing step. In this paper, we present a surprisingly simple method to automate this step for gesturing and conversing characters. In a controlled environment, we carefully captured and post-processed finger and body motions from multiple actors. To augment the body motions of virtual characters with plausible and detailed finger movements, our method selects finger motion segments from the resulting database taking into account the similarity of the arm motions and the smoothness of consecutive finger motions. We investigate which parts of the arm motion best discriminate gestures with leave-one-out cross-validation and use the result as a metric to select appropriate finger motions. Our approach provides good results for a number of examples with different gesture types and is validated in a perceptual experiment.
References:
1. Athitsos, V., and Sclaroff, S. 2003. Estimating 3d hand pose from a cluttered image. In IEEE Computer Vision and Pattern Recognition (CVPR), vol. 2, 432–439.
2. Cassell, J., Vilhjálmsson, H. H., and Bickmore, T. 2001. Beat: the behavior expression animation toolkit. In ACM SIGGRAPH 2001, 477–486.
3. CyberGlove Systems, 2012. www.cyberglovesystems.com/products/cyberglove-ii/overview.
4. ElKoura, G., and Singh, K. 2003. Handrix: animating the human hand. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 110–119.
5. Hoyet, L., Ryall, K., McDonnell, R., and O’Sullivan, C. 2012. Sleight of hand: perception of finger motion from reduced marker sets. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, I3D ’12, 79–86.
6. Jörg, S., Hodgins, J., and O’Sullivan, C. 2010. The perception of finger motions. In Proceedings of the 7th ACM Symposium on Applied Perception in Graphics and Visualization (APGV 2010), 129–133.
7. Kahlesz, F., Zachmann, G., and Klein, R. 2004. Visual-fidelity dataglove calibration. In Computer Graphics International, 403–410.
8. Kendon, A. 2004. Gesture — Visible Action as Utterance. Cambridge University Press, Cambridge, UK.
9. Kipp, M., Neff, M., and Albrecht, I. 2007. An annotation scheme for conversational gestures: How to economically capture timing and form. In Journal on Language Resources and Evaluation, vol. 41, 325–339.
10. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. ACM Transactions on Graphics 21, 3, 473–482.
11. Kry, P. G., and Pai, D. K. 2006. Interaction capture and synthesis. ACM Transactions on Graphics 25, 3, 872–880.
12. Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Transactions on Graphics 21, 3, 491–500.
13. Levine, S., Theobalt, C., and Koltun, V. 2009. Real-time prosody-driven synthesis of body language. ACM Transactions on Graphics 28, 172:1–172:10.
14. Levine, S., Krähenbühl, P., Thrun, S., and Koltun, V. 2010. Gesture controllers. ACM Transactions on Graphics 29, 4, 1–11.
15. Liu, C. K. 2008. Synthesis of interactive hand manipulation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 163–171.
16. Liu, C. K. 2009. Dextrous manipulation from a grasping pose. ACM Transactions on Graphics 28, 3, 3:1–3:6.
17. Majkowska, A., Zordan, V. B., and Faloutsos, P. 2006. Automatic splicing for hand and body animations. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 309–316.
18. McNeill, D. 1992. Hand and Mind: what gestures reveal about thought. The University of Chicago Press.
19. Measurand, 2012. www.shapehand.com.
20. Neff, M., and Seidel, H.-P. 2006. Modeling relaxed hand shape for character animation. In Articulated Motion and Deformable Objects, vol. 4069 of Lecture Notes in Computer Science. Springer, 262–270.
21. Neff, M., Kipp, M., Albrecht, I., and Seidel, H.-P. 2008. Gesture modeling and animation based on a probabilistic recreation of speaker style. ACM Transactions on Graphics 27, 1, 1–24.
22. Palastanga, N., Field, D., and Soames, R. 2006. Anatomy and Human Movement — Structure and Function, 5 ed. Butter-worth Heinemann Elsevier.
23. Pollard, N. S., and Zordan, V. B. 2005. Physically based grasping control from example. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 311–318.
24. Rose, C., Bodenheimer, B., and Cohen, M. F. 1998. Verbs and adverbs: Multidimensional motion interpolation using radial basis functions. IEEE Computer Graphics and Applications 18, 32–40.
25. Samadani, A.-A., DeHart, B. J., Robinson, K., Kulić, D., Kubican, E., and Gorbet, R. 2011. A study of human performance in recognizing expressive hand movements. IEEE International Symposium on Robot and Human Interactive Communication, 93–100.
26. Stone, M., DeCarlo, D., Oh, I., Rodriguez, C., Stere, A., Lees, A., and Bregler, C. 2004. Speaking with hands: creating animated conversational characters from recordings of human performance. ACM Transactions on Graphics 23, 3, 506–513.
27. Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 319–328.
28. Wang, R. Y., and Popović, J. 2009. Real-time hand-tracking with a color glove. ACM Transactions on Graphics 28, 3, 1–8.
29. Wang, R. Y., Paris, S., and Popovic, J. 2011. 6d hands: markerless hand-tracking for computer aided design. In ACM Symposium on User Interface Software and Technology, 549–558.
30. Ye, Y., and Liu, C. K. 2012. Synthesis of detailed hand manipulations using contact sampling. ACM Transactions on Graphics 31, 4.
31. Zhao, W., Chai, J., and Xu, Y.-Q. 2012. Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation.
