“Capturing and animating skin deformation in human motion” by Park and Hodgins

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.

1. Albrecht, I., Haber, J., and Seidel, H.-P. 2003. Construction and animation of anatomically based human hand models. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 98–109. Google ScholarDigital Library
2. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Transactions on Graphics 21, 3, 612–619. Google ScholarDigital Library
3. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics 22, 3, 587–594. Google ScholarDigital Library
4. Anguelov, D., Koller, D., Pang, H., Srinivasan, P., and Thrun, S. 2004. Recovering articulated object models from 3d range data. In the 20th Conference on Uncertainty in Artificial Intelligence, 18–26. Google ScholarDigital Library
5. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. ACM Transactions on Graphics 24, 3, 408–416. Google ScholarDigital Library
6. Chadwick, J. E., Haumann, D. R., and Parent, R. E. 1989. Layered construction for deformable animated characters. Computer Graphics (Proceedings of SIGGRAPH 89) 23, 3, 243–252. Google ScholarDigital Library
7. Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3d facial animation. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 193–206. Google ScholarDigital Library
8. Choe, B., Lee, H., and Ko, H.-S. 2001. Performance-driven muscle-based facial animation. The Journal of Visualization and Computer Animation 12, 2, 67–79.Google ScholarCross Ref
9. Chuang, E., and Bregler, C. 2005. Mood swings: expressive speech animation. ACM Transactions on Graphics 24, 2, 331–347. Google ScholarDigital Library
10. Cosker, D., Paddock, S., Marshall, D., Rosin, P. L., and Rushton, S. 2004. Towards perceptually realistic talking heads: models, methods and mcgurk. In APGV 2004, 151–157. Google ScholarDigital Library
11. Dong, F., Clapworthy, G. J., Krokos, M. A., and Yao, J. 2002. An anatomy-based approach to human muscle modeling and deformation. IEEE Transactions on Visualization and Computer Graphics 8, 2, 154–170. Google ScholarDigital Library
12. Fidaleo, D., and Neumann, U. 2004. Analysis of co-articulation regions for performance-driven facial animation. Computer Animation and Virtual Worlds 15, 1, 15–26. Google ScholarDigital Library
13. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 55–66. Google ScholarDigital Library
14. Guo, Z., and Wong, K. C. 2005. Skinning with deformable chunks. Computer Graphics Forum 24, 3, 373–382.Google ScholarCross Ref
15. Horn, B. K. P. 1987. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 4, 629–642.Google ScholarCross Ref
16. Huang, K.-S., Chang, C.-F., Hsu, Y.-Y., and Yang, S.-N. 2005. Key probe: a technique for animation keyframe extraction. The Visual Computer 21, 8–10, 532–541.Google ScholarCross Ref
17. Hwang, B.-W., and Lee, S.-W. 2003. Reconstruction of partially damaged face images based on a morphable face model. IEEE Trans. Pattern Anal. Mach. Intell. 25, 3, 365–372. Google ScholarDigital Library
18. Hyun, D.-E., Yoon, S.-H., Chang, J.-W., Seong, J.-K., Kim, M.-S., and Jüttler, B. 2005. Sweep-based human deformation. The Visual Computer 21, 8–10, 542–550.Google ScholarCross Ref
19. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Transactions on Graphics 24, 3, 1134–1141. Google ScholarDigital Library
20. James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Transactions on Graphics 24, 3, 399–407. Google ScholarDigital Library
21. Kanatani, K. 1994. Analysis of 3-d rotation fitting. IEEE Trans. Pattern Anal. Mach. Intell. 16, 5, 543–549. Google ScholarDigital Library
22. Kavan, L., and Zara, J. 2005. Spherical blend skinning: A real-time deformation of articulated models. In 2005 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, ACM Press, 9–16. Google ScholarDigital Library
23. Kshirsagar, S., Molet, T., and Magnenat-Thalmann, N. 2001. Principal components of expressive speech animation. In Computer Graphics International 2001, 38–44. Google ScholarDigital Library
24. Kurihara, T., and Miyata, N. 2004. Modeling deformable human hands from medical images. In 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 355–363. Google ScholarDigital Library
25. La Torre, F. D., and Black, M. J. 2001. Dynamic coupled component analysis. In CVPR, 643–650.Google Scholar
26. Larboulette, C., Cani, M.-P., and Arnaldi, B. 2005. Dynamic skinning: adding real-time dynamic effects to an existing character animation. In Spring Conference on Computer Graphics 2005, 87–93. Google ScholarDigital Library
27. Lemos, R. R., Rokne, J., Baranoski, G. V. G., Kawakami, Y., and Kurihara, T. 2005. Modeling and simulating the deformation of human skeletal muscle based on anatomy and physiology. Computer Animation and Virtual Worlds 16, 3–4, 319–330. Google ScholarDigital Library
28. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformations: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 165–172. Google ScholarDigital Library
29. Lin, I.-C., and Ouhyoung, M. 2005. Mirror mocap: Automatic and efficient capture of dense 3d facial motion parameters from video. The Visual Computer 21, 6, 355–372.Google ScholarCross Ref
30. Lin, I.-C., Yeng, J.-S., and Ouhyoung, M. 2002. Extracting 3d facial animation parameters from multiview video clips. IEEE Computer Graphics & Applications 22, 6, 72–80. Google ScholarDigital Library
31. Lipman, Y., Sorkine, O., Levin, D., and Cohen-Or, D. 2005. Linear rotation-invariant coordinates for meshes. ACM Transactions on Graphics 24, 3, 479–487. Google ScholarDigital Library
32. Magnenat-Thalmann, N., and Thalmann, D. 2005. Virtual humans: thirty years of research, what next? The Visual Computer 21, 12, 997–1015.Google ScholarCross Ref
33. Mohr, A., and Gleicher, M. 2003. Building efficient, accurate character skins from examples. ACM Transactions on Graphics 22, 3, 562–568. Google ScholarDigital Library
34. Müller, M., Heidelberger, B., Teschner, M., and Gross, M. 2005. Meshless deformations based on shape matching. ACM Transactions on Graphics 24, 3, 471–478. Google ScholarDigital Library
35. Nedel, L. P., and Thalmann, D. 2000. Anatomic modeling of deformable human bodies. The Visual Computer 16, 6, 306–321.Google ScholarCross Ref
36. Pratscher, M., Coleman, P., Laszlo, J., and Singh, K. 2005. Outside-in anatomy based character rigging. In 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 329–338. Google ScholarDigital Library
37. Pronost, N., Dumont, G., Berillon, G., and Nicolas, G. 2006. Morphological and stance interpolations in database for simulating bipedalism of virtual humans. The Visual Computer 22, 1, 4–13.Google ScholarCross Ref
38. Sand, P., Mcmillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM Transactions on Graphics 22, 3, 578–586. Google ScholarDigital Library
39. Scheepers, F., Parent, R. E., Carlson, W. E., and May, S. F. 1997. Anatomy-based modeling of the human musculature. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 163–172. Google ScholarDigital Library
40. Seo, H., and Magnenat-Thalmann, N. 2003. An automatic modeling of human bodies from sizing parameters. In 2003 ACM Symposium on Interactive 3D Graphics, 19–26. Google ScholarDigital Library
41. Seo, H., Cordier, F., and Magnenat-Thalmann, N. 2003. Synthesizing animatable body models with parameterized shape modifications. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 120–125. Google ScholarDigital Library
42. Shum, H.-Y., Ikeuchi, K., and Reddy, R. 1995. Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 17, 9, 854–867. Google ScholarDigital Library
43. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Transactions on Graphics 24, 3, 417–425. Google ScholarDigital Library
44. Singh, K., and Kokkevis, E. 2000. Skinning characters using surface oriented free-form deformations. In Graphics Interface, 35–42.Google Scholar
45. Sloan, P.-P. J., III, C. F. R., and Cohen, M. F. 2001. Shape by example. In 2001 ACM Symposium on Interactive 3D Graphics, 135–144. Google ScholarDigital Library
46. Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-based inverse kinematics. ACM Transactions on Graphics 24, 3, 488–495. Google ScholarDigital Library
47. Sun, W., Hilton, A., Smith, R., and Illingworth, J. 2001. Layered animation of captured data. The Visual Computer 17, 8, 457–474.Google ScholarCross Ref
48. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Transactions on Visualization and Computer Graphics 11, 3, 317–328. Google ScholarDigital Library
49. Teran, J., Sifakis, E., Irving, G., and Fedkiw, R. 2005. Robust quasistatic finite elements and flesh simulation. In 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 181–190. Google ScholarDigital Library
50. Vicon Motion Systems, 2006. http://www.vicon.com/.Google Scholar
51. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Transactions on Graphics 24, 3, 426–433. Google ScholarDigital Library
52. Wallraven, C., Breidt, M., Cunningham, D. W., and Bülthoff, H. H. 2005. Psychophysical evaluation of animated facial expressions. In APGV 2005, 17–24. Google ScholarDigital Library
53. Wang, X. C., and Phillips, C. 2002. Multi-weight enveloping: Least-squares approximation techniques for skin animation. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 129–138. Google ScholarDigital Library
54. Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dynamic 3-d facial expressions. Computer Graphics Forum 23, 3, 677–686.Google ScholarCross Ref
55. Wilhelms, J., and Gelder, A. V. 1997. Anatomically based modeling. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 173–180. Google ScholarDigital Library
56. Zalewski, L., and Gong, S. 2005. 2d statistical models of facial expressions for realistic 3d avatar animation. In CVPR, 217–222. Google ScholarDigital Library
57. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3, 548–558. Google ScholarDigital Library
58. Zordan, V. B., and Horst, N. C. V. D. 2003. Mapping optical motion capture data to skeletal motion using a physical model. In 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 245–250. Google ScholarDigital Library
59. Zordan, V. B., Celly, B., Chiu, B., and Dilorenzo, P. C. 2004. Breathe easy: model and control of simulated respiration for animation. In 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 29–37. Google ScholarDigital Library