“Video-based reconstruction of animatable human characters”
Conference:
Type(s):
Title:
- Video-based reconstruction of animatable human characters
Session/Category Title: Curves, characters & crowds
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a new performance capture approach that incorporates a physically-based cloth model to reconstruct a rigged fully-animatable virtual double of a real person in loose apparel from multi-view video recordings. Our algorithm only requires a minimum of manual interaction. Without the use of optical markers in the scene, our algorithm first reconstructs skeleton motion and detailed time-varying surface geometry of a real person from a reference video sequence. These captured reference performance data are then analyzed to automatically identify non-rigidly deforming pieces of apparel on the animated geometry. For each piece of apparel, parameters of a physically-based real-time cloth simulation model are estimated, and surface geometry of occluded body regions is approximated. The reconstructed character model comprises a skeleton-based representation for the actual body parts and a physically-based simulation model for the apparel. In contrast to previous performance capture methods, we can now also create new real-time animations of actors captured in general apparel.
References:
1. Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and posedependent body shape variation for real-time synthesis. In Proc. of SCA, 147–156. Google ScholarDigital Library
2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape completion and animation of people. In ACM TOG (Proc. SIGGRAPH ’05). Google ScholarDigital Library
3. Balan, A. O., and Black, M. J. 2008. The naked truth: Estimating body shape under clothing. In ECCV (2), 15–29. Google ScholarDigital Library
4. Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3d characters. ACM TOG (Proc. SIGGRAPH ’07). Google ScholarDigital Library
5. Bhat, K. S., Twigg, C. D., Hodgins, J. K., Khosla, P. K., Popović, Z. Z., and Seitz, S. M. 2003. Estimating cloth simulation parameters from video. In Proc. of SCA. Google ScholarDigital Library
6. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM TOG (Proc. SIGGRAPH ’08). Google ScholarDigital Library
7. Carranza, J., Theobalt, C., Magnor, M., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. In ACM TOG (Proc. SIGGRAPH ’03). Google ScholarDigital Library
8. Choi, K.-J., and Ko, H.-S. 2005. Research problems in clothing simulation. Computer-Aided Design 37, 6, 585–592. Google ScholarDigital Library
9. de Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008. Automatic conversion of mesh animations into skeleton-based animations. Proc. Eurographics EG’08.Google Scholar
10. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. In ACM TOG (Proc. SIGGRAPH’08). Google ScholarDigital Library
11. Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P. 2009. Motion capture using simultaneous skeleton tracking and surface estimation. In Proc. IEEE CVPR.Google Scholar
12. Hansen, N., Niederberger, A. S. P., Guzzella, L., and Koumoutsakos, P. 2009. A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE TEC 13, 1. Google ScholarDigital Library
13. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. CGF (Proc. Eurographics 2009) 28, 2.Google Scholar
14. Hasler, N., Stoll, C., Rosenhahn, B., Thormählen, T., and Seidel, H.-P. 2009. Estimating body shape of dressed humans. Comput. Graph. 33, 3, 211–216. Google ScholarDigital Library
15. James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM TOG (Proc. SIGGRAPH’05). Google ScholarDigital Library
16. Kavan, L., Collins, S., Žára, J., and O’Sullivan, C. 2007. Skinning with dual quaternions. In Symposium on Interactive 3D graphics and games, 39–46. Google ScholarDigital Library
17. Kawabata, S. 1980. The standardization and analysis of hand evaluation. Textile Machinery Society of Japan.Google Scholar
18. Kircher, S., and Garland, M. 2006. Editing arbitrarily deforming surface animations. In ACM TOG (Proc. SIGGRAPH’06). Google ScholarDigital Library
19. Lowe, D. G. 1999. Object recognition from local scale-invariant features. In Proc. ICCV, vol. 2, 1150ff. Google ScholarDigital Library
20. Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. 2000. Image-based visual hulls. In ACM TOG (Proc. SIGGRAPH’00). Google ScholarDigital Library
21. Müller, M., Heidelberger, B., Hennix, M., and Ratcliff, J. 2007. Position based dynamics. Journal Vis. Com. Google ScholarDigital Library
22. Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM TOG (Proc. SIGGRAPH’08). Google ScholarDigital Library
23. Popa, T., Zhou, Q., Bradley, D., Kraevoy, V., Fu, H., Sheffer, A., and Heidrich, W. 2009. Wrinkling captured garments using space-time data-driven deformation. Computer Graphics Forum (Proc. Eurographics) 28, 2.Google ScholarCross Ref
24. Poppe, R. 2007. Vision-based human motion analysis: An overview. CVIU 108, 1–2, 4–18. Google ScholarDigital Library
25. Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. In Proc. Eurographics EG’03.Google Scholar
26. Sand, P., McMillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM TOG (Proc. SIGGRAPH’03). Google ScholarDigital Library
27. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Proc. Eurographics EG’05.Google Scholar
28. Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2008. Example-based dynamic skinning in real time. ACM TOG (Proc. SIGGRAPH ’08). Google ScholarDigital Library
29. Starck, J., and Hilton, A. 2007. Surface capture for performance based animation. IEEE CGAA 27(3), 21–31. Google ScholarDigital Library
30. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM TOG (Proc. SIGGRAPH ’08). Google ScholarDigital Library
31. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., and Matusik, W. 2009. Dynamic shape capture using multi-view photometric stereo. In ACM TOG (Proc. SIGGRAPH Asia ’09). Google ScholarDigital Library
32. Waschbüsch, M., Würmlin, S., Cotting, D., Sadlo, F., and Gross, M. 2005. Scalable 3D video of dynamic scenes. In Proc. Pacific Graphics, 629–638.Google Scholar
33. White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. In ACM TOG (Proc. SIGGRAPH’07). Google ScholarDigital Library
34. Xu, W., Zhou, K., Yu, Y., Tan, Q., Peng, Q., and Guo, B. 2007. Gradient domain editing of deforming mesh sequences. In ACM TOG (Proc. SIGGRAPH ’07). Google ScholarDigital Library
35. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM TOG (Proc. SIGGRAPH ’04). Google ScholarDigital Library
