“MovieReshape: tracking and reshaping of humans in videos”
Conference:
Type(s):
Title:
- MovieReshape: tracking and reshaping of humans in videos
Session/Category Title: Image & video editing
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a system for quick and easy manipulation of the body shape and proportions of a human actor in arbitrary video footage. The approach is based on a morphable model of 3D human shape and pose that was learned from laser scans of real people. The algorithm commences by spatio-temporally fitting the pose and shape of this model to the actor in either single-view or multi-view video footage. Once the model has been fitted, semantically meaningful attributes of body shape, such as height, weight or waist girth, can be interactively modified by the user. The changed proportions of the virtual human model are then applied to the actor in all video frames by performing an image-based warping. By this means, we can now conveniently perform spatio-temporal reshaping of human actors in video footage which we show on a variety of video sequences.
References:
1. Agarwal, A., and Triggs, B. 2006. Recovering 3d human pose from monocular images. IEEE Trans. PAMI 28, 1, 44–58. Google ScholarDigital Library
2. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. In Proc. ACM SIGGRAPH ’03, 587–594. Google ScholarDigital Library
3. Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proc. SCA, 147–156. Google ScholarDigital Library
4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape completion and animation of people. In ACM TOG (Proc. SIGGRAPH ’05). Google ScholarDigital Library
5. Barrett, W. A., and Cheney, A. S. 2002. Object-based image editing. In Proc. ACM SIGGRAPH ’02, ACM, 777–784. Google ScholarDigital Library
6. Bennett, E. P., and McMillan, L. 2003. Proscenium: a framework for spatio-temporal video editing. In Proc. ACM MULTIMEDIA ’03, 177–184. Google ScholarDigital Library
7. Bǎlan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In Proc. IEEE CVPR.Google Scholar
8. Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A sketching interface for articulated figure animation. In Proc. SCA, 320–328. Google ScholarDigital Library
9. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. In ACM TOG (Proc. SIGGRAPH ’08). Google ScholarDigital Library
10. Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P. 2009. Motion capture using simultaneous skeleton tracking and surface estimation. In Proc. IEEE CVPR.Google Scholar
11. Guan, P., Weiss, A., Bǎlan, A. O., and Black, M. J. 2009. Estimating human shape and pose from a single image. In Proc. IEEE ICCV.Google Scholar
12. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In CGF (Proc. Eurographics 2008), vol. 2.Google Scholar
13. Hasler, N., Ackermann, H., Rosenhahn, B., Thormählen, T., and Seidel, H.-P. 2010. Multilinear pose and body shape estimation of dressed subjects from image sets. In Proc. IEEE CVPR.Google Scholar
14. Hornung, A., Dekkers, E., and Kobbelt, L. 2007. Character animation from 2d pictures and 3d motion data. ACM TOG 26, 1, 1. Google ScholarDigital Library
15. Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. In Proc. ACM SIGGRAPH Asia ’09, 1–10. Google ScholarDigital Library
16. Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2008. Data-driven enhancement of facial attractiveness. ACM TOG 27, 3, 1–9. Google ScholarDigital Library
17. Li, Y., Sun, J., and Shum, H.-Y. 2005. Video object cut and paste. ACM TOG 24, 3, 595–600. Google ScholarDigital Library
18. Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. In Proc. ACM SIGGRAPH ’05, 519–526. Google ScholarDigital Library
19. Müller, M., Heidelberger, B., Teschner, M., and Gross, M. 2005. Meshless deformations based on shape matching. ACM TOG 24, 3, 471–478. Google ScholarDigital Library
20. Parameswaran, V., and Chellappa, R. 2004. View independent human body pose estimation from a single perspective image. In Proc. IEEE CVPR, II: 16–22. Google ScholarDigital Library
21. Poppe, R. 2007. Vision-based human motion analysis: An overview. CVIU 108, 1–2, 4–18. Google ScholarDigital Library
22. Ritschel, T., Okabe, M., Thormählen, T., and Seidel, H.-P. 2009. Interactive reflection editing. ACM TOG (Proc. SIGGRAPH Asia ’09) 28, 5. Google ScholarDigital Library
23. Rosales, R., and Sclaroff, S. 2006. Combining generative and discriminative models in a framework for articulated pose estimation. Int. J. Comput. Vision 67, 3, 251–276. Google ScholarDigital Library
24. Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM TOG (Proc. SIGGRAPH ’08) 27, 3, 1–9. Google ScholarDigital Library
25. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM TOG 25, 3, 533–540. Google ScholarDigital Library
26. Scholz, V., and Magnor, M. 2006. Texture replacement of garments in monocular video sequences. In Proc. EGSR, 305–312. Google ScholarDigital Library
27. Scholz, V., El-Abed, S., Seidel, H.-P., and Magnor, M. A. 2009. Editing object behaviour in video sequences. CGF 28, 6, 1632–1643.Google ScholarCross Ref
28. Seo, H., and Magnenat-Thalmann, N. 2004. An example-based approach to human body manipulation. Graph. Models 66, 1, 1–23. Google ScholarDigital Library
29. Sigal, L., Balan, A. O., and Black, M. J. 2007. Combined discriminative and generative articulated pose and nonrigid shape estimation. In Proc. NIPS.Google Scholar
30. Stolfi, J. 1991. Oriented Projective Geometry: A Framework for Geometric Computation. Academic Press. Google ScholarDigital Library
31. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM TOG 24, 3, 426–433. Google ScholarDigital Library
32. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM TOG (Proc. SIGGRAPH ’08). Google ScholarDigital Library
33. Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. In Proc. ACM SIGGRAPH ’05, ACM, 585–594. Google ScholarDigital Library
34. Wang, J., Drucker, S. M., Agrawala, M., and Cohen, M. F. 2006. The cartoon animation filter. ACM TOG (Proc. SIGGRAPH ’06), 1169–1173. Google ScholarDigital Library
35. Wang, H., Xu, N., Raskar, R., and Ahuja, N. 2007. Videoshop: A new framework for spatio-temporal video editing in gradient domain. Graph. Models 69, 1, 57–70. Google ScholarDigital Library
36. Wei, X., and Chai, J. 2010. Videomocap: modeling physically realistic human motion from monocular video sequences. ACM TOG (Proc. SIGGRAPH ’10) 29, 4. Google ScholarDigital Library
37. Zhou, S., Fu, H., Liu, L., Cohen-Or, D., and Han, X. 2010. Parametric reshaping of human bodies in images. ACM TOG (Proc. SIGGRAPH ’10) 29, 4. Google ScholarDigital Library
