“Shadow theatre: discovering human motion from a sequence of silhouettes”

  • ©Jungdam Won and Jehee Lee

Conference:


Type:


Title:

    Shadow theatre: discovering human motion from a sequence of silhouettes

Presenter(s)/Author(s):


Session Title: EXPRESSIVE ANIMATION

Moderator(s):


Abstract:


    Shadow theatre is a genre of performance art in which the actors are only visible as shadows projected on the screen. The goal of this study is to generate animated characters, the shadows of which match a sequence of target silhouettes. This poses several challenges. The motion of multiple characters are carefully coordinated to form a target silhouette on the screen, and each character’s pose should be stable, balanced, and plausible. The resulting character animation should be smooth and coherent spatially and temporally. We formulate the problem as nonlinear constrained optimization with objectives, which were designed to generate plausible human motions. Our optimization algorithm was primarily inspired by the heuristic strategies of professional shadow theatre actors. Their know-how was studied and then incorporated into our optimization formulation. We demonstrate the effectiveness of our approach with a variety of target silhouettes and 3D fabrication of the results.

References:


    1. Agarwal, A., and Triggs, B. 2004. 3d human pose from silhouettes by relevance vector regression. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, vol. 2, II-882–II-888 Vol. 2. Google ScholarDigital Library
    2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: Shape completion and animation of people. ACM Transactions on Graphics, 408–416. Google ScholarDigital Library
    3. Appel, A. 1968. Some techniques for shading machine renderings of solids. In Proceedings of the April 30–May 2, 1968, Spring Joint Computer Conference, AFIPS ’68 (Spring), 37–45. Google ScholarDigital Library
    4. Attraction. Attraction official website. http://www.attraction.hu/.Google Scholar
    5. Bermano, A., Baran, I., Alexa, M., and Matusk, W. 2012. Shadowpix: Multiple images from self shadowing. Comp. Graph. Forum, 593–602. Google ScholarDigital Library
    6. Crow, F. C. 1977. Shadow algorithms for computer graphics. In Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’77, 242–248. Google ScholarDigital Library
    7. Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A sketching interface for articulated figure animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’03, 320–328. Google ScholarDigital Library
    8. Ek, C., Torr, P., and Lawrence, N. 2008. Gaussian process latent variable models for human pose estimation. In Machine Learning for Multimodal Interaction, vol. 4892. 132–143. Google ScholarDigital Library
    9. Fireflies. Fireflies official website. http://fireflies.com.ua/.Google Scholar
    10. Gal, R., Sorkine, O., Popa, T., Sheffer, A., and Cohen-Or, D. 2007. 3d collage: Expressive non-realistic modeling. In Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering, NPAR ’07, 7–14. Google ScholarDigital Library
    11. Goral, C. M., Torrance, K. E., Greenberg, D. P., and Battaile, B. 1984. Modeling the interaction of light between diffuse surfaces. In Proceedings of the 11th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’84, 213–222. Google ScholarDigital Library
    12. Guan, P., Weiss, A., Balan, A., and Black, M. J. 2009. Estimating human shape and pose from a single image. In Int. Conf. on Computer Vision, ICCV, 1381–1388.Google Scholar
    13. Guay, M., Cani, M.-P., and Ronfard, R. 2013. The line of action: An intuitive interface for expressive character posing. ACM Transactions on Graphics 32, 205:1–205:8. Google ScholarDigital Library
    14. Hansen, N., and Ostermeier, A. 1996. Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation. In Proceedings of IEEE International Conference on Evolutionary Computation, 312–317.Google Scholar
    15. Hausner, A. 2001. Simulating decorative mosaics. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’01, 573–580. Google ScholarDigital Library
    16. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. SIGGRAPH ’05, 1134–1141. Google ScholarDigital Library
    17. Jain, E., Sheikh, Y., and Hodgins, J. 2009. Leveraging the talent of hand animators to create three-dimensional animation. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’09, 93–102. Google ScholarDigital Library
    18. Jensen, H. W. 1996. Global illumination using photon maps. In Proceedings of the Eurographics Workshop on Rendering Techniques ’96, 21–30. Google ScholarDigital Library
    19. Kim, J., and Pellacini, F. 2002. Jigsaw image mosaics. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’02, 657–664. Google ScholarDigital Library
    20. Kim, M., Hwang, Y., Hyun, K., and Lee, J. 2012. Tiling motion patches. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’12, 117–126. Google ScholarDigital Library
    21. Kry, P. G., James, D. L., and Pai, D. K. 2002. Eigenskin: Real time large deformation character skinning in hardware. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’02, 153–159. Google ScholarDigital Library
    22. Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Transactions on Graphics (SIGGRAPH 2002) 21, 3, 491–500. Google ScholarDigital Library
    23. Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics 28, 4. Google ScholarDigital Library
    24. Lee, J. 2000. A Hierarchical Approach to Motion Analysis and Synthesis for Articulated Figures. PhD thesis, Department of Computer Science, Korea Advanced Institute of Science and Technology.Google Scholar
    25. Lin, J., Igarashi, T., Mitani, J., Liao, M., and He, Y. 2012. A sketching interface for sitting pose design in the virtual environment. IEEE Transactions on Visualization and Computer Graphics 18. Google ScholarDigital Library
    26. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. 2015. SMPL: A skinned multi-person linear model. ACM Transactions on Graphics (SIGGRAPH Asia 2015) 34, 6. Google ScholarDigital Library
    27. Magicplay. Magicplay official website. http://www.magicplay.co.kr/.Google Scholar
    28. Mattausch, O., Igarashi, T., and Wimmer, M. 2013. Freeform shadow boundary editing. 175–184.Google Scholar
    29. Mitra, N. J., and Pauly, M. 2009. Shadow art. ACM Transactions on Graphics 28, 5. Google ScholarDigital Library
    30. OGRE. Object-oriented graphics rendering engine. http://www.ogre3d.org/.Google Scholar
    31. Pellacini, F., Tole, P., and Greenberg, D. P. 2002. A user interface for interactive cinematic shadow design. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’02, 563–566. Google ScholarDigital Library
    32. Poppe, R., and Poel, M. 2006. Comparison of silhouette shape descriptors for example-based human pose recovery. In Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference on, 541–546. Google ScholarDigital Library
    33. Poppe, R. 2007. Evaluating example-based pose estimation: Experiments on the humaneva sets. In CVPR 2nd Workshop on Evaluation of Articulated Human Motion and Pose Estimation (EHuM2),.Google Scholar
    34. Ramakrishna, V., Kanade, T., and Sheikh, Y. 2012. Reconstructing 3d human pose from 2d image landmarks. In Proceedings of the 12th European Conference on Computer Vision-Volume Part IV, 573–586. Google ScholarDigital Library
    35. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., and Moore, R. 2013. Real-time human pose recognition in parts from single depth images. Commun. ACM 56, 116–124. Google ScholarDigital Library
    36. SilhouetteSquad. Silhouette Squad official website. https://www.talentcult.com/SilhouetteSquad/.Google Scholar
    37. Sminchisescu, C., and Telea, A. 2002. Human pose estimation from silhouettes, a consistent approach using distance level sets. In In WSCG International Conference on Computer Graphics, Visualization and Computer Vision.Google Scholar
    38. Sorkine, O., and Alexa, M. 2007. As-rigid-as-possible surface modeling. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing, SGP ’07, 109–116. Google ScholarDigital Library
    39. Wei, X., and Chai, J. 2011. Intuitive interactive human-character posing with millions of example poses. IEEE Comput. Graph. Appl. 31, 78–88. Google ScholarDigital Library
    40. Whitted, T. 1980. An improved illumination model for shaded display. Commun. ACM 23, 343–349. Google ScholarDigital Library
    41. Williams, L. 1978. Casting curved shadows on curved surfaces. In Proceedings of the 5th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’78, 270–274. Google ScholarDigital Library

ACM Digital Library Publication: