“Inverse-Foley animation: synchronizing rigid-body motions to sound” by Langlois and James

  • ©Timothy R. Langlois and Doug L. James

Conference:


Type:


Title:

    Inverse-Foley animation: synchronizing rigid-body motions to sound

Session/Category Title: Sound & Light

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    In this paper, we introduce Inverse-Foley Animation, a technique for optimizing rigid-body animations so that contact events are synchronized with input sound events. A precomputed database of randomly sampled rigid-body contact events is used to build a contact-event graph, which can be searched to determine a plausible sequence of contact events synchronized with the input sound’s events. To more easily find motions with matching contact times, we allow transitions between simulated contact events using a motion blending formulation based on modified contact impulses. We fine tune synchronization by slightly retiming ballistic motions. Given a sound, our system can synthesize synchronized motions using graphs built with hundreds of thousands of precomputed motions, and millions of contact events. Our system is easy to use, and has been used to plan motions for hundreds of sounds, and dozens of rigid-body models.

References:


    1. Arikan, O., and Forsyth, D. A. 2002. Interactive motion generation from examples. ACM Transactions on Graphics 21, 3 (July), 483–490. Google ScholarDigital Library
    2. Barzel, R., Hughes, J. F., and Wood, D. N. 1996. Plausible motion simulation for computer graphics animation. In EGCAS 96: Seventh International Workshop on Computer Animation and Simulation, 183–197. Google ScholarDigital Library
    3. Belta, C., and Kumar, V. 2002. An SVD-based projection method for interpolation on SE(3). Robotics and Automation, IEEE Transactions on 18, 3, 334–345.Google Scholar
    4. Bender, J., Erleben, K., and Trinkle, J. 2013. Interactive simulation of rigid body dynamics in computer graphics. In Computer Graphics Forum, Wiley Online Library.Google Scholar
    5. Bhat, K. S., Seitz, S. M., Popović, J., and Khosla, P. K. 2002. Computing the physical parameters of rigid-body motion from video. In Computer VisionECCV 2002. Springer, 551–565. Google ScholarDigital Library
    6. Bruderlin, A., and Williams, L. 1995. Motion signal processing. In Proceedings of SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, 97–104. Google ScholarDigital Library
    7. Cardle, M., Barthe, L., Brooks, S., and Robinson, P. 2002. Music-driven motion editing: local motion transformations guided by music analysis. Proceedings 20th Eurographics UK Conference, 38–44. Google ScholarDigital Library
    8. Chadwick, J. N., An, S. S., and James, D. L. 2009. Harmonic Shells: A Practical Nonlinear Sound Model for Near-Rigid Thin Shells. ACM Transactions on Graphics 28, 5 (Dec.), 119:1–119:10. Google ScholarDigital Library
    9. Chadwick, J. N., Zheng, C., and James, D. L. 2012. Precomputed acceleration noise for improved rigid-body sound. ACM Transactions on Graphics 31, 4 (July), 103:1–103:9. Google ScholarDigital Library
    10. Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. Proceedings of the 27th annual conference on, 219–228. Google ScholarDigital Library
    11. Chion, M. 1994. Audio-Vision: Sound on Screen. Columbia University Press.Google Scholar
    12. Cohen, M. 1992. Interactive spacetime control for animation. In ACM SIGGRAPH Computer Graphics, ACM, vol. 26, 293–302. Google ScholarDigital Library
    13. Duff, D. J., Morwald, T., Stolkin, R., and Wyatt, J. 2011. Physical simulation for monocular 3d model based tracking. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 5218–5225.Google Scholar
    14. Hofer, M., and Pottmann, H. 2004. Energy-minimizing splines in manifolds. ACM Transactions on Graphics 23, 3 (Aug.), 284–293. Google ScholarDigital Library
    15. Kim, T.-h., Park, S. I., and Shin, S. Y. 2003. Rhythmicmotion synthesis based on motion-beat analysis. ACM Transactions on Graphics 22, 3, 392–401. Google ScholarDigital Library
    16. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion Graphs. ACM Transactions on Graphics 21, 3, 473–482. Google ScholarDigital Library
    17. Kuffner, J. 2004. Effective sampling and distance metrics for 3D rigid body path planning. In Robotics and Automation, 2004. Proceedings. ICRA’04. 2004 IEEE International Conference on, vol. 4, IEEE, 3993–3998.Google ScholarCross Ref
    18. Lee, H.-C., and Lee, I.-K. 2005. Automatic synchronization of background music and motion in computer animation. Computer Graphics Forum 24, 3, 353–361.Google ScholarCross Ref
    19. Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Transactions on Graphics 21, 3 (July), 491–500. Google ScholarDigital Library
    20. O’Brien, J. F., Cook, P. R., and Essl, G. 2001. Synthesizing sounds from physically based motion. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 529–536. Google ScholarDigital Library
    21. O’Sullivan, C., Dingliana, J., Giang, T., and Kaiser, M. K. 2003. Evaluating the visual fidelity of physically based animations. ACM Transactions on Graphics 22, 3 (July) (July), 527–536. Google ScholarDigital Library
    22. Popović, J., Seitz, S., Erdmann, M., Popović, Z., and Witkin, A. 2000. Interactive manipulation of rigid body simulations. In Proceedings of ACM SIGGRAPH 2000, ACM Press/Addison-Wesley Publishing Co., 209–218. Google ScholarDigital Library
    23. Popović, J., Seitz, S. M., and Erdmann, M. 2003. Motion sketching for control of rigid-body simulations. ACM Transactions on Graphics 22, 4 (Oct.), 1034–1054. Google ScholarDigital Library
    24. Shin, H. J., Lee, J., Shin, S. Y., and Gleicher, M. 2001. Computer puppetry: An importance-based approach. ACM Trans. Graph. 20, 2 (Apr.), 67–94. Google ScholarDigital Library
    25. Takala, T., and Hahn, J. 1992. Sound rendering. In Computer Graphics (Proceedings of SIGGRAPH 92), 211–220. Google ScholarDigital Library
    26. Tang, D., Ngo, J. T., and Marks, J. 1995. N-body spacetime constraints. The Journal of Visualization and Computer Animation 6, 3 (July-Sept.), 143–154.Google ScholarCross Ref
    27. Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM Transactions on Graphics 26, 3 (July), 14:1–14:8. Google ScholarDigital Library
    28. Twigg, C. D., and James, D. L. 2008. Backward steps in rigid body simulation. ACM Transactions on Graphics 27, 3 (Aug.), 25:1–25:10. Google ScholarDigital Library
    29. van den Doel, K., Kry, P. G., and Pai, D. K. 2001. Foleyautomatic: Physically-based sound effects for interactive simulation and animation. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 537–544. Google ScholarDigital Library
    30. Witkin, A., and Kass, M. 1988. Spacetime constraints. In Computer Graphics (Proceedings of ACM SIGGRAPH ’88), ACM, vol. 22, 159–168. Google ScholarDigital Library
    31. Yilmaz, A., Javed, O., and Shah, M. 2006. Object tracking: A survey. ACM Computing Surveys 38, 4, 13. Google ScholarDigital Library
    32. Zheng, C., and James, D. L. 2010. Rigid-Body Fracture Sound with Precomputed Soundbanks. ACM Transactions on Graphics 29, 4 (July), 69:1–69:13. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: