“Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models” by Alexanderson, Nagy, Beskow and Henter

  • ©Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter

Conference:


Type(s):


Title:

    Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models

Session/Category Title:   Character Animation: Knowing What To Do With Your Hands


Presenter(s)/Author(s):


Moderator(s):



Abstract:


    We present diffusion models for audio-driven synthesis of high-quality 3D motion, with dancing and co-speech gesticulation as example applications. Our architecture uses Conformers and translation-invariant self attention. Optional style control is provided through classifier-free guidance. We also demonstrate results on path-driven locomotion and a novel formulation of diffusion-model ensembles.


Additional Images:

©Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter

Overview Page: