“Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models” by Alexanderson, Nagy, Beskow and Henter
Conference:
Type:
Title:
- Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Session/Category Title: Character Animation: Knowing What To Do With Your Hands
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present diffusion models for audio-driven synthesis of high-quality 3D motion, with dancing and co-speech gesticulation as example applications. Our architecture uses Conformers and translation-invariant self attention. Optional style control is provided through classifier-free guidance. We also demonstrate results on path-driven locomotion and a novel formulation of diffusion-model ensembles.
Additional Images:
![©Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter](https://history.siggraph.org/wp-content/uploads/2024/02/2023-Tech-Papers-Alexanderson_Listen-Denoise-Action-Audio-Driven-Motion-Synthesis-with-Diffusion-Models.jpg)