“Text-Based Motion Synthesis with a Hierarchical Two-Stream RNN” by Ghosh, Cheema, Oguz, Theobalt and Slusallek – ACM SIGGRAPH HISTORY ARCHIVES

“Text-Based Motion Synthesis with a Hierarchical Two-Stream RNN” by Ghosh, Cheema, Oguz, Theobalt and Slusallek

  • ©

  • ©

  • ©

Conference:


Type(s):


Entry Number: 30

Title:

    Text-Based Motion Synthesis with a Hierarchical Two-Stream RNN

Presenter(s)/Author(s):



Abstract:


    We present a learning-based method for generating animated 3D pose sequences depicting multiple sequential or superimposed actions provided in long, compositional sentences.We propose a hierarchical two-stream sequential model to explore a finer joint-level mapping between natural language sentences and the corresponding 3D pose sequences of the motions. We learn two manifold representations of the motion — one each for the upper body and the lower body movements. We evaluate our proposed model on the publicly available KIT Motion-Language Dataset containing 3D pose data with human-annotated sentences. Experimental results show that our model advances the state-of-the-art on text-based motion synthesis in objective evaluations by a margin of 50%.


Acknowledgements:


    This research is funded by the BMBF grants XAINES (01|W20005) and IMPRESS (01|S20076), the EU Horizon 2020 grant Carousel+ (101017779), an IMPRS-CS Fellowship. Computational resources were provided by the BMWi grants 01MK20004D and 01MD19001B.


PDF:



ACM Digital Library Publication:



Overview Page:



Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org