“Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters” by Omine and Kawabata
Conference:
Type(s):
Title:
- Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters
Session/Category Title:
- Animation & Simulation
Presenter(s)/Author(s):
Abstract:
This paper introduces innovative methods for generating gestures, facial expressions, and exaggerated emotional expressions for non-photorealistic characters using comics-extracted expression data and dialogue-specific semantic gestures for conversational AI, achieving significantly enhanced user satisfaction compared to a state-of-the-art photorealistic method.
References:
[1] Kiyoharu Aizawa, Azuma Fujimoto, Atsushi Otsubo, Toru Ogawa, Yusuke Matsui, Koki Tsubota, and Hikaru Ikuta. 2020. Building a Manga Dataset Manga109 with Annotations for Multimedia Applications. IEEE MultiMedia 27, 2 (2020), 8–18.
[2] Haiyang Liu, Zihao Zhu, and Giorgio et al. Becherini. 2024. EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1144–1154.
[3] Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. 2022. BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis. arXiv preprint arXiv:https://arXiv.org/abs/2203.05297 (2022).
[4] Kaisiyuan Wang, Qianyi Wu, Linsen Song, Zhuoqian Yang, Wayne Wu, Chen Qian, Ran He, Yu Qiao, and Chen Change Loy. 2020. MEAD: A Large-scale Audio-visual Dataset for Emotional Talking-face Generation. In ECCV.
