“Decaf: Monocular Deformation Capture for Face and Hand Interactions” by Shimada, Golyanik, Perez and Theobalt
Conference:
Type(s):
Title:
- Decaf: Monocular Deformation Capture for Face and Hand Interactions
Session/Category Title: Humans & Characters
Presenter(s)/Author(s):
Abstract:
Existing methods for 3D tracking from monocular RGB videos predominantly consider articulated and rigid objects (e.g., two hands or humans interacting with rigid environments). Modelling dense non-rigid object deformations in this setting (e.g., when hand are interacting with a face), remained largely unaddressed so far, although such effects can improve the realism of the downstream applications such as AR/VR, 3D virtual avatar communications, and character animations. This is due to the severe ill-posedness of the monocular view setting and the associated challenges (e.g., in acquiring a dataset for training and evaluation or obtaining the reasonable non-uniform stiffness of the deformable object). While it is possible to na\”{i}vely track multiple non-rigid objects independently using 3D templates or parametric 3D models
References:
unnatural intra-object collisions and missing or implausible deformations. Hence
