“Virtual Hands in VR: Motion Capture, Synthesis, and Perception” by Ye, Neff, Mueller and Zordan
Conference:
Type(s):
Entry Number: 09
Title:
- Virtual Hands in VR: Motion Capture, Synthesis, and Perception
Course Organizer(s):
Presenter(s)/Author(s):
Abstract:
Prerequisites
While some parts of our course are accessible to a broad audience, other parts are aimed for an audience with some general knowledge in computer animation. For example, newer motion capturing techniques use deep learning and some motion synthesis methods use physics-based animation or motion graphs.
Intended Audience
Our target audience includes researchers at all levels, developers especially of motion capture devices, designers of virtual reality (VR) applications, as well as users of games and applications in virtual environments. Several topics are accessible for a wider audience without previous knowledge, such as current methods of interactions with virtual hands, the perception of virtual hands including the virtual hand illusion, as well as some of the basics of motion capturing hands. Other parts are aimed for an audience with some general knowledge in computer graphics, such as newer motion capturing techniques using deep learning or motion synthesis methods using physics-based animation.
Abstract
We use our hands every day: to grasp a cup of coffee, write text on a keyboard, or signal that we are about to say something important. We use our hands to interact with our environment and to help us communicate with each other without thinking about it. Wouldn’t it be great to be able to do the same in virtual reality? However, accurate hand motions are not trivial to capture. In this course, we present the current state of the art when it comes to virtual hands. Starting with current examples for controlling and depicting hands in virtual reality (VR), we dive into the latest methods and technologies to capture hand motions. As hands can currently not be captured in every situation and as constraints stopping us from intersecting with objects are typically not available in VR, we present research on how to synthesize hand motions and simulate grasping motions. Finally, we provide an overview of our knowledge of how virtual hands are being perceived, resulting in practical tips on how to represent and handle virtual hands.
Our goals are (a) to present a broad state of the art of the current usage of hands in VR, (b) to provide more in-depth knowledge about the functioning of current hand motion tracking and hand motion synthesis methods, (c) to give insights on our perception of hand motions in VR and how to use those insights when developing new applications, and finally (d) to identify gaps in knowledge that might be investigated next. While the focus of this course is on VR, many parts also apply to augmented reality, mixed reality, and character animation in general, and some content originates from these areas.
