“Commonsense Knowledge-Driven Joint Reasoning Approach for Object Retrieval in Virtual Reality” by Jiang, Weng, Dongye, Luo and Zhang
Conference:
Type(s):
Title:
- Commonsense Knowledge-Driven Joint Reasoning Approach for Object Retrieval in Virtual Reality
Session/Category Title: Technical Papers Fast-Forward
Presenter(s)/Author(s):
Abstract:
Out-of-reach object retrieval is an important task in virtual reality (VR). Gesture-based approach, one of the most commonly used approaches, enables bare-hand, eyes-free, and direct retrieval by using assigned gestures. However, it is difficult to retrieve an object from plenty of objects accurately by using gestures due to the one-to-one mapping metaphor, the limitation of finger poses, and memory burdens. Previous work has focused on gesture design, ignoring the context. In fact, there is a consensus that objects and contexts are related. This indicates the object expected to be retrieved is related to the context including the scene and the objects users interact with. Therefore, we proposed a commonsense knowledge-driven joint reasoning approach for object retrieval, where the human grasping gesture and the context are modeled by an And-Or graph (AOG). This approach enables users to accurately retrieve objects from plenty of candidate objects by using natural grasping gestures according to their experience of grasping physical objects. The experimental results show that our proposed model improves retrieval accuracy. Finally, we propose an object retrieval system based on the proposed approach and two user studies demonstrate that the system enables efficient object retrieval in virtual environments.
