“Enhanced Eyes for Better Gaze-Awareness in Mixed Reality”

Gaze awareness and eye contact contain important nonverbal information for the smooth communication among multiple people. In a shared mixed-reality (shared MR) space, however, the gaze awareness is lost as a side-effect of wearing a head-mounted dis- play. When we try to use the shared MR space for some collaborative tasks, the loss of gaze awareness becomes a critical problem. We have proposed a scheme to restore gaze awareness and eye con- tact by diminishing the HMDs using MR technologies [Takemura and Ohta 2002]. This is realized by synthesizing the user’s CG face with natural eye-expressions in real time and by overlaying it on the real user’s face wearing an HMD, which is to be diminished. Figure 1 illustrates a snapshot of a partner’s view of the system. The goal of this work was to reproduce the user’s CG face with complete fidelity and to overlay the CG face to the real face as seamlessly as possible.

Takemura, M., and Ohta, Y. Oct, 2002. Diminishing head-mounted display for shared mixedreality. In IEEE and ACM International Symposium on Mixed and Augmented Reality (IS-MAR 2002), 149–156.