“Just Follow Me: A VR-Based Motion Training System” by Yang

Training has been considered one of the most natural application areas of virtual reality (VR). This project demonstrates VR’s utility for learning limb-motion profiles required in sports, dance, and the arts (for example, a golf swing, martial arts, calligraphy, etc.) The central concept, called Just Follow Me (JFM), is based on an intuitive interaction method called the “ghost” metaphor (Figure 1). Through the Ghost metaphor, the motion of the trainer is visualized in real time as a ghost (initially superimposed on the trainee) moving out of one’s body. The trainee, who sees the motion from different viewpoints, is to “follow” the ghostly master as close as (or as fast as) possible.

The training process can be facilitated by showing other guidance cues (for example, the master’s trail, annotations, etc.) and performance feedback (for example, indications of how well the trainee is following), and by adjusting the learning requirements (for example, relaxation of accuracy goals, restricting the motion’s degrees of freedom). Evaluation results showed that JFM produced training and transfer effects as good as, and in certain situations better than, the real one. Thus when the system is reinforced and augmented with presence cues, more robust tracking and lighter head-mounted displays, and rich informative graphics and images, VR-based training methods will be attractive alternatives to the traditional “trainer-in-residence” or video-based motor-skill learning method.

The JFM system can be configured in many different ways. For the desktop/arcade version (Figure 2), it is difficult to effectively present the first-person viewpoint. Consequently, the ghost has been modified to what are called “sliding” ghosts; instead of showing continuous movement, discrete freezeframes of next imminent postures from fixed viewpoints (front, back, side) flow toward the dancing character to guide the motion (Figure 3). Another possibility is the “first-person viewpoint” version with the head-mounted display and head tracking, in which the original concept of the Ghost is used (Figure 4). The system uses four cameras mounted on the ceiling to track five highly reflective markers (worn on users’ wrists, ankles, and belly). The tracked motion is compared to the reference motion for both online and offline evaluation. As trainees try to follow and imitate the character’s dance on the screen, they get a feel for how well they are following the evaluations and corresponding special effects at the key posture frames, and they receive a final score at the end.