“Moving objects in space: exploiting proprioception in virtual-environment interaction” by Mine, Brooks Jr. and Séquin

Manipulation in immersive virtual environments is difficult partly because users must do without the haptic contact with real objects they rely on in the real world to orient themselves and their manipulanda. To compensate for this lack, we propose exploiting the one real object every user has in a virtual environment, his body. We present a unified framework for virtual-environment interaction based on proprioception, a person’s sense of the position and orientation of his body and limbs. We describe three forms of body-relative interaction:
• Direct manipulation—ways to use body sense to help control manipulation
• Physical mnemonics—ways to store/recall information relative to the body
• Gestural actions—ways to use body-relative actions to issue commands
Automatic scaling is a way to bring objects instantly within reach so that users can manipulate them using proprioceptive cues. Several novel virtual interaction techniques based upon automatic scaling and our proposed framework of proprioception allow a user to interact with a virtual world intuitively, efficiently, precisely, and lazily. We report the results of both informal user trials and formal user studies of the usability of the body-relative interaction techniques presented.

1. Bier, E.A., M.C. Stone, K. Pier, W. Buxton and T.D. DeRose. “Toolglass and magic lenses: the see-through interface.” Proceedings of SIGGRAPH 93, Anaheim, CA, ACM: 73-80.
2. Boff, K.R., L. Kaufman and J.P. Thomas, Eds. Handbook of Perception and Human Pelformance. New York, John Wiley and Sons.
3. Bowman, D. and L.F. Hodges. “An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments.” Proceedings of the 1997 Symposium on Interactive 3D Graphics, Providence, RI, ACM: 35-38.
4. Brooks, F.P., Jr. “Walkthrough-a dynamic graphics system for simulating virtual buildings.” Computer Graphics. 1986 Workshop on Interactive 3D Graphics 21(1): 3.
5. Bryson, S. and C. Levit. “The Virtual Wind Tunnel.” IEEE Computer Graphics & Applications :25-34.
6. Bukowski, R.W. and C.H. Sequin. “Object Associations: A Simple and Practical Approach to Virtual 3D Manipulation.” Proceedings of the 1995 Symposium on Interactive 3D Graphics, Monterey, CA, ACM: 131-138.
7. Butterworth, J., A. Davidson, S. Hench and T.M. Olano. “3DM: A Three Dimensional Modeler Using a Head-Mounted Display.” Computer Graphics. Proceedings 1992 Symposium on Interactive 3D Graphics 25(2): 135-138.
8. Buxton, W. and B.A. Myers. “A Study in Two-Handed Input.” Human Factors in Computing Systems (ACM SIGCHI 86 Conference Proceedings), Boston, MA, ACM, New York: 321-326.
9. Chung, J. Intuitive Navigation in the Targeting of Radiation Therapy Treatment Beams. University of North Carolina, Ph.D. Thesis
10. Conner, D.B., S.S. Snibbe, K.P. Herndon, D.C. Robbins, R.C. Zeleznik and A. vanDam. “Three-dimensional widgets.” Computer Graphics (1992 Symposium on Interactive 3D Graphics) 25(2): 183-188.
11. Cutler, L.D., B. Fr6hlich and P. Hanrahan. “Two-Handed Direct Manipulation on the Responsive Workbench.” Proceedings of the 1997 Symposium on Interactive 3D Graphics, Providence, RI, ACM: 107-114.
12. Davies, C. and J. Harrison. “Osmose: Towards Broadening the Aesthetics of Virtual Reality.” Computer Graphics 30(4):
13. Durlach, N.I. and A.S. Mavor, Eds. Virtual Reality: Scientific and Technological Challenges. Washington, D.C., National Academy Press.
14. Goble, J.C., K. Hinckley, R. Pausch, J. W. Snell and N.F. Kassell. “Two-handed spatial interface tools for neurosurgical planning.” Computer 28,(7): p. 20-6.
15. Guiard, Y. “Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model.” The Journal of Motor Behavior 19(4): 486-517.
16. Hunter, I.W., D.D. Tilemachos, S.R. Lafontaine, P.G. Charette, L.A. Jones, M.A. Sagar, G.D. Mallinson and P.J. Hunter. “A Teleoperated Microsurgical Robot and Associated Virtual Environment for Eye Surgery.” Presence 2(4): 265-280.
17. Kijima, R. and M. Hirose. “Representative Spherical Plane Method and Composition of Object Manipulation Methods.” 1996 Virtual Reality Annual International Symposium, Santa Clara, CA, IEEE: 196-202.
18. Liang, J. and M. Green. “JDCAD: a highly interactive 3D modeling system.” Computers & Graphics 18(4): 499-506.
19. Macedonia, M.R., M.J. Zyda, D.R. Pratt, P.T. Barham and S. Zeswitz. “NPSNET: A Network Software Architecture for Large Scale Virtual Environments.” Presence 3(4): 265-287.
20. Mapes, D.P. and J.M. Moshell. “A Two-Handed Interface for Object Manipulation in Virtual Environments.” Presence 4(4): 403-416.
21. Mine, M.R. “ISAAC: A Meta-CAD System for Virtual Environments.” Computer-Aided Design : to appear.
22. Mine, M.R. “Working in a Virtual World: Interaction Techniques Used in the Chapel Hill Immersive Modeling Program.” University of North Carolina, Technical Report TR96-029
23. MultiGen Inc. “SmartSceneTM.”for more information see: http ://www.multi gen.com/ smart.htm
24. Norman, D.A. The psychology of everyday things. New York, Basic Books.
25. Pausch, R., T. Burnette, D. Brockway and M.E. Weiblen. “Navigation and locomotion in virtual worlds via flight into hand-held miniatures.” Proceedings of SIGGRAPH 95, Los Angeles, CA, ACM: 399-400.
26. Pausch, R., J. Snoddy, R. Taylor, S. Watson and E. Haseltine. “Disney’s Aladdin: First Steps Toward Storytelling in Virtual Reality.” Proceedings of SIGGRAPH 96, New Orleans, LA, ACM: 193-202.
27. Pierce, J.S., A. Forsberg, M.J. Conway, S. Hong, R. Zeleznik and M.R. Mine. “Image Plane Interaction Techniques in 3D Immersive Environments.” Proceedings of the 1997 Symposium on Interactive 3D Graphics, Providence, RI, ACM: 39-44.
28. Poupyrev, I., M. Billinghurst, S. Weghorst and T. Ichikawa. “The Go-Go Interaction Technique: Non-Linear Mapping for Direct Manipulation in VR.” Proceedings of UIST 96, Seattle, WA, ACM:
29. Rothbaum, B., L. Hodges, R. Kooper, D. Opdyke, J. Williford and M. North. “Effectiveness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia.” American Journal of Psychiatry 152(4): 626-628.
30. Sachs, E., A. Roberts and D. Stoops. “3-Draw: a tool for designing 3D shapes.” IEEE Computer Graphics and Applications 11(6): 18- 26.
31. Shaw, C. and M. Green. “Two-handed polygonal surface design.” Proceedings of UIST 94, Marina del Rey, CA, p. 226, 205-12.
32. Stoakley, R., M.J. Conway and R. Pausch. “Virtual Reality on a WIM: Interactive Worlds in Miniature.” Proceedings of CHI 95, Denver, CO, ACM: 265-272.
33. Taylor, R.M., W. Robinett, V.L. Chi, F.P. Brooks Jr., W.V. Wright, S. Williams and E.J. Snyder. “The Nanomanipulator: A Virtual-Reality Interface for a Scanning Tunnel Microscope.” Proceedings of SIGGRAPH 93, Anaheim, CA, ACM:
34. Teller, S.J. and C.H. Sequin. “Visibility Preprocessing for Interactive Walkthroughs.” Computer Graphics (Proceedings of SIGGRAPH 91) 25(4): 61-69.
35. Wilson, J.R., D.J. Brown, S.V. Cobb, M.M. D’Cruz and R.M. Eastgate. “Manufacturing Operations in Virtual Environments (MOVE).” Presence 4(3): 306-317.
36. Witmer, B.G., J.H. Bailey and B. Knerr, W. “Training Dismounted Soldiers in Virtual Environments: Route Learning and Transfer.” United States Army Research Institute for the Behavioral and Social Sciences, Technical Report 1022
37. Wloka, M.M. and E. Greenfield. “The Virtual Tricorder: A Uniform Interface for Virtual Reality.” Proceedings UIST 95, Pittsburgh, PA, ACM: 39- 40.
38. Zeleznik, R.C., A.S. Forsberg and P.S. Strauss. “Two pointer input for 3D interaction.” Proceedings of the 1997 Symposium on Interactive 3D Graphics, Providence, RI, ACM: 115-120.
39. Zhai, S., P. Milgram and W. Buxton. “The Influence of Muscle Groups on Performance of Multiple Degree-of-Freedom Input.” Proceedings of CHI 96, Vancouver, BC, Canada, ACM: 308-315.