“Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies” by State, Livingston, Garrett, Hirota, Whitton, et al. …
Conference:
Type(s):
Title:
- Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies
Presenter(s)/Author(s):
Abstract:
We present a real-time stereoscopic video-see-through augmented reality (AR) system applied to the medical procedure known as ultrasound-guided needle biopsy of the breast. The AR system was used by a physician during procedures on breast models and during non-invasive examinations of human subjects. The system merges rendered live ultrasound data and geometric elements with stereo images of the patient acquired through head-mounted video cameras and presents these merged images to the physician in a head-mounted display. The physician sees a volume visualization of the ultrasound data directly under the ultrasound probe, properly registered within the patient and with the biopsy needle. Using this system, a physician successfully guided a needle into an artificial tumor within a training phantom of a human breast. We discuss the construction of the AR system and the issues and decisions which led to the system architecture and the design of the video see-through head-mounted display. We designed methods to properly resolve occlusion of the real and synthetic image elements. We developed techniques for real-time volume visualization of time- and position-varying ultrasound data. We devised a hybrid tracking system which achieves improved registration of synthetic and real imagery and we improved on previous techniques for calibration of a magnetic tracker.
References:
- AZUMA, R. A Survey of Augmented Reality. SIGGRAPH 1995 Course Notes #9 (Developing Advanced Virtual Reality Applications), pp. 20-1 through 20-38.
- AZUMA, R., BISHOP, G. Improving Static and Dynamic Registration in an Optical See-through HMD. Proceedings of SIGGRAPH 94 (Orlando, Florida, July 24-29, 1994). In Computer Graphics Proceedings, Annual Conference Series, 1994, ACM SIGGRAPH, pp. 197-204.
- BAJURA, M., FUCHS, H., OHBUCHI, R. Merging Virtual Objects with the Real World. Proceedings of SIGGRAPH ’92 (Chicago, Illinois, July 26- 31, 1992). In Computer Graphics 26, 2 (July 1992), ACM SIGGRAPH, New York, 1992, pp. 203-210.
- BAJURA, M., NEUMANN, U. Dynamic Registration Correction in Video- Based Augmented Reality Systems. IEEE Computer Graphics and Applications (September, 1995), pp. 52-60.
- BARBER, C.B., DOBKIN, D.P., HUHDANPAA, H. The Quickhull Algorithm for Convex Hull. Geometry Center Technical Report GCG53, 1993.
- BRYSON, S. Measurement and Calibration of Static Distortion of Position Data from 3D Trackers. SPIE Stereoscopic Displays and Applications III, 1992, pp. 244-255.
- CABRAL, B., CAM, N., FORAN, J. Accelerated Volume Rendering and Tomographic Reconstruction Using Texture Mapping Hardware. Proceedings of the 1994 Symposium on Volume Visualization (Washington, D. C., October 1994), pp. 91-97, 131, back cover.
- COLUCCI, D., CHI, V. Computer Glasses: A Compact, Lightweight, and Cost-effective Display for Monocular and Tiled Wide Field-of-View Systems. Proceedings ofSPIE, Vol. 2537, pp. 61-70 (1995).
- CULLIP, T.J., NEUMANN, U. Accelerating Volume Reconstruction With 3D Texture Hardware. University of North Carolina Department of Computer Science Technical Report TR93-027.
- FUCHS, H., ABRAM, G., GRANT, E. Near Real-time Shaded Display of Rigid Objects. Proceedings of SIGGRAPH ’83 (July 1983). In Computer Graphics; 17, 3 (July 1983), ACM SIGGRAPH, New York, 1983, pp. 65- 72.
- FUCHS, H., KEDEM, Z., NAYLOR, B. On Visible Surface Generation by a Priori Tree Structures. Proceedings of SIGGRAPH ’80 (July 1980). In Computer Graphics; 14, 3 (July 1980), ACM SIGGRAPH, New York, 1980, pp. 124-133.
- FURNESS, T. The super cockpit and its human factors challenges. Proceedings of the Human Factors Society, 30, 1968, pp. 48-52.
- GARRETT, W.F., FUCHS, H., STATE, A., WHITTON, M.C. Real-Time Incremental Visualization of Dynamic Ultrasound Volumes Using Parallel BSP Trees. University of North Carolina Department of Computer Science Technical Report TR96-018.
- GHAZlSAEDY, M., ADAMCZYK, D., SANDIN, D., KENYON, R., DEFANTI, T. Ultrasonic Calibration of a Magnetic Tracker in a Virtual Reality Space. Proceedings of the Virtual Reality Annual International Symposium ’95, IEEE Computer Society, Los Alamitos, CA, 1995, pp. 179-188.
- LALOUCHE, R.C., BICKMORE, D., TESSLER, F., MANKOVICH, H.K., KANGARALOO, H. Three-dimensional reconstruction of ultrasound images. SPIE’89, Medical Imaging, SPIE, 1989, pp. 59-66.
- LIVINGSTON, M.A., STATE, A. Magnetic Tracker Calibration for Improved Registration in Augmented Reality Systems. University of North Carolina Department of Computer Science Technical Report TR95- 037.
- LORENSEN, W., CLINE, H., NAFIS, C., KIKINIS, R., ALTOBELLI, D., GLEASON, L. Enhancing reality in the operating room. Proceedings of Visualization 1993 (Los Alamitos, CA, October 1993), pp. 410-415.
- MELLOR, J.P. Enhanced Reality Visualization in a Surgical Environment. MS Thesis, MIT Department of Electrical Engineering, 1995.
- NAYLOR, B., AMANATIDES, J., THIBAULT, W. Merging BSP Tree Yields Polyhedral Set Operations. Proceedings of SIGGRAPH ’90 (Dallas, Texas, August 1990). In Computer Graphics; 24, 4 (August 1990), pp. 115-124.
- NELSON, T., ELVINS, T. Visualization of 3D Ultrasound Data. IEEE Computer Graphics and Applications, (November, 1993), pp. 50-57.
- OHBUCHI, R., CHEN, D.T., FUCHS, H.Incremental Volume Reconstruction and Rendering for 3D Ultrasound Imaging. University of North Carolina Department of Computer Science Technical Report TR92- 037.
- ROBINETT, W., ROLLAND, J.P. A Computational Model for the Stereoscopic Optics of a Head-Mounted Display. Presence, Vol. 1, No. 1 (Winter 1992), pp. 45-62.
- ROLLAND, J.P., BIOCCA, F., BARLOW, T., KANCHERLA, A.R. Quantification of Adaptation to Virtual-Eye Location in See-Thru Head- Mounted Displays. Proceedings of the Virtual Reality: Annual International Symposium ’95, IEEE Computer Society, Los Alamitos, CA, 1995, pp. 56-66.
- ROSE, E., BREEN, D., AHLERS, K., CRAMPTON, C., TUCERYAN, M., WHITAKER, R., GREER, D. Annotating Real-World Objects Using Augmented Reality. Proceedings of Computer Graphics International 1995 (Leeds, UK, June, 25-30, 1995), pp. 357-370.
- SIMS, D. New Realities in Aircraft Design and Manufacture. IEEE Computer Graphics and Applications, (March, 1994), p. 91.
- STATE, A., CHEN, D.T., TECTOR, C., BRANDT, A., CHEN, H., OHBUCHI, R., BAJURA, M., FUCHS, H. Case Study: Observing a Volume-Rendered Fetus within a Pregnant Patient. Proceedings of IEEE Visualization ’94, October 1994, pp. 364-368, CP-41.
- STATE, A., HIROTA, G., CHEN, D.T., GARRETT, W.F., LIVINGSTON, M.A. Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking. Proceedings of SIGGRAPH ’96 (New Orleans, LA, August 4-9, 1996). In Computer Graphics Proceedings, Annual Conference Series, 1996, ACM SIGGRAPH.
- STATE, A., MCALLISTER, J., NEUMANN, U., CHEN, H., CULLIP, T.J., CHEN, D.T., FUCHS, H. Interactive Volume Visualization on a Heterogeneous Message-Passing Multicomputer. Proceedings of the 1995 Symposium on Interactive 3D Graphics (Monterey, CA, April 1995), pp. 69-74, 208, front cover.
- SUTHERLAND, I. A head-mounted three dimensional display. Proceedings of the Fall Joint Computer Conference, 1968.
- WLOKA, M.M., ANDERSON, B.G. Resolving Occlusion in Augmented Reality. Proceedings of the 1995 Symposium on Interactive 3D Graphics (Monterey, CA, April 1995), pp. 5-12.