“Gaze-orchestrated dynamic windows” by Bolt

Consider a large-format display before the user, bearing a multiplicity of “windows,” like little movies, the majority dynamic and in color. There are upwards of 20 windows, say, more than a person can ordinarily absorb at once. Some of the windows come and go, reflecting their nature as direct TV linkages into real-time, real-world events. Others are non-real-time, some dynamic, others static but capable of jumping into motion. Such an ensemble of information inputs reflects the managerial world of the top-level executive of the not too distant electronic future: a world of brevity, fragmentation, variety, above all one of an overwhelming onslaught of events. The multiplicity and simultaneity of such a display situation ordinarily would make coping with it untenable. The intent of the reported research is to introduce order and control, through the creation of a dynamic, gaze-interactive interface. Making the behavior and reactivity of the “windows” contingent upon measured eyemovements – the point-of-regard of the observer – aims both to help the observer to cope with the onslaught of events on the one hand, yet enable on the other hand continuing close contact with that everchanging ensemble. A simulation of such a world is described and demonstrated in the composite medium of computer, videodisc, and video special effects. Eye-tracking technology, integrated with speech and manual inputs, controls the display’s visual dynamics, and orchestrates its sound accompaniments. All elements are combined to form a testbed for the conception generally, and to explore the associated human factors and stagecraft.

1. FOULDS, R. A. Studies in the use of visual line of gaze as a selection technique in nonvocal communication. Proposal to National Science Foundation by Tufts-New England Medical Center, Department of Rehabilitation Medicine, Richard A. Foulds, Principal Investigator, July, 1980.
2. KAHNEMAN D. Attention and effort. Englewood Cliffs, N.J.: Prentice Hall, 1973.
3. KEEN, P. G. W. & SCOTT-MORTON, M.S. Decision support systems: an organizational perspective. Reading, MA: Addison-Wesley, 1978.
4. MARKS, L. E. Multimodel perception. In E. C. Carterette & M. P. Friedman, Eds., Handbook of Perception, Vol. VII (Perceptual Coding), New York: Academic Press, 1978.
5. MONTY, R. A. & SENDERS J. W. (Eds.) Eye movements and psychological processes. Hillsdale, N.J.: Erlbaum Associates, 1976.
6. NEISSER, U. & BECKLEN, R. Selective looking: attending to visually specified events, Cognitive Psychology, 7(4), 480-494, 1975.
7. RABB, F. H., BLOOD, E. B., STEINER, T. O. & JONES, H.R. Magnetic position and orientation tracking system. IEEE Transaction on Aerospace and Electronic Systems, Vol. AES-15, No. 5, September 1979, 709-718.
8. RINARD, G. A. & RUGG, D. E. Current state of development and testing of an ocular control device. 4th Conference of Systems and Devices for the Disabled, Seattle, WA, June 1977.
9. RINARD, G. A. & RUGG, D. E. An ocular control device for use by the severely handicapped. 1976 Conference on Systems and Devices for the Disabled, Boston, MA, 1976.
10. SENDERS, J. W. FISHER, D. G. & MONTY, R. A. (Eds.) Eye movements and higher psychological processes. Hillsdale, N.J.: Erlbaum Associates, 1978.