“8D display: a relightable glasses-free 3D display” by Hirsch, Izadi, Holtzman and Raskar

Imagine a display that behaves like a window. Glancing through it, viewers perceive a virtual 3D scene with correct parallax, without the need to wear glasses or track the user. Light that passes through the display correctly illuminates the virtual scene. While researchers have considered such displays, or prototyped subsets of these capabilities, we contribute a new, interactive, relightable, glasses-free 3D display. By simultaneously capturing a 4D light field, and displaying a 4D light field, we are able to realistically modulate the incident light on rendered content. We present our optical design, and GPU pipeline. Beyond mimicking the physical appearance of objects under natural lighting, an 8D display can create arbitrary directional illumination patterns and record their interaction with physical objects. Our hardware points the way towards novel 3D interfaces, in which users interact with digital content using light widgets, physical objects, and gesture.

1. Fuchs, M., Raskar, R., Seidel, H.-P., and Lensch, H. P. A. 2008. Towards passive 6d reflectance field displays. ACM Trans. Graph. 27, 3 (Aug.), 58:1–58:8.
2. Hirsch, M., Lanman, D., Holtzman, H., and Raskar, R. 2009. Bidi screen: a thin, depth-sensing lcd for 3d interaction using light fields. ACM Trans. Graph. 28, 5 (Dec.), 159:1–159:9.
3. Hullin, M. B., Lensch, H. P. A., Raskar, R., Seidel, H.-P., and Ihrke, I. 2011. Dynamic display of brdfs. Computer Graphics Forum 30, 2, 475–483.
4. Nayar, S. K., Belhumeur, P. N., and Boult, T. E. 2004. Lighting sensitive display. ACM Trans. Graph. 23, 4 (Oct.), 963–979.