“Rendering for an interactive 360° light field display” by Jones, Yamada, Bolas and Debevec

We describe a set of rendering techniques for an autostereoscopic light field display able to present interactive 3D graphics to multiple simultaneous viewers 360 degrees around the display. The display consists of a high-speed video projector, a spinning mirror covered by a holographic diffuser, and FPGA circuitry to decode specially rendered DVI video signals. The display uses a standard programmable graphics card to render over 5,000 images per second of interactive 3D graphics, projecting 360-degree views with 1.25 degree separation up to 20 updates per second. We describe the system’s projection geometry and its calibration process, and we present a multiple-center-of-projection rendering technique for creating perspective-correct images from arbitrary viewpoints around the display. Our projection technique allows correct vertical perspective and parallax to be rendered for any height and distance when these parameters are known, and we demonstrate this effect with interactive raster graphics using a tracking system to measure the viewer’s height and distance. We further apply our projection technique to the display of photographed light fields with accurate horizontal and vertical parallax. We conclude with a discussion of the display’s visual accommodation performance and discuss techniques for displaying color imagery.

1. Agocs, T., Balogh, T., Forgacs, T., Bettio, F., Gobbetti, E., Zanetti, G., and Bouvier, E. 2006. A large scale interactive holographic display. In VR ’06: Proceedings of the IEEE Virtual Reality Conference (VR 2006), IEEE Computer Society, Washington, DC, USA, 57. Google ScholarDigital Library
2. Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Transactions on Graphics 23, 3 (Aug.), 804–813. Google ScholarDigital Library
3. Balogh, T., Dobranyi, Z., Forgacs, T., Molnar, A., Szloboda, L., Gobbetti, E., Marton, F., Bettio, F., Pintore, G., Zanetti, G., Bouvier, E., and Klein, R. 2006. An interactive multi-user holographic environment. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Emerging technologies, ACM Press, New York, NY, USA, 18. Google ScholarDigital Library
4. Batchko, R. G. 1994. Three-hundred-sixty degree electro-holographic stereogram and volumetric display system. In Proc. SPIE, vol. 2176, 30–41.Google ScholarCross Ref
5. Chai, J.-X., Tong, X., Chan, S.-C., and Shum, H.-Y. 2000. Plenoptic sampling. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 307–318. Google ScholarDigital Library
6. Cossairt, O. S., and Napoli, J., 2005. Radial multi-view 3-dimensional displays. United States Patent Application 2005/0180007, Aug.Google Scholar
7. Cossairt, O., Travis, A. R., Moller, C., and Benton, S. A. 2004. Novel view sequential display based on dmd technology. In Proc. SPIE, Stereoscopic Displays and Virtual Reality Systems XI, A. J. Woods, J. O. Merritt, S. A. Benton, and M. T. Bolas, Eds., vol. 5291, 273–278.Google Scholar
8. Cossairt, O. S., Napoli, J., Hill, S. L., Dorval, R. K., and Favalora, G. E. 2007. Occlusion-capable multiview volumetric three-dimensional display. Applied Optics 46, 8 (Mar), 1244–1250.Google ScholarCross Ref
9. Dodgson, N. A. 2005. Autostereoscopic 3D displays. Computer 38, 8, 31–36. Google ScholarDigital Library
10. Dorsey, J. O., Sillion, F. X., and Greenberg, D. P. 1991. Design and simulation of opera lighting and projection effects. In SIGGRAPH ’91: Proceedings of the 18th annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 41–50. Google ScholarDigital Library
11. Einarsson, P., Chabert, C.-F., Jones, A., Ma, W.-C., Lamond, B., Hawkins, T., Bolas, M., Sylwan, S., and Debevec, P. 2006. Relighting human locomotion with flowed reflectance fields. In Rendering Techniques 2006: 17th Eurographics Symposium on Rendering, 183–194. Google ScholarCross Ref
12. Endo, T., Kajiki, Y., Honda, T., and Sato, M. 2000. Cylindrical 3D video display observable from all directions. In 8th Pacific Conference on Computer Graphics and Applications, 300–306. Google ScholarDigital Library
13. Favalora, G. E. 2005. Volumetric 3D displays and application infrastructure. Computer 38, 8, 37–44. Google ScholarDigital Library
14. Forsyth, D. A., and Ponce, J. 2002. Computer Vision: A Modern Approach. Prentice Hall, ch. 3, 45. Google ScholarDigital Library
15. Freundenberg, B., Masuch, M., and Strothotte, T. 2004. Game Programming Gems 4. Thomson/Delmar Learning, ch. Real-Time Halftoning: Fast and Simple Stylized Shading, 443–449.Google Scholar
16. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 43–54. Google ScholarDigital Library
17. Halle, M. W., Benton, S. A., Klug, M. A., and Underkoffler, J. S. 1991. The ultragram: A generalized holographic stereogram. In Proceedings of the SPIE Practical Holography V.Google Scholar
18. Han, J. Y., and Perlin, K. 2003. Measuring bidirectional texture reflectance with a kaleidoscope. ACM Transactions on Graphics 22, 3 (July), 741–748. Google ScholarDigital Library
19. Hou, X., Wei, L.-Y., Shum, H.-Y., and Guo, B. 2006. Realtime multi-perspective rendering on graphics hardware. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 93–102. Google ScholarCross Ref
20. Isaksen, A., McMillan, L., and Gortler, S. J. 2000. Dynamically reparameterized light fields. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 297–306. Google ScholarDigital Library
21. Jones, A., Debevec, P., Bolas, M., and McDowall, I. 2006. Concave surround optics for rapid multiview imaging. In Proceedings of the 25th Army Science Conference.Google ScholarDigital Library
22. Jones, A., Gardner, A., Bolas, M., McDowall, I., and Debevec, P. 2006. Performance geometry capture for spatially varying relighting. In 3rd European Conference on Visual Media Production (CVMP 2006), 127–133.Google Scholar
23. Kajiya, J. T., and Kay, T. L. 1989. Rendering fur with three dimensional textures. In SIGGRAPH ’89: Proceedings of the 16th annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 271–280. Google ScholarDigital Library
24. Levoy, M., and Hanrahan, P. M. 1996. Light field rendering. In Proceedings of ACM SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 31–42. Google ScholarDigital Library
25. Lippman, G. 1908. Epreuves reversibles donnant la sensation du relief. Journal of Physics 7, 4 (Nov), 821–835.Google Scholar
26. Maeda, H., Hirose, K., Yamashita, J., Hirota, K., and Hirose, M. 2003. All-around display for video avatar in real world. In ISMAR ’03: Proceedings of the The 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, IEEE Computer Society, Washington, DC, USA, 288. Google ScholarDigital Library
27. Matusik, W., and Pfister, H. 2004. 3D tv: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Transactions on Graphics 23, 3 (Aug.), 814–824. Google ScholarDigital Library
28. McDowall, I., and Bolas, M. 2005. Display, sensing, and control applications for digital micromirror displays. In IEEE VR 2005 – Emerging Display Technologies, 35–36.Google Scholar
29. Miller, J. R. 1987. Geometric approaches to nonplanar quadric surface intersection curves. ACM Transactions on Graphics 6, 4 (Oct.), 274–307. Google ScholarDigital Library
30. Ostromoukhov, V. 2001. A simple and efficient error-diffusion algorithm. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, 567–572. Google ScholarDigital Library
31. Otsuka, R., Hoshino, T., and Horry, Y. 2006. Transpost: A novel approach to the display and transmission of 360 degrees-viewable 3D solid images. IEEE Transactions on Visualization and Computer Graphics 12, 2, 178–185. Google ScholarDigital Library
32. Poulin, P., and Fournier, A. 1990. A model for anisotropic reflection. 273–282.Google Scholar
33. Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. 1998. The office of the future: A unified approach to image-based modeling and spatially immersive displays. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, 179–188. Google ScholarDigital Library
34. Sullivan, A. 2003. A solid-state multi-planar volumetric display. SID Symposium Digest of. Technical Papers 32, 1 (May), 1531–1533.Google ScholarCross Ref
35. Tanaka, K., and Aoki, S. 2006. A method for the real-time construction of a full parallax light field. In Stereoscopic Displays and Virtual Reality Systems XIII. Proceedings of the SPIE, Volume 6055, pp. 397–407 (2006)., A. J. Woods, N. A. Dodgson, J. O. Merritt, M. T. Bolas, and I. E. McDowall, Eds., 397–407.Google ScholarCross Ref
36. Travis, A. R. L. 1997. The display of three-dimensional video images. Proceedings of the IEEE 85, 11 (Nov), 1817–1832.Google ScholarCross Ref
37. Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. ACM Transactions on Graphics 24, 3 (Aug), 765–776. Google ScholarDigital Library
38. Yang, J. C., Everett, M., Buehler, C., and McMILLAN, L. 2002. A real-time distributed light field camera. In Rendering Techniques 2002: 13th Eurographics Workshop on Rendering, 77–86. Google ScholarDigital Library
39. Yendo, T., Kawakami, N., and Tachi, S. 2005. Seelinder: the cylindrical lightfield display. In SIGGRAPH ’05: ACM SIGGRAPH 2005 Emerging technologies, ACM Press, New York, NY, USA, 16. Google ScholarDigital Library
40. Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3D displays. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 73–82. Google ScholarCross Ref