“The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues”
Conference:
Type(s):
Title:
- The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues
Session/Category Title: VR, Display, and Interaction
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Over the last few years, virtual reality (VR) has re-emerged as a technology that is now feasible at low cost via inexpensive cellphone components. In particular, advances of high-resolution micro displays, low-latency orientation trackers, and modern GPUs facilitate immersive experiences at low cost. One of the remaining challenges to further improve visual comfort in VR experiences is the vergence-accommodation conflict inherent to all stereoscopic displays. Accurate reproduction of all depth cues is crucial for visual comfort. By combining well-known stereoscopic display principles with emerging factored light field technology, we present the first wearable VR display supporting high image resolution as well as focus cues. A light field is presented to each eye, which provides more natural viewing experiences than conventional near-eye displays. Since the eye box is just slightly larger than the pupil size, rank-1 light field factorizations are sufficient to produce correct or nearly-correct focus cues; no time-multiplexed image display or gaze tracking is required. We analyze lens distortions in 4D light field space and correct them using the afforded high-dimensional image formation. We also demonstrate significant improvements in resolution and retinal blur quality over related near-eye displays. Finally, we analyze diffraction limits of these types of displays.
References:
1. Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) 23, 804–813. Google ScholarDigital Library
2. Benton, S., and Bove, V. 2006. Holographic Imaging. John Wiley and Sons. Google ScholarDigital Library
3. Brown, D. 2000. Decentering distortion of lenses. Photogrammetric Engineering 32, 3, 444462.Google Scholar
4. Cakmakci, O., and Rolland, J. 2006. Head-worn displays: a review. Journal of Display Technology 2, 3, 199–216.Google ScholarCross Ref
5. Clarberg, P., Toth, R., Hasselgren, J., Nilsson, J., and Akenine-Möller, T. 2014. Amfs: Adaptive multi-frequency shading for future graphics processors. ACM Trans. Graph. 33, 4 (July), 141:1–141:12. Google ScholarDigital Library
6. Cutting, J. E., and Vishton, P. M. 1995. Perceiving layout and knowing distances: The interaction, relative potency, and contextual use of different information about depth. W. Epstein and S. Rogers (Eds.), Perception of space and motion, 69–117.Google Scholar
7. Favalora, G. E. 2005. Volumetric 3D displays and application infrastructure. IEEE Computer 38, 37–44. Google ScholarDigital Library
8. Hale, K., and Stanney, K. 2014. Handbook of Virtual Environments. CRC Press. Google ScholarDigital Library
9. Heide, F., Lanman, D., Reddy, D., Kautz, J., Pulli, K., and Luebke, D. 2014. Cascaded displays: Spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. (SIGGRAPH) 33, 4, 60:1–60:11. Google ScholarDigital Library
10. Held, R., Cooper, E., and Banks, M. 2012. Blur and Disparity Are Complementary Cues to Depth. Current Biology 22, R163–R165.Google ScholarCross Ref
11. Hirsch, M., Wetzstein, G., and Raskar, R. 2014. A Compressive Light Field Projection System. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 4, 1–12. Google ScholarDigital Library
12. Hoffman, D. M., and Banks, M. S. 2010. Focus information is used to interpret binocular images. Journal of Vision 10, 5, 13.Google ScholarCross Ref
13. Hua, H., and Javidi, B. 2014. A 3d integral imaging optical see-through head-mounted display. OSA Opt. Exp. 22, 11, 13484–13491.Google ScholarCross Ref
14. Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field displays. ACM Trans. Graph. (SIGGRAPH) 33, 4, 59:1–59:12. Google ScholarDigital Library
15. Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. (SIGGRAPH) 26, 40:1–40:10. Google ScholarDigital Library
16. Kelly, D. H. 1979. Motion and vision. ii. stabilized spatiotemporal threshold surface. Journal of the Optical Society of America 69, 1340–1349.Google ScholarCross Ref
17. Kress, B., and Starner, T. 2013. A review of head-mounted displays (hmd) technologies and applications for consumer electronics. In Proc. SPIE, vol. 8720, 87200A–87200A–13.Google Scholar
18. Lanman, D., and Luebke, D. 2013. Near-eye light field displays. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6, 220:1–220:10. Google ScholarDigital Library
19. Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. (SIGGRAPH Asia) 29, 163:1–163:10. Google ScholarDigital Library
20. Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In ACM SGGRAPH, 31–42. Google ScholarDigital Library
21. Liu, S., Cheng, D., and Hua, H. 2008. An optical see-through head mounted display with addressable focal planes. In Proc. ISMAR, 33–42. Google ScholarDigital Library
22. Love, G. D., Hoffman, D. M., Hands, P. J., Gao, J., Kirby, A. K., and Banks, M. S. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. OSA Optics Express 17, 18, 15716–15725.Google ScholarCross Ref
23. MacKenzie, K. J., Hoffman, D. M., and Watt, S. J. 2010. Accommodation to multiplefocalplane displays: Implications for improving stereoscopic displays and for accommodation control. Journal of Vision 10, 8.Google ScholarCross Ref
24. Maimone, A., and Fuchs, H. 2013. Computational augmented reality eyeglasses. In Proc. ISMAR, 29–38.Google Scholar
25. Maimone, A., Wetzstein, G., Hirsch, M., Lanman, D., Raskar, R., and Fuchs, H. 2013. Focus 3d: Compressive accommodation display. ACM Trans. Graph. 32, 5, 153:1–153:13. Google ScholarDigital Library
26. Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., and Fuchs, H. 2014. Pinlight displays: Wide field of view augmented reality eyeglasses using defocused point light sources. ACM Trans. Graph. (SIGGRAPH) 33, 4. Google ScholarDigital Library
27. Marshall, J. A., Marshall, J. A., Ariely, D., Burbeck, C. A., Aricly, T. D., Rolland, J. P., and Martin, K. E. 1996. Occlusion edge blur: A cue to relative visual depth. OSA JOSAA 13, 681–688.Google ScholarCross Ref
28. Marwah, K., Wetzstein, G., Bando, Y., and Raskar, R. 2013. Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections. ACM Trans. Graph. (Proc. SIGGRAPH). Google ScholarDigital Library
29. Rothbaum, B., Hodges, L., Ready, D., Graap, K., and Alarcon, R. 2001. Virtual reality exposure therapy for vietnam veterans with posttraumatic stress disorder. Ann Surg 62, 8, 617–22.Google Scholar
30. Ruch, T., and Fulton, J. F. 1960. Medical physiology and biophysics. W. B. Saunders Company.Google Scholar
31. Rushton, S. K., and Riddell, P. M. 1999. Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence. Appl Ergonomics 30, 69–78.Google ScholarCross Ref
32. Ryana, L., MacKenziea, K., and Watta, S. 2012. Multiple-focal-planes 3d displays: A practical solution to the vergence-accommodation conflict? In Proc. 3D Imaging (IC3D), 1–6.Google Scholar
33. Schowengerdt, B. T., and Seibel, E. J. 2006. True 3-d scanned voxel displays using single or multiple light sources. Journal of the Society for Information Display 14, 2, 135–143.Google ScholarCross Ref
34. Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., Whitehead, L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High dynamic range display systems. ACM Trans. Graph. (SIGGRAPH) 23, 3, 760–768. Google ScholarDigital Library
35. Sullivan, A. 2003. A solid-state multi-planar volumetric display. In SID Digest, vol. 32, 207–211.Google Scholar
36. Sutherland, I. 1968. A head-mounted three dimensional display. In Proc. AFIPS Fall Joint Computer Conference. Google ScholarDigital Library
37. Takaki, Y., Tanaka, K., and Nakamura, J. 2011. Super multi-view display with a lower resolution flat-panel display. Opt. Express 19, 5, 4129–4139.Google ScholarCross Ref
38. Takaki, Y. 2006. High-Density Directional Display for Generating Natural Three-Dimensional Images. Proc. IEEE 94, 3.Google ScholarCross Ref
39. Watt, S., Akeley, K., Ernst, M., and Banks, M. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10, 834–862.Google ScholarCross Ref
40. Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. (SIGGRAPH) 30, 1–11. Google ScholarDigital Library
41. Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1–11. Google ScholarDigital Library
42. Wheatstone, C. 1838. Contributions to the physiology of vision. part the first. on some remarkable, and hitherto unobserved, phenomena of binocular vision. Philosophical Transactions of the Royal Society of London 128, 371–394.Google ScholarCross Ref
43. Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE Trans. PAMI 22, 11, 1330–1334. Google ScholarDigital Library
44. Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3D displays. In EGSR. Google ScholarDigital Library
