“Holographic near-eye displays for virtual and augmented reality” by Maimone, Georgiou and Kollin
Conference:
Type:
Title:
- Holographic near-eye displays for virtual and augmented reality
Session/Category Title: Time to Focus
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present novel designs for virtual and augmented reality near-eye displays based on phase-only holographic projection. Our approach is built on the principles of Fresnel holography and double phase amplitude encoding with additional hardware, phase correction factors, and spatial light modulator encodings to achieve full color, high contrast and low noise holograms with high resolution and true per-pixel focal control. We provide a GPU-accelerated implementation of all holographic computation that integrates with the standard graphics pipeline and enables real-time (≥90 Hz) calculation directly or through eye tracked approximations. A unified focus, aberration correction, and vision correction model, along with a user calibration process, accounts for any optical defects between the light source and retina. We use this optical correction ability not only to fix minor aberrations but to enable truly compact, eyeglasses-like displays with wide fields of view (80°) that would be inaccessible through conventional means. All functionality is evaluated across a series of hardware prototypes; we discuss remaining challenges to incorporate all features into a single device.
References:
1. Kurt Akeley, Simon J. Watt, Ahna Reza Girshick, and Martin S. Banks. 2004. A Stereo Display Prototype with Multiple Focal Distances. In ACM SIGGRAPH 2004 Papers (SIGGRAPH ’04). ACM, New York, NY, USA, 804–813. Google ScholarDigital Library
2. A. J. Cable, E. Buckley, P. Mash, N. A. Lawrence, T. D. Wilkinson, and W. A. Crossland. 2004. 53.1: Real-time Binary Hologram Generation for High-quality Video Projection Applications. SID Symposium Digest of Technical Papers 35, 1 (2004), 1431–1433.Google Scholar
3. J.-S. Chen and D. P. Chu. 2015. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. Opt. Express 23, 14 (Jul 2015). Google ScholarCross Ref
4. Reiner Eschbach. 1991. Comparison of error diffusion methods for computer-generated holograms. Appl. Opt. 30, 26 (Sep 1991). Google ScholarCross Ref
5. Jonathan P. Freeman, Timothy D. Wilkinson, and Paul Wisely. 2010. Visor projected HMD for fast jets using a holographic video projector. Proc. SPIE 7690 (2010), 76901H–76901H-12. Google ScholarCross Ref
6. Qiankun Gao, Juan Liu, Jian Han, and Xin Li. 2016. Monocular 3D see-through head-mounted display via complex amplitude modulation. Opt. Express 24, 15 (Jul 2016), 17372–17383. Google ScholarCross Ref
7. A Georgiou, J Christmas, N Collings, J Moore, and W A Crossland. 2008. Aspects of hologram calculation for video frames. Journal of Optics A: Pure and Applied Optics 10, 3 (2008), 035302.Google ScholarCross Ref
8. R. W. Gerchberg and W. Owen Saxton. 1972. A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik 35 (1972), 237–246.Google Scholar
9. Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. 2012. Foveated 3D Graphics. ACM Trans. Graph. 31, 6, Article 164 (Nov. 2012), 10 pages.Google ScholarDigital Library
10. R. Häussler, S. Reichelt, N. Leister, E. Zschau, R. Missbach, and A. Schwerdtner. 2009. Large real-time holographic displays: from prototypes to a consumer product. Proc. SPIE 7237, 72370S–72370S-9. Google ScholarCross Ref
11. Jisoo Hong, Youngmin Kim, Sunghee Hong, Choonsung Shin, and Hoonjong Kang. 2016. Gaze contingent hologram synthesis for holographic head-mounted display. Proc. SPIE 9771 (2016), 97710K–97710K-6. Google ScholarCross Ref
12. C. K. Hsueh and A. A. Sawchuk. 1978. Computer-generated double-phase holograms. Appl. Opt. 17, 24 (Dec 1978), 3874–3883. Google ScholarCross Ref
13. Hong Hua and Bahram Javidi. 2014. A 3D integral imaging optical see-through head-mounted display. Opt. Express 22, 11 (Jun 2014). Google ScholarCross Ref
14. F. Huang, K. Chen, and G. Wetzstein. 2015. The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues. ACM Trans. Graph. (SIGGRAPH) 4 (2015). Issue 34.Google Scholar
15. F. Huang, G. Wetzstein, B. Barsky, and R. Raskar. 2014. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 4 (2014), 1–12.Google ScholarDigital Library
16. Changwon Jang, Chang-Kun Lee, Jinsoo Jeong, Gang Li, Seungjae Lee, Jiwoon Yeom, Keehoon Hong, and Byoungho Lee. 2016. Recent progress in see-through three-dimensional displays using holographic optical elements. Appl. Opt. 55, 3 (Jan 2016), A71–A85. Google ScholarCross Ref
17. Paul V. Johnson, Jared AQ. Parnell, Joohwan Kim, Christopher D. Saunter, Gordon D. Love, and Martin S. Banks. 2016. Dynamic lens and monovision 3D displays to improve viewer comfort. Opt. Express 24, 11 (May 2016), 11808–11827.Google ScholarCross Ref
18. A. Kaczorowski, G. S. Gordon, A. Palani, S. Czerniawski, and T. D. Wilkinson. 2015. Optimization-Based Adaptive Optical Correction for Holographic Projectors. Journal of Display Technology 11, 7 (July 2015), 596–603. Google ScholarCross Ref
19. Andrzej Kaczorowski, George S. D. Gordon, and Timothy D. Wilkinson. 2016. Adaptive, spatially-varying aberration correction for real-time holographic projectors. Opt. Express 24, 14 (Jul 2016). Google ScholarCross Ref
20. Robert Konrad, Emily A. Cooper, and Gordon Wetzstein. 2016. Novel Optical Configurations for Virtual Reality: Evaluating User Preference and Performance with Focus-tunable and Monovision Near-eye Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 1211–1220. Google ScholarDigital Library
21. G. Kramida. 2016. Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays. IEEE Transactions on Visualization and Computer Graphics 22, 7 (July 2016), 1912–1931. Google ScholarCross Ref
22. Douglas Lanman and David Luebke. 2013. Near-eye Light Field Displays. ACM Trans. Graph. 32, 6, Article 220 (Nov. 2013), 10 pages.Google ScholarDigital Library
23. Seungjae Lee, Changwon Jang, Seokil Moon, Jaebum Cho, and Byoungho Lee. 2016. Additive Light Field Displays: Realization of Augmented Reality with Holographic Optical Elements. ACM Trans. Graph. 35, 4, Article 60 (July 2016), 13 pages.Google ScholarDigital Library
24. Gang Li, Dukho Lee, Youngmo Jeong, Jaebum Cho, and Byoungho Lee. 2016. Holographic display for see-through augmented reality using mirror-lens holographic optical element. Opt. Lett. 41, 11 (Jun 2016). Google ScholarCross Ref
25. S. Liu, H. Hua, and D. Cheng. 2010. A Novel Prototype for an Optical See-Through Head-Mounted Display with Addressable Focus Cues. IEEE Transactions on Visualization and Computer Graphics 16, 3 (May 2010), 381–393. Google ScholarDigital Library
26. Gordon D. Love, David M. Hoffman, Philip J.W. Hands, James Gao, Andrew K. Kirby, and Martin S. Banks. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Opt. Express 17, 18 (Aug 2009), 15716–15725. Google ScholarCross Ref
27. Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David Luebke, and Henry Fuchs. 2014. Pinlight Displays: Wide Field of View Augmented Reality Eyeglasses Using Defocused Point Light Sources. ACM Trans. Graph. 33, 4, Article 89 (July 2014), 11 pages.Google ScholarDigital Library
28. Michal Makowski, Izabela Ducin, Karol Kakarenko, Jaroslaw Suszek, Maciej Sypek, and Andrzej Kolodziejczyk. 2012. Simple holographic projection in color. Opt. Express 20, 22 (Oct 2012). Google ScholarCross Ref
29. Eunkyong Moon, Myeongjae Kim, Jinyoung Roh, Hwi Kim, and Joonku Hahn. 2014. Holographic head-mounted display with RGB light emitting diode light source. Opt. Express 22, 6 (Mar 2014), 6526–6534. Google ScholarCross Ref
30. Rahul Narain, Rachel A. Albert, Abdullah Bulbul, Gregory J. Ward, Martin S. Banks, and James F. O’Brien. 2015. Optimal Presentation of Imagery with Focus Cues on Multi-plane Displays. ACM Trans. Graph. 34, 4, Article 59 (July 2015), 12 pages.Google ScholarDigital Library
31. Nitish Padmanaban, Robert Konrad, Tal Stramer, Emily A. Cooper, and Gordon Wetzstein. 2017. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proceedings of the National Academy of Sciences 114, 9 (2017), 2183–2188. Google ScholarCross Ref
32. Vitor F. Pamplona, Manuel M. Oliveira, Daniel G. Aliaga, and Ramesh Raskar. 2012. Tailored Displays to Compensate for Visual Aberrations. ACM Trans. Graph. 31, 4, Article 81 (July 2012), 12 pages.Google ScholarDigital Library
33. Yijun Qi, Chenliang Chang, and Jun Xia. 2016. Speckleless holographic display by complex modulation based on double-phase method. Opt. Express 24, 26 (Dec 2016), 30368–30378. Google ScholarCross Ref
34. Weidong Qu, Huarong Gu, Hao Zhang, and Qiaofeng Tan. 2015. Image magnification in lensless holographic projection using double-sampling Fresnel diffraction. Appl. Opt. 54, 34 (Dec 2015), 10018–10021. Google ScholarCross Ref
35. Stephan Reichelt and Norbert Leister. 2013. Computational hologram synthesis and representation on spatial light modulators for real-time 3D holographic imaging. Journal of Physics: Conference Series 415, 1 (2013), 012038.Google ScholarCross Ref
36. Brian T. Schowengerdt, Mrinal Murari, and Eric J. Seibel. 2010. 44.1: Volumetric Display using Scanned Fiber Array. SID Symposium Digest of Technical Papers 41, 1 (2010), 653–656.Google Scholar
37. P.W.M. Tsang and T. C. Poon. 2013. Novel method for converting digital Fresnel hologram to phase-only hologram based on bidirectional error diffusion. Opt. Express 21, 20 (Oct 2013). Google ScholarCross Ref
38. P.W.M. Tsang and T.-C. Poon. 2015. Fast generation of digital holograms based on warping of the wavefront recording plane. Opt. Express 23, 6 (Mar 2015), 7667–7673. Google ScholarCross Ref
39. P. W. M. Tsang and T. C. Poon. 2016. Review on the State-of-the-Art Technologies for Acquisition and Display of Digital Holograms. IEEE Transactions on Industrial Informatics 12, 3 (June 2016), 886–901. Google ScholarCross Ref
40. Fahri Yaraş, Hoonjong Kang, and Levent Onural. 2010. State of the Art in Holographic Displays: A Survey. J. Display Technol. 6, 10 (Oct 2010). Google ScholarCross Ref
41. Han-Ju Yeom, Hee-Jae Kim, Seong-Bok Kim, HuiJun Zhang, BoNi Li, Yeong-Min Ji, Sang-Hoo Kim, and Jae-Hyeung Park. 2015. 3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation. Opt. Express 23, 25 (Dec 2015), 32025–32034. Google ScholarCross Ref
42. Hiroshi Yoshikawa, Takeshi Yamaguchi, and Hiroki Uetake. 2016. Image quality evaluation and control of computer-generated holograms. Proc. SPIE 9771 (2016), 97710N–97710N-9. Google ScholarCross Ref
43. Hao Zhang, Neil Collings, Jing Chen, Bill Crossland, Daping Chu, and Jinghui Xie. 2011. Full parallax three-dimensional display with occlusion effect using computer generated hologram. Optical Engineering 50, 7 (2011), 074003-074003-5. Google ScholarCross Ref
