“Holographic near-eye displays based on overlap-add stereograms” by Padmanaban, Peng and Wetzstein
Conference:
Type(s):
Title:
- Holographic near-eye displays based on overlap-add stereograms
Session/Category Title: Thoughts on Display
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Holographic near-eye displays are a key enabling technology for virtual and augmented reality (VR/AR) applications. Holographic stereograms (HS) are a method of encoding a light field into a hologram, which enables them to natively support view-dependent lighting effects. However, existing HS algorithms require the choice of a hogel size, forcing a tradeoff between spatial and angular resolution. Based on the fact that the short-time Fourier transform (STFT) connects a hologram to its observable light field, we develop the overlap-add stereogram (OLAS) as the correct method of “inverting” the light field into a hologram via the STFT. The OLAS makes more efficient use of the information contained within the light field than previous HS algorithms, exhibiting better image quality at a range of distances and hogel sizes. Most remarkably, the OLAS does not degrade spatial resolution with increasing hogel size, overcoming the spatio-angular resolution tradeoff that previous HS algorithms face. Importantly, the optimal hogel size of previous methods typically varies with the depth of every object in a scene, making the OLAS not only a hogel size-invariant method, but also nearly scene independent. We demonstrate the performance of the OLAS both in simulation and on a prototype near-eye display system, showing focusing capabilities and view-dependent effects.
References:
1. J. Allen. 1977. Short term spectral analysis, synthesis, and modification by discrete Fourier transform. IEEE Trans. ASSP 25, 3 (1977), 235–238.Google ScholarCross Ref
2. Martin J Bastiaans. 1978. The Wigner distribution function applied to optical signals and systems. Optics communications 25, 1 (1978), 26–30.Google Scholar
3. Stephen A. Benton. 1983. Survey Of Holographic Stereograms. In Proc. SPIE, Vol. 0367.Google Scholar
4. Stephen A. Benton and V. Michael Bove. 2008. Holographic Imaging. Wiley-Interscience.Google Scholar
5. CB Burckhardt. 1970. A simplification of Lee’s method of generating holograms by computer. Applied optics 9, 8 (1970), 1949–1949.Google Scholar
6. Jen-Hao Rick Chang, BVK Kumar, and Aswin C Sankaranarayanan. 2018. Towards multifocal displays with dense focal stacks. In SIGGRAPH Asia 2018 Technical Papers. ACM, 198.Google Scholar
7. Jhen-Si Chen and Daping Chu. 2015. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. OSA Opt. Express 23, 14 (2015), 18143–18155.Google ScholarCross Ref
8. Rick H-Y Chen and Timothy D Wilkinson. 2009. Computer generated hologram with geometric occlusion using GPU-accelerated depth buffer rasterization for three-dimensional display. Applied optics 48, 21 (2009), 4246–4255.Google Scholar
9. Rainer G Dorsch, Adolf W Lohmann, and Stefan Sinzinger. 1994. Fresnel ping-pong algorithm for two-plane computer-generated hologram display. OSA Applied optics 33, 5 (1994), 869–875.Google Scholar
10. James R Fienup. 1978. Reconstruction of an object from the modulus of its Fourier transform. Optics letters 3, 1 (1978), 27–29.Google Scholar
11. James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758–2769.Google Scholar
12. Qiankun Gao, Juan Liu, Jian Han, and Xin Li. 2016. Monocular 3D see-through head-mounted display via complex amplitude modulation. OSA Opt. Express 24, 15 (2016), 17372–17383.Google ScholarCross Ref
13. Ralph W Gerchberg. 1972. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35 (1972), 237–246.Google Scholar
14. Joseph W Goodman. 2005. Introduction to Fourier optics. Roberts and Company Publishers.Google Scholar
15. Joseph W. Goodman. 2014. Holography Viewed from the Perspective of the Light Field Camera. In Fringe 2013, Wolfgang Osten (Ed.). Springer Berlin Heidelberg, 3–15.Google Scholar
16. Joonku Hahn, Hwi Kim, Yongjun Lim, Gilbae Park, and Byoungho Lee. 2008. Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators. OSA Opt. Express 16, 16 (2008), 12372–12386.Google ScholarCross Ref
17. Stephen Hamann, Liang Shi, Olav Solgaard, and Gordon Wetzstein. 2018. Time-multiplexed light field synthesis via factored Wigner distribution function. OSA Opt. Lett. 43, 3 (2018), 599–602.Google ScholarCross Ref
18. R Häussler, S Reichelt, N Leister, E Zschau, R Missbach, and A Schwerdtner. 2009. Large real-time holographic displays: from prototypes to a consumer product. In Proc. Stereoscopic Displays and Applications, Vol. 7237.Google ScholarCross Ref
19. Keehoon Hong, Soon-gi Park, Jiwoon Yeom, Jonghyun Kim, Ni Chen, Kyungsuk Pyun, Chilsung Choi, Sunil Kim, Jungkwuen An, Hong-Seok Lee, et al. 2013. Resolution enhancement of holographic printer using a hogel overlapping method. Optics express 21, 12 (2013), 14047–14055.Google Scholar
20. CK Hsueh and AA Sawchuk. 1978. Computer-generated double-phase holograms. Applied optics 17, 24 (1978), 3874–3883.Google Scholar
21. Hong Hua and Bahram Javidi. 2014. A 3D integral imaging optical see-through head-mounted display. Optics express 22, 11 (2014), 13484–13491.Google Scholar
22. Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein. 2015. The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues. ACM Trans. Graph. (SIGGRAPH) 34, 4 (2015), 60.Google ScholarDigital Library
23. Kishore Jaganathan, Yonina C Eldar, and Babak Hassibi. 2015. Phase retrieval: An overview of recent developments. arXiv preprint arXiv:1510.07713 (2015).Google Scholar
24. Changwon Jang, Kiseung Bang, Gang Li, and Byoungho Lee. 2018. Holographic Near-eye Display with Expanded Eye-box. ACM Trans. Graph. (SIGGRAPH Asia) 37, 6 (2018).Google Scholar
25. Changwon Jang, Kiseung Bang, Seokil Moon, Jonghyun Kim, Seungjae Lee, and Byoungho Lee. 2017. Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina. ACM Trans. Graph. (SIGGRAPH Asia) 36, 6 (2017), 190.Google ScholarDigital Library
26. Hoonjong Kang, Elena Stoykova, and Hiroshi Yoshikawa. 2016. Fast phase-added stereogram algorithm for generation of photorealistic 3D content. Applied optics 55, 3 (2016), A135–A143.Google Scholar
27. Hoonjong Kang, Takeshi Yamaguchi, and Hiroshi Yoshikawa. 2008. Accurate phase-added stereogram to improve the coherent stereogram. OSA Appl. Opt. 47, 19 (2008).Google Scholar
28. Douglas Lanman, Matthew Hirsch, Yunhee Kim, and Ramesh Raskar. 2010. Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. (SIGGRAPH Asia) 29, 6 (2010), 163:1–163:10.Google Scholar
29. Douglas Lanman and David Luebke. 2013. Near-eye light field displays. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6 (2013), 220.Google ScholarDigital Library
30. Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, and Ramesh Raskar. 2011. Polarization Fields: Dynamic Light Field Display Using Multi-layer LCDs. ACM Trans. Graph. 30, 6, Article 186 (2011), 186:1–186:10 pages.Google ScholarDigital Library
31. Seungjae Lee, Youngjin Jo, Dongheon Yoo, Jaebum Cho, Dukho Lee, and Byoungho Lee. 2019. Tomographic near-eye displays. Nature Communications 10, 1 (2019), 2497.Google ScholarCross Ref
32. Wai Hon Lee. 1970. Sampled Fourier transform hologram generated by computer. Applied Optics 9, 3 (1970), 639–643.Google ScholarCross Ref
33. Marc Levoy and Pat Hanrahan. 1996. Light Field Rendering. In Proc. SIGGRAPH. 31–42.Google ScholarDigital Library
34. Gang Li, Dukho Lee, Youngmo Jeong, Jaebum Cho, and Byoungho Lee. 2016. Holographic display for see-through augmented reality using mirror-lens holographic optical element. OSA Opt. Lett. 41, 11 (2016), 2486–2489.Google ScholarCross Ref
35. Adolf W. Lohmann, Rainer G. Dorsch, David Mendlovic, Zeev Zalevsky, and Carlos Ferreira. 1996. Space-bandwidth product of optical signals and systems. OSA J. Opt. Soc. Am. A 13, 3 (1996), 470–473.Google ScholarCross Ref
36. Mark Lucente. 1994. Diffraction-Specific Fringe Computation for Electro-Holography. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarDigital Library
37. Mark Lucente and Tinsley A Galyean. 1995. Rendering interactive holographic images. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. ACM, 387–394.Google ScholarDigital Library
38. Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Transactions on Graphics (TOG) 36, 4 (2017), 85.Google ScholarDigital Library
39. Kyoji Matsushima and Sumio Nakahara. 2009. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method. Applied optics 48, 34 (2009), H54–H63.Google Scholar
40. Eunkyong Moon, Myeongjae Kim, Jinyoung Roh, Hwi Kim, and Joonku Hahn. 2014. Holographic head-mounted display with RGB light emitting diode light source. OSA Opt. Express 22, 6 (2014), 6526–6534.Google ScholarCross Ref
41. Jae-Hyeung Park. 2017. Recent progress in computer-generated holography for three-dimensional scenes. Journal of Information Display 18, 1 (2017), 1–12.Google ScholarCross Ref
42. Jae-Hyeung Park and Mehdi Askari. 2019. Non-hogel-based computer generated hologram from light field using complex field recovery technique from Wigner distribution function. Optics express 27, 3 (2019), 2562–2574.Google Scholar
43. Yifan Peng, Xiong Dun, Qilin Sun, and Wolfgang Heidrich. 2017. Mix-and-match holography. ACM Trans. Graph. 36, 6 (2017), 191.Google ScholarDigital Library
44. Kishore Rathinavel, Hanpeng Wang, Alex Blate, and Henry Fuchs. 2018. An extended depth-at-field volumetric near-eye augmented reality display. IEEE transactions on visualization and computer graphics 24, 11 (2018), 2857–2866.Google ScholarCross Ref
45. Pasi Saarikko and Pasi Kostamo. 2016. Waveguide. US Patent 9,513,480 B2.Google Scholar
46. Brian Schowengerdt, Dianmin Lin, and Pierre St. Hillaire. 2018. Waveguide. US Patent Application 0,052,277 A1.Google Scholar
47. Liang Shi, Fu-Chung Huang, Ward Lopes, Wojciech Matusik, and David Luebke. 2017. Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics. ACM Trans. Graph. (SIGGRAPH Asia) 36, 6, Article 236 (2017), 236:1–236:17 pages.Google Scholar
48. Chris Slinger, Colin Cameron, and Maurice Stanley. 2005. Computer-generated holography as a generic display technology. Computer 38, 8 (2005), 46–53.Google ScholarDigital Library
49. Quinn YJ Smithwick, James Barabas, Daniel E Smalley, and V Michael Bove Jr. 2010. Interactive holographic stereograms with accommodation cues. In Practical Holography XXIV: Materials and Applications, Vol. 7619. International Society for Optics and Photonics, 761903.Google Scholar
50. Koki Wakunami, Hiroaki Yamashita, and Masahiro Yamaguchi. 2013. Occlusion culling for computer generated hologram based on ray-wavefront conversion. Optics express 21, 19 (2013), 21811–21822.Google Scholar
51. Gordon Wetzstein, Douglas Lanman, Wolfgang Heidrich, and Ramesh Raskar. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. 30, 4, Article 95 (2011), 95:1–95:12 pages.Google ScholarDigital Library
52. Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 4 (2012), 1–11.Google ScholarDigital Library
53. Masahiro Yamaguchi, Hideshi Hoshino, Toshio Honda, and Nagaaki Ohyama. 1993. Phase-added stereogram: calculation of hologram using computer graphics technique. In Practical Holography VII: Imaging and Materials, Vol. 1914. International Society for Optics and Photonics, 25–32.Google ScholarCross Ref
54. Fahri Yaras, Hoonjong Kang, and Levent Onural. 2010. State of the Art in Holographic Displays: A Survey. Journal of Display Technology 6, 10 (2010), 443–454.Google ScholarCross Ref
55. Toyohiko Yatagai. 1976. Stereoscopic approach to 3-D display using computer-generated holograms. Applied optics 15, 11 (1976), 2722–2729.Google Scholar
56. Han-Ju Yeom, Hee-Jae Kim, Seong-Bok Kim, HuiJun Zhang, BoNi Li, Yeong-Min Ji, Sang-Hoo Kim, and Jae-Hyeung Park. 2015. 3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation. OSA Opt. Express 23, 25 (2015), 32025–32034.Google ScholarCross Ref
57. Hao Zhang, Liangcai Cao, and Guofan Jin. 2017. Computer-generated hologram with occlusion effect using layer-based processing. Applied optics 56, 13 (2017), F138–F143.Google Scholar
58. Hao Zhang, Neil Collings, Jing Chen, Bill A Crossland, Daping Chu, and Jinghui Xie. 2011. Full parallax three-dimensional display with occlusion effect using computer generated hologram. Optical Engineering 50, 7 (2011), 074003.Google ScholarCross Ref
59. Hao Zhang, Yan Zhao, Liangcai Cao, and Guofan Jin. 2015. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Optics express 23, 4 (2015), 3901–3913.Google Scholar
60. Zhengyun Zhang and M. Levoy. 2009. Wigner distributions and how they relate to the light field. In Proc. ICCP. 1–10.Google Scholar
61. Zichen Zhang, Zheng You, and Daping Chu. 2014. Fundamentals of phase-only liquid crystal on silicon (LCOS) devices. Light: Science & Applications 3 (2014), e213.Google ScholarCross Ref
62. Remo Ziegler, Simon Bucheli, Lukas Ahrenberg, Marcus Magnor, and Markus Gross. 2007. A Bidirectional Light Field-Hologram Transform. In Computer Graphics Forum (Eurographics), Vol. 26. 435–446.Google ScholarCross Ref
63. Matthias Zwicker, Wojciech Matusik, Fredo Durand, and Hans-Peter Pfister. 2006. Antialiasing for automultiscopic 3D displays. In Eurographics Symposium on Rendering.Google ScholarDigital Library
