“Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3D computer graphics”
Conference:
Type(s):
Title:
- Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3D computer graphics
Session/Category Title: Multi-View 3D
Presenter(s)/Author(s):
Abstract:
Holograms display a 3D image in high resolution and allow viewers to focus freely as if looking through a virtual window, yet computer generated holography (CGH) hasn’t delivered the same visual quality under plane wave illumination and due to heavy computational cost. Light field displays have been popular due to their capability to provide continuous focus cues. However, light field displays must trade off between spatial and angular resolution, and do not model diffraction.We present a light field-based CGH rendering pipeline allowing for reproduction of high-definition 3D scenes with continuous depth and support of intra-pupil view-dependent occlusion. Our rendering accurately accounts for diffraction and supports various types of reference illuminations for hologram. We avoid under- and over-sampling and geometric clipping effects seen in previous work. We also demonstrate an implementation of light field rendering plus the Fresnel diffraction integral based CGH calculation which is orders of magnitude faster than the state of the art [Zhang et al. 2015], achieving interactive volumetric 3D graphics.To verify our computational results, we build a see-through, near-eye, color CGH display prototype which enables co-modulation of both amplitude and phase. We show that our rendering accurately models the spherical illumination introduced by the eye piece and produces the desired 3D imagery at the designated depth. We also analyze aliasing, theoretical resolution limits, depth of field, and other design trade-offs for near-eye CGH.
References:
1. Lukas Ahrenberg, Philip Benzie, Marcus Magnor, and John Watson. 2008. Computer generated holograms from three dimensional meshes using an analytic light transport model. Applied optics 47, 10 (2008), 1567–1574.
2. Miguel A. Alonso. 2011. Wigner functions in optics: describing beams as ray bundles and pulses as particle ensembles. Adv. Opt. Photon. 3, 4 (Dec 2011), 272–365. Cross Ref
3. Ho Bartelt, K-H Brenner, and AW Lohmann. 1980. The Wigner distribution function and its optical production. Optics Communications 32, 1 (1980), 32–38. Cross Ref
4. Martin J Bastiaans. 1997. Application of the Wigner distribution function in optics. The Wigner Distribution – Theory and Applications in Signal Processing (1997), 375–426.
5. Muharrem Bayraktar and Meriç Özcan. 2010. Method to calculate the far field of three-dimensional objects for computer-generated holography. Appl. Opt. 49, 24 (Aug 2010), 4647–4654. Cross Ref
6. Stephen A. Benton and Jr. Bove, V. Michael. 2008. Holographic Imaging. Wiley-Interscience, New York, NY, USA.
7. William P Bleha and LiJuan Alice Lei. 2013. Advances in liquid crystal on silicon (LCOS) spatial light modulator technology. (2013), 87360A-87360A.
8. Bryon R Brown and Adolf W Lohmann. 1966. Complex spatial filtering with binary masks. Applied Optics 5, 6 (1966), 967–969. Cross Ref
9. J-S Chen and DP Chu. 2015. Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications. Optics express 23, 14 (2015), 18143–18155.
10. Rick H-Y Chen and Timothy D Wilkinson. 2009. Computer generated hologram from point cloud using graphics processor. Applied optics 48, 36 (2009), 6841–6850.
11. TACM Claasen and WFG Mecklenbräuker. 1980. TIME-FREQUENCY SIGNAL ANALYSIS. Philips Journal of Research 35, 6 (1980), 372–389.
12. Dominic J DeBitetto. 1969. Holographic panoramic stereograms synthesized from white light recordings. Applied optics 8, 8 (1969), 1740–1741.
13. Rainer G Dorsch, Adolf W Lohmann, and Stefan Sinzinger. 1994. Fresnel ping-pong algorithm for two-plane computer-generated hologram display. Applied optics 33, 5 (1994), 869–875.
14. James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758–2769.
15. Martin Fuchs, Ramesh Raskar, Hans-Peter Seidel, and Hendrik P. A. Lensch. 2008. Towards Passive 6D Reflectance Field Displays. In ACM SIGGRAPH 2008 Papers (SIGGRAPH ’08). ACM, New York, NY, USA, Article 58, 8 pages.
16. Ralph W Gerchberg. 1972. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35 (1972), 237.
17. Daniel Glasner, Todd Zickler, and Anat Levin. 2014. A Reflectance Display. ACM Trans. Graph. 33, 4, Article 61 (July 2014), 12 pages.
18. Joseph W Goodman. 2005. Introduction to Fourier optics. Roberts and Company Publishers.
19. Joseph W. Goodman. 2014. Holography Viewed from the Perspective of the Light Field Camera. (2014), 3–15.
20. Joonku Hahn, Hwi Kim, Yongjun Lim, Gilbae Park, and Byoungho Lee. 2008. Wide viewing angle dynamic holographic stereogram with a curved array of spatial light modulators. Optics express 16, 16 (2008), 12372–12386.
21. Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein. 2015. The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-eye Light Field Displays with Focus Cues. ACM Trans. Graph. 34, 4, Article 60 (July 2015), 12 pages.
22. Fu-Chung Huang, Gordon Wetzstein, Brian A. Barsky, and Ramesh Raskar. 2014. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays. ACM Trans. Graph. 33, 4, Article 59 (July 2014), 12 pages.
23. R Jagannathan, R Simon, ECG Sudarshan, and R Vasudevan. 1987. Dynamical maps and nonnegative phase-space distribution functions in quantum mechanics. Physics Letters A 120, 4 (1987), 161–164. Cross Ref
24. Jia Jia, Juan Liu, Guofan Jin, and Yongtian Wang. 2014. Fast and effective occlusion culling for 3D holographic displays by inverse orthographic projection with low angular sampling. Applied optics 53, 27 (2014), 6287–6293.
25. Hoonjong Kang, Elena Stoykova, and Hiroshi Yoshikawa. 2016. Fast phase-added stereogram algorithm for generation of photorealistic 3D content. Appl. Opt. 55, 3 (Jan 2016), A135–A143. Cross Ref
26. Hwi Kim, Joonku Hahn, and Byoungho Lee. 2008. Mathematical modeling of triangle-mesh-modeled three-dimensional surface objects for digital holography. Applied optics 47, 19 (2008), D117–D127.
27. Tomasz Kozacki. 2010. On resolution and viewing of holographic image generated by 3D holographic display. Opt. Express 18, 26 (Dec 2010), 27118–27129. Cross Ref
28. Tomasz Kozacki, Małgorzata Kujawińska, Grzegorz Finke, Bryan Hennelly, and Nitesh Pandey. 2012. Extended viewing angle holographic display system with tilted SLMs in a circular configuration. Applied optics 51, 11 (2012), 1771–1780.
29. Douglas Lanman and David Luebke. 2013. Near-eye Light Field Displays. ACM Trans. Graph. 32, 6, Article 220 (Nov. 2013), 10 pages.
30. Seungjae Lee, Changwon Jang, Seokil Moon, Jaebum Cho, and Byoungho Lee. 2016. Additive Light Field Displays: Realization of Augmented Reality with Holographic Optical Elements. ACM Trans. Graph. 35, 4, Article 60 (July 2016), 13 pages.
31. Detlef Leseberg and Christian Frere. 1988. Computer-generated holograms of 3-D objects composed of tilted planar segments. Appl. Opt. 27, 14 (Jul 1988), 3020–3024. Cross Ref
32. LB Lesem, PM Hirsch, and JA Jordan. 1969. The kinoform: a new wavefront reconstruction device. IBM Journal of Research and Development 13, 2 (1969), 150–155.
33. U Levy, E Marom, and D Mendlovic. 2001. Simultaneous multicolor image formation with a single diffractive optical element. Optics letters 26, 15 (2001), 1149–1151.
34. Gang Li, Dukho Lee, Youngmo Jeong, Jaebum Cho, and Byoungho Lee. 2016. Holographic display for see-through augmented reality using mirror-lens holographic optical element. Opt. Lett. 41, 11 (Jun 2016), 2486–2489.
35. Mark Lucente and Tinsley A. Galyean. 1995. Rendering Interactive Holographic Images. (1995), 387–394.
36. Mark E Lucente. 1993. Interactive computation of holograms using a look-up table. J. Electronic Imaging 2, 1 (1993), 28–34. Cross Ref
37. A. Maimone and H. Fuchs. 2013. Computational augmented reality eyeglasses. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 29–38.
38. Andrew Maimone, Andreas Georgiou, and Joel Kollin. 2017. Holographic Near-Eye Displays for Virtual and Augmented Reality. ACM Transactions on Graphics 36 (July 2017), 85:1–85:16.
39. Andrew Maimone, Douglas Lanman, Kishore Rathinavel, Kurtis Keller, David Luebke, and Henry Fuchs. 2014. Pinlight Displays: Wide Field of View Augmented Reality Eyeglasses Using Defocused Point Light Sources. ACM Trans. Graph. 33, 4, Article 89 (July 2014), 11 pages.
40. Andrew Maimone, Gordon Wetzstein, Matthew Hirsch, Douglas Lanman, Ramesh Raskar, and Henry Fuchs. 2013. Focus 3D: Compressive Accommodation Display. ACM Trans. Graph. 32, 5, Article 153 (Oct. 2013), 13 pages.
41. MichaÃĚÂĆ Makowski, Maciej Sypek, Andrzej Kolodziejczyk, and Grzegorz MikuÃĚÂĆa. 2005. Three-plane phase-only computer hologram generated with iterative Fresnel algorithm. Optical Engineering 44, 12 (2005), 125805-125805-7. Cross Ref
42. Nobuyuki Masuda, Tomoyoshi Ito, Takashi Tanaka, Atsushi Shiraki, and Takashige Sugie. 2006. Computer generated holography using a graphics processing unit. Optics Express 14, 2 (2006), 603–608. Cross Ref
43. Kyoji Matsushima. 2005. Computer-generated holograms for three-dimensional surface objects with shade and texture. Applied optics 44, 22 (2005), 4607–4614.
44. Kyoji Matsushima and Sumio Nakahara. 2009. Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method. Applied optics 48, 34 (2009), H54–H63.
45. Kyoji Matsushima, Masaki Nakamura, and Sumio Nakahara. 2014. Silhouette method for hidden surface removal in computer holography and its acceleration using the switch-back technique. Optics express 22, 20 (2014), 24450–24465.
46. Rahul Narain, Rachel A. Albert, Abdullah Bulbul, Gregory J. Ward, Martin S. Banks, and James F. O’Brien. 2015. Optimal Presentation of Imagery with Focus Cues on Multi-plane Displays. ACM Trans. Graph. 34, 4, Article 59 (July 2015), 12 pages.
47. N. Padmanaban, R. Konrad, T. Stramer, E.A. Cooper, and G. Wetzstein. 2017. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proceedings of the National Academy of Sciences (2017).
48. Vitor F. Pamplona, Manuel M. Oliveira, Daniel G. Aliaga, and Ramesh Raskar. 2012. Tailored Displays to Compensate for Visual Aberrations. ACM Trans. Graph. 31, 4, Article 81 (July 2012), 12 pages.
49. Vitor F. Pamplona, Erick B. Passos, Jan Zizka, Manuel M. Oliveira, Everett Lawson, Esteban Clua, and Ramesh Raskar. 2011. CATRA: Interactive Measuring and Modeling of Cataracts. In ACM SIGGRAPH 2011 Papers (SIGGRAPH ’11). ACM, New York, NY, USA, Article 47, 8 pages.
50. Christoph Petz and Marcus Magnor. 2003. Fast hologram synthesis for 3D geometry models using graphics hardware. In Electronic Imaging 2003. International Society for Optics and Photonics, 266–275.
51. Weidong Qu, Huarong Gu, Hao Zhang, and Qiaofeng Tan. 2015. Image magnification in lensless holographic projection using double-sampling Fresnel diffraction. Appl. Opt. 54, 34 (Dec 2015), 10018–10021. Cross Ref
52. Stephan Reichelt, Ralf Häussler, Gerald Fütterer, Norbert Leister, Hiromi Kato, Naru Usukura, and Yuuichi Kanbayashi. 2012. Full-range, complex spatial light modulator for real-time holography. Optics letters 37, 11 (2012), 1955–1957.
53. Yusuke Sato and Yuji Sakamoto. 2012. Calculation method for reconstruction at arbitrary depth in CGH with Fourier transform optical system. In SPIE OPTO. International Society for Optics and Photonics, 82810W–82810W.
54. Takanori Senoh, Tomoyuki Mishina, Kenji Yamamoto, Ryutaro Oi, and Taiichiro Kurita. 2011. Viewing-zone-angle-expanded color electronic holography system using ultra-high-definition liquid crystal displays with undesirable light elimination. Journal of display technology 7, 7 (2011), 382–390. Cross Ref
55. Quinn YJ Smithwick, James Barabas, Daniel E Smalley, and V Michael Bove Jr. 2010. Interactive holographic stereograms with accommodation cues. (2010), 761903–761903.
56. Adrian Stern and Bahram Javidi. 2004. Sampling in the light of Wigner distribution. J. Opt. Soc. Am. A 21, 3 (Mar 2004), 360–366.
57. Tullio Tommasi and Bruno Bianco. 1993. Computer-generated holograms of tilted planes by a spatial frequency approach. JOSA A 10, 2 (1993), 299–305. Cross Ref
58. Robert Toth, Jon Hasselgren, and Tomas Akenine-Möller. 2015. Perception of Highlight Disparity at a Distance in Consumer Head-mounted Displays. In Proceedings of the 7th Conference on High-Performance Graphics (HPG ’15). ACM, New York, NY, USA, 61–66.
59. E. G. van Putten, I. M. Vellekoop, and A. P. Mosk. 2008. Spatial amplitude and phase modulation using commercial twisted nematic LCDs. Appl. Opt. 47, 12 (Apr 2008). Cross Ref
60. Koki Wakunami, Hiroaki Yamashita, and Masahiro Yamaguchi. 2013. Occlusion culling for computer generated hologram based on ray-wavefront conversion. Opt. Express 21, 19 (Sep 2013), 21811–21822. Cross Ref
61. James P Waters. 1966. Holographic image synthesis utilizing theoretical methods. Applied physics letters 9, 11 (1966), 405–407.
62. Gordon Wetzstein, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar. 2012. Tensor Displays: Compressive Light Field Synthesis Using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. 31, 4, Article 80 (July 2012), 11 pages.
63. Masahiro Yamaguchi, Hideshi Hoshino, Toshio Honda, and Nagaaki Ohyama. 1993. Phase-added stereogram: calculation of hologram using computer graphics technique. (1993), 25–31.
64. Fahri Yaraş, Hoonjong Kang, and Levent Onural. 2011. Circular holographic video display system. Optics Express 19, 10 (2011), 9147–9156. Cross Ref
65. Toyohiko Yatagai. 1976. Stereoscopic approach to 3-D display using computer-generated holograms. Appl. Opt. 15, 11 (Nov 1976), 2722–2729. Cross Ref
66. Genzhi Ye, Sundeep Jolly, V. Michael Bove, Jr., Qionghai Dai, Ramesh Raskar, and Gordon Wetzstein. 2014. Toward BxDF Display Using Multilayer Diffraction. ACM Trans. Graph. 33, 6, Article 191 (Nov. 2014), 14 pages.
67. Marina Zannoli, Gordon D. Love, Rahul Narain, and Martin S. Banks. 2016. Blur and the perception of depth at occlusions. Journal of Vision 16, 6 (2016), 17. Cross Ref
68. Hao Zhang, Qiaofeng Tan, and Guofan Jin. 2012. Holographic display system of a three-dimensional image with distortion-free magnification and zero-order elimination. Optical Engineering 51, 7 (2012), 075801-1–075801-5.
69. Hao Zhang, Yan Zhao, Liangcai Cao, and Guofan Jin. 2015. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Opt. Express 23, 4 (Feb 2015), 3901–3913.
70. Hao Zhang, Yan Zhao, Liangcai Cao, and Guofan Jin. 2016. Layered holographic stereogram based on inverse Fresnel diffraction. Applied optics 55, 3 (2016).
71. Zhengyun Zhang and Marc Levoy. 2009. Wigner distributions and how they relate to the light field. (April 2009), 1–10.
72. Yan Zhao, Liangcai Cao, Hao Zhang, Dezhao Kong, and Guofan Jin. 2015. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Optics express 23, 20 (2015), 25440–25449.
73. Remo Ziegler, Simon Bucheli, Lukas Ahrenberg, Marcus Magnor, and Markus Gross. 2007. A Bidirectional Light Field-Hologram Transform. In Computer Graphics Forum, Vol. 26. Wiley Online Library, 435–446.
