“Holographic Near-eye Display with Real-time Embedded Rendering” by Gilles, Gargasson, Hocquet and Gioia
Conference:
Type(s):
Title:
- Holographic Near-eye Display with Real-time Embedded Rendering
Session/Category Title: Holography
Presenter(s)/Author(s):
Abstract:
We present a wearable full-color holographic augmented reality headset with binocular vision support and real-time embedded hologram calculation. Contrarily to most previously proposed prototypes, our headset employs high-speed amplitude-only microdisplays and embeds a compact and lightweight electronic board to drive and synchronize the microdisplays and light source engines. In addition, to enable a standalone usage of the headset, we developed a real-time hologram rendering engine capable of computing full-color binocular holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform. Finally, we provide a comparison of the efficiency of laser diodes and superluminescent diodes for the reduction of speckle noise, which greatly affects the reconstructed image’s quality. Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues and reduced speckle noise in real-time.
References:
[1]
Kaan Akşit, Ward Lopes, Jonghyun Kim, Peter Shirley, and David Luebke. 2017. Near-eye Varifocal Augmented Reality Display Using See-through Screens. ACM Trans. Graph. 36, 6 (Nov. 2017), 189:1–189:13. https://doi.org/10.1145/3130800.3130892
[2]
Misty Antonioli, Corinne Blake, and Kelly Sparks. 2014. Augmented Reality Applications in Education. The Journal of Technology Studies 40, 1/2 (2014), 96–107. https://www.jstor.org/stable/43604312 Publisher: Epsilon Pi Tau, Inc.
[3]
E. Z. Barsom, M. Graafland, and M. P. Schijven. 2016. Systematic review on the effectiveness of augmented reality applications in medical training. Surgical Endoscopy 30, 10 (Oct. 2016), 4174–4183. https://doi.org/10.1007/s00464-016-4800-6
[4]
Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends in Human–Computer Interaction 8, 2-3 (March 2015), 73–272. https://doi.org/10.1561/1100000049 Publisher: Now Publishers, Inc.
[5]
Pierre-Alexandre Blanche. 2021. Holography, and the future of 3D display. Light: Advanced Manufacturing 2, 4 (Dec. 2021), 446–459. https://doi.org/10.37188/lam.2021.028 Publisher: Light: Advanced Manufacturing.
[6]
Raffaello Bonghi. 2022. Jetson stats. https://github.com/rbonghi/jetson_stats original-date: 2018-11-24T19:42:07Z.
[7]
Olof Bryngdahl and Adolf Lohmann. 1968. Single-Sideband Holography. JOSA 58, 5 (May 1968), 620–624. https://doi.org/10.1364/JOSA.58.000620
[8]
Chenliang Chang, Kiseung Bang, Gordon Wetzstein, Byoungho Lee, and Liang Gao. 2020. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7, 11 (Nov. 2020), 1563–1578. https://doi.org/10.1364/OPTICA.406004 Publisher: Optica Publishing Group.
[9]
Hung-Lin Chi, Shih-Chung Kang, and Xiangyu Wang. 2013. Research trends and opportunities of augmented reality applications in architecture, engineering, and construction. Automation in Construction 33 (Aug. 2013), 116–122. https://doi.org/10.1016/j.autcon.2012.12.017
[10]
Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O’Toole, and Gordon Wetzstein. 2022. Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators. In ACM SIGGRAPH 2022 Conference Proceedings(SIGGRAPH ’22). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3528233.3530734
[11]
Yuanbo Deng and Daping Chu. 2017. Coherence properties of different light sources and their effect on the image sharpness and speckle of holographic displays. Scientific Reports 7, 1 (July 2017), 5893. https://doi.org/10.1038/s41598-017-06215-x
[12]
Xinhui Duan, Juan Liu, Xueliang Shi, Zhiqi Zhang, and Jiasheng Xiao. 2020. Full-color see-through near-eye holographic display with 80° field of view and an expanded eye-box. Optics Express 28, 21 (Oct. 2020), 31316–31329. https://doi.org/10.1364/OE.399359 Publisher: Optical Society of America.
[13]
Nam-Duong Duong, Christophe Cutullic, Jean-Marie Henaff, and Jérôme Royan. 2022. AR Cloud: Towards Collaborative Augmented Reality at a Large-Scale. In 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). 733–738. https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00155 ISSN: 2771-1110.
[14]
Qiankun Gao, Juan Liu, Xinhui Duan, Tao Zhao, Xin Li, and Peilin Liu. 2017. Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating filter. Optics Express 25, 7 (April 2017), 8412–8424. https://doi.org/10.1364/OE.25.008412
[15]
Antonin Gilles. 2021. Real-time embedded hologram calculation for augmented reality glasses. In 2021 International Conference on Visual Communications and Image Processing (VCIP). IEEE, 1–5. https://doi.org/10.1109/VCIP53242.2021.9675435 ISSN: 2642-9357.
[16]
Antonin Gilles and Patrick Gioia. 2020. Real-time computer-generated hologram calculation using pre-computed angular spectra. In Optics, Photonics and Digital Technologies for Imaging Applications VI, Vol. 11353. International Society for Optics and Photonics, 1135304. https://doi.org/10.1117/12.2554537
[17]
Manu Gopakumar, Jonghyun Kim, Suyeon Choi, Yifan Peng, and Gordon Wetzstein. 2021. Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays. Optics Letters 46, 23 (Dec. 2021), 5822–5825. https://doi.org/10.1364/OL.442851 Publisher: Optica Publishing Group.
[18]
David M. Hoffman, Ahna R. Girshick, Kurt Akeley, and Martin S. Banks. 2008. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3 (March 2008), 33. https://doi.org/10.1167/8.3.33
[19]
Changwon Jang, Kiseung Bang, Minseok Chae, Byoungho Lee, and Douglas Lanman. 2022. Waveguide Holography: Towards True 3D Holographic Glasses. https://doi.org/10.48550/arXiv.2211.02784 arXiv:2211.02784 [physics].
[20]
Changwon Jang, Kiseung Bang, Gang Li, and Byoungho Lee. 2018. Holographic Near-eye Display with Expanded Eye-box. ACM Trans. Graph. 37, 6 (Dec. 2018), 195:1–195:14. https://doi.org/10.1145/3272127.3275069
[21]
Hojung Kim, Yongkyu Kim, Hyunwook Ji, Hyunsik Park, Jungkwuen An, Hoon Song, Yun Tae Kim, Hong-Seok Lee, and Kichul Kim. 2019. A Single-Chip FPGA Holographic Video Processor. IEEE Transactions on Industrial Electronics 66, 3 (March 2019), 2066–2073. https://doi.org/10.1109/TIE.2018.2835424 Conference Name: IEEE Transactions on Industrial Electronics.
[22]
Jonghyun Kim, Manu Gopakumar, Suyeon Choi, Yifan Peng, Ward Lopes, and Gordon Wetzstein. 2022. Holographic Glasses for Virtual Reality. In ACM SIGGRAPH 2022 Conference Proceedings(SIGGRAPH ’22). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3528233.3530739
[23]
Mugeon Kim, Sungjin Lim, Geunseop Choi, Youngmin Kim, Hwi Kim, and Joonku Hahn. 2018. Expanded Exit-Pupil Holographic Head-Mounted Display With High-Speed Digital Micromirror Device. ETRI Journal 40, 3 (June 2018), 366–375. https://doi.org/10.4218/etrij.2017-0166
[24]
Bernard C. Kress. 2020. Optical Architectures for Augmented-, Virtual-, and Mixed-Reality Headsets. SPIE. https://doi.org/10.1117/3.2559304
[25]
Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chun Chen, and Byoungho Lee. 2022. High-contrast, speckle-free, true 3D holography via binary CGH optimization. Scientific Reports 12, 1 (Feb. 2022), 2811. https://doi.org/10.1038/s41598-022-06405-2 Number: 1 Publisher: Nature Publishing Group.
[26]
Andrew Maimone, Andreas Georgiou, and Joel S. Kollin. 2017. Holographic Near-eye Displays for Virtual and Augmented Reality. ACM Trans. Graph. 36, 4 (July 2017), 85:1–85:16. https://doi.org/10.1145/3072959.3073624
[27]
Nathan Matsuda, Alexander Fix, and Douglas Lanman. 2017. Focal Surface Displays. ACM Trans. Graph. 36, 4 (July 2017), 86:1–86:14. https://doi.org/10.1145/3072959.3073590
[28]
Eishin Murakami, Yuki Oguro, and Yuji Sakamoto. 2017. Study on Compact Head-Mounted Display System Using Electro-Holography for Augmented Reality. IEICE Transactions on Electronics E100.C (Nov. 2017), 965–971. https://doi.org/10.1587/transele.E100.C.965
[29]
A. Y. C. Nee, S. K. Ong, G. Chryssolouris, and D. Mourtzis. 2012. Augmented reality applications in design and manufacturing. CIRP Annals 61, 2 (Jan. 2012), 657–679. https://doi.org/10.1016/j.cirp.2012.05.010
[30]
Jae-Hyeung Park and Seong-Bok Kim. 2018. Optical see-through holographic near-eye-display with eyebox steering and depth of field control. Optics Express 26, 21 (Oct. 2018), 27076–27088. https://doi.org/10.1364/OE.26.027076
[31]
Jae-Hyeung Park and Byoungho Lee. 2022. Holographic techniques for augmented reality and virtual reality near-eye displays. Light: Advanced Manufacturing 3, 1 (Jan. 2022), 1–14. https://doi.org/10.37188/lam.2022.009 Publisher: Light: Advanced Manufacturing.
[32]
Yifan Peng, Suyeon Choi, Jonghyun Kim, and Gordon Wetzstein. 2021. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Science Advances 7, 46 (Nov. 2021), eabg5040. https://doi.org/10.1126/sciadv.abg5040 Publisher: American Association for the Advancement of Science.
[33]
Liang Shi, Beichen Li, and Wojciech Matusik. 2022. End-to-end learning of 3D phase-only holograms for holographic display. Light: Science & Applications 11, 1 (Aug. 2022), 247. https://doi.org/10.1038/s41377-022-00894-6 Number: 1 Publisher: Nature Publishing Group.
[34]
Takaaki Ueno and Yasuhiro Takaki. 2018. Super multi-view near-eye display to solve vergence-accommodation conflict. Optics Express 26, 23 (Nov. 2018), 30703–30715. https://doi.org/10.1364/OE.26.030703
[35]
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (April 2004), 600–612. https://doi.org/10.1109/TIP.2003.819861 Conference Name: IEEE Transactions on Image Processing.
[36]
Yota Yamamoto, Nobuyuki Masuda, Ryuji Hirayama, Hirotaka Nakayama, Takashi Kakue, Tomoyoshi Shimobaba, and Tomoyoshi Ito. 2019. Special-purpose computer for electroholography in embedded systems. OSA Continuum 2, 4 (April 2019), 1166–1173. https://doi.org/10.1364/OSAC.2.001166 Publisher: Optical Society of America.
[37]
Takuo Yoneyama, Eishin Murakami, Yuki Oguro, Hibiki Kubo, Kazuhiro Yamaguchi, and Yuji Sakamoto. 2018. Holographic head-mounted display with correct accommodation and vergence stimuli. Optical Engineering 57, 6 (May 2018), 061619. https://doi.org/10.1117/1.OE.57.6.061619
[38]
Roberts Zabels, Krišs Osmanis, Mārtiņš Narels, Uģis Gertners, Ainārs Ozols, Kārlis Rūtenbergs, and Ilmārs Osmanis. 2019. AR Displays: Next-Generation Technologies to Solve the Vergence–Accommodation Conflict. Applied Sciences 9, 15 (Jan. 2019), 3147. https://doi.org/10.3390/app9153147 Number: 15 Publisher: Multidisciplinary Digital Publishing Institute.
[39]
Zhiqi Zhang, Juan Liu, Qiankun Gao, Xinhui Duan, and Xueliang Shi. 2019. A full-color compact 3D see-through near-eye display system based on complex amplitude modulation. Optics Express 27, 5 (March 2019), 7023–7035. https://doi.org/10.1364/OE.27.007023 Publisher: Optical Society of America.
