“Multi-color Holograms Improve Brightness in Holographic Displays” by Kavakli, Shi, Urey, Matusik and Akşit
Conference:
Type(s):
Title:
- Multi-color Holograms Improve Brightness in Holographic Displays
Session/Category Title: Holography
Presenter(s)/Author(s):
Abstract:
Holographic displays generate Three-Dimensional (3D) images by displaying single-color holograms time-sequentially, each lit by a single-color light source. However, representing each color one by one limits brightness in holographic displays. This paper introduces a new driving scheme for realizing brighter images in holographic displays. Unlike the conventional driving scheme, our method utilizes three light sources to illuminate each displayed hologram simultaneously at various intensity levels. In this way, our method reconstructs a multiplanar three-dimensional target scene using consecutive multi-color holograms and persistence of vision. We co-optimize multi-color holograms and required intensity levels from each light source using a gradient descent-based optimizer with a combination of application-specific loss terms. We experimentally demonstrate that our method can increase the intensity levels in holographic displays up to three times, reaching a broader range and unlocking new potentials for perceptual realism in holographic displays.
References:
[1]
Eirikur Agustsson and Radu Timofte. 2017. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[2]
Jorge Albero, Pascuala García-Martínez, José Luis Martínez, and Ignacio Moreno. 2013. Second order diffractive optical elements in a spatial light modulator with large phase dynamic range. Optics and Lasers in Engineering 51, 2 (2013), 111–115.
[3]
Ozan Cakmakci, Yi Qin, Peter Bosel, and Gordon Wetzstein. 2021. Holographic pancake optics for thin and lightweight optical see-through augmented reality. Optics Express 29, 22 (2021), 35206–35215.
[4]
Praneeth Chakravarthula, Ethan Tseng, Tarun Srivastava, Henry Fuchs, and Felix Heide. 2020. Learned Hardware-in-the-Loop Phase Retrieval for Holographic near-Eye Displays. ACM Trans. Graph. 39, 6, Article 186 (nov 2020), 18 pages. https://doi.org/10.1145/3414685.3417846
[5]
Chenliang Chang, Kiseung Bang, Gordon Wetzstein, Byoungho Lee, and Liang Gao. 2020. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7, 11 (Nov 2020), 1563–1578. http://www.osapublishing.org/optica/abstract.cfm?URI=optica-7-11-1563
[6]
Brian Chao, Manu Gopakumar, Suyeon Choi, and Gordon Wetzstein. 2023. High-brightness holographic projection. Opt. Lett. 48, 15 (Aug 2023), 4041–4044. https://doi.org/10.1364/OL.489617
[7]
Suyeon Choi, Manu Gopakumar, Jonghyun Kim, Matthew O’Toole, Gordon Wetzstein, 2022. Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators. arXiv preprint arXiv:2205.02367 (2022).
[8]
Hyon-Gon Choo, Maksymilian Chlipała, and Tomasz Kozacki. 2018. Image blur and visual perception for rainbow holographic display. In Optics, Photonics, and Digital Technologies for Imaging Applications V, Vol. 10679. SPIE, 195–201.
[9]
Gerwin Damberg, James Gregson, and Wolfgang Heidrich. 2016. High brightness HDR projection using dynamic freeform lensing. ACM Transactions on Graphics (TOG) 35, 3 (2016), 1–11.
[10]
Jeffrey A Davis, Benjamin K Gutierrez, Ignacio Moreno, and Don M Cottrell. 2020. Spatial light modulators with large phase-modulation: application to encode lenses with very short focal lengths. In Advances in Display Technologies X, Vol. 11304. SPIE, 57–62.
[11]
Fu-Chung Huang, Dawid Pajak, Jonghyun Kim, Jan Kautz, and David Luebke. 2017. Mixed-primary factorization for dual-frame computational displays.ACM Trans. Graph. 36, 4 (2017), 149–1.
[12]
Susmija Jabbireddy, Yang Zhang, Martin Peckerar, Mario Dagenais, and Amitabh Varshney. 2022. Sparse Nanophotonic Phased Arrays for Energy-Efficient Holographic Displays. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 553–562.
[13]
Changwon Jang, Kiseung Bang, Minseok Chae, Byoungho Lee, and Douglas Lanman. 2022. Waveguide Holography: Towards True 3D Holographic Glasses. arXiv preprint arXiv:2211.02784 (2022).
[14]
Changwon Jang, Olivier Mercier, Kiseung Bang, Gang Li, Yang Zhao, and Douglas Lanman. 2020. Design and fabrication of freeform holographic optical elements. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–15.
[15]
Youngjin Jo, Dongheon Yoo, Dukho Lee, Minkwan Kim, and Byoungho Lee. 2022. Multi-illumination 3D holographic display using a binary mask. Optics Letters 47, 10 (2022), 2482–2485.
[16]
Andrew Kadis, Youchao Wang, Fan Yang, Ralf Mouthaan, Benjamin Wetherfield, Daoming Dong, and Timothy Wilkinson. 2022. High dynamic range (HDR) imaging for camera-in-the-loop computer-generated holography (CGH) using spatially varying pixel exposures. In Practical Holography XXXVI: Displays, Materials, and Applications, Vol. 12026. SPIE, 71–76.
[17]
Sergey Kastryulin, Jamil Zakirov, Denis Prokopenko, and Dmitry V. Dylov. 2022. PyTorch Image Quality: Metrics for Image Quality Assessment. https://doi.org/10.48550/ARXIV.2208.14818
[18]
Isaac Kauvar, Samuel J Yang, Liang Shi, Ian McDowall, and Gordon Wetzstein. 2015. Adaptive color display via perceptually-driven factored spectral projection.ACM Trans. Graph. 34, 6 (2015), 165–1.
[19]
Koray Kavakli and Kaan Akşit. 2022. Introduction to Odak: a Differentiable Toolkit for Optical Sciences, Vision Sciences and Computer Graphics, In Frontiers in Optics + Laser Science 2022 (FIO, LS). Frontiers in Optics + Laser Science 2022 (FIO, LS), FTu1A.1. https://doi.org/10.1364/FIO.2022.FTu1A.1
[20]
Koray Kavaklı, Yuta Itoh, Hakan Urey, and Kaan Akşit. 2023. Realistic Defocus Blur for Multiplane Computer-Generated Holography. IEEE VR 2023 (March 2023). https://doi.org/10.1109/VR55154.2023.00057
[21]
Koray Kavaklı, Hakan Urey, and Kaan Akşit. 2022. Learned holographic light transport. Applied Optics 61, 5 (2022), B50–B55.
[22]
Koray Kavakli, David Robert Walton, Nick Antipa, Rafał Mantiuk, Douglas Lanman, and Kaan Akşit. 2022. Optimizing vision and visuals: lectures on cameras, displays and perception. In ACM SIGGRAPH 2022 Courses. 1–66.
[23]
Dongyeon Kim, Seung-Woo Nam, Byounghyo Lee, Jong-Mo Seo, and Byoungho Lee. 2022b. Accommodative holography: improving accommodation response for perceptually realistic holographic displays. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–15.
[24]
Jonghyun Kim, Manu Gopakumar, Suyeon Choi, Yifan Peng, Ward Lopes, and Gordon Wetzstein. 2022a. Holographic glasses for virtual reality. In ACM SIGGRAPH 2022 Conference Proceedings. 1–9.
[25]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[26]
George Alex Koulieris, Kaan Akşit, Michael Stengel, Rafał K Mantiuk, Katerina Mania, and Christian Richardt. 2019. Near-eye display and tracking technologies for virtual and augmented reality. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 493–519.
[27]
Tomasz Kozacki, Maksymilian Chlipala, and Hyon-Gon Choo. 2018. Fourier rainbow holography. Optics express 26, 19 (2018), 25086–25097.
[28]
Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chun Chen, and Byoungho Lee. 2022. High-contrast, speckle-free, true 3D holography via binary CGH optimization. Sci. Rep. 12, 1 (Feb. 2022), 2811.
[29]
Seung Hee Lee, Sung Min Kim, and Shin-Tson Wu. 2009. Emerging vertical-alignment liquid-crystal technology associated with surface modification using UV-curable monomer. Journal of the Society for Information Display 17, 7 (2009), 551–559.
[30]
Yonghee Lee, Peng Xia, Ryosuke Yonesaka, Yasuhiro Awatsuji, Kenzo Nishio, and Osamu Matoba. 2015. Experimental demonstration of high dynamic-range digital holography. In Digital Holography and Three-Dimensional Imaging. Optica Publishing Group, DM3A–4.
[31]
Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. 36, 4 (July 2017), 1–16.
[32]
Nathan Matsuda, Alex Chapiro, Yang Zhao, Clinton Smith, Romain Bachy, and Douglas Lanman. 2022. Realistic Luminance in VR. In SIGGRAPH Asia 2022 Conference Papers. 1–8.
[33]
Kyoji Matsushima and Tomoyoshi Shimobaba. 2009. Band-limited angular spectrum method for numerical simulation of free-space propagation in far and near fields. Optics express 17, 22 (2009), 19662–19673.
[34]
Patrick I Oden, Terry A Bartlett, William C McDonald, James C Baker, and James N Hall. 2020. Innovations with a massively paralleled, microelectromechanical systems (MEMS) toward piston-mode-based phase light modulator (PLM). In Emerging Digital Micromirror Device Based Systems and Applications XII, Vol. 11294. SPIE, 72–78.
[35]
YongKeun Park, Jae-Hyeung Park, Hwi Kim, Hong-Seok Lee, Yoshio Hayasaki, and Ting-Chung Poon. 2023. The knight of holographic displays. Nature Photonics (19 Jan 2023). https://doi.org/10.1038/s41566-022-01150-4
[36]
Yifan Peng, Suyeon Choi, Nitish Padmanaban, and Gordon Wetzstein. 2020. Neural holography with camera-in-the-loop training. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–14.
[37]
Elisabet Pérez-Cabré and María S Millán. 2016. First-order and multi-order diffractive lens using a device with 8π phase modulation range. In 2016 15th Workshop on Information Optics (WIO). IEEE, 1–3.
[38]
Dapu Pi, Juan Liu, and Yongtian Wang. 2022a. Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display. Light: Science & Applications 11, 1 (26 Jul 2022), 231. https://doi.org/10.1038/s41377-022-00916-3
[39]
Dapu Pi, Jie Wang, Juan Liu, Jingnan Li, Yutong Sun, Yan Yang, Weirui Zhao, and Yongtian Wang. 2022b. Color dynamic holographic display based on complex amplitude modulation with bandwidth constraint strategy. Opt. Lett. 47, 17 (Sep 2022), 4379–4382. https://doi.org/10.1364/OL.469463
[40]
Krishnendu Samanta, Joby Joseph, Balpreet Singh Ahluwalia, and Krishna Agarwal. 2019. Study of electric fields of diffraction from spatial light modulator: discussion. JOSA A 36, 10 (2019), 1778–1786.
[41]
Brian P Schmidt, Maureen Neitz, and Jay Neitz. 2014. Neurobiological hypothesis of color appearance and hue perception.Journal of the Optical Society of America. A, Optics, image science, and vision 31 4 (2014), A195–207.
[42]
Karl Schulmeister and Mathieu Jean. 2010. The risk of retinal injury from Class 2 and visible Class 3R lasers, including medical laser aiming beams. Medical Laser Application 25, 2 (2010), 99–110.
[43]
Helge Seetzen, Wolfgang Heidrich, Wolfgang Stuerzlinger, Greg Ward, Lorne Whitehead, Matthew Trentacoste, Abhijeet Ghosh, and Andrejs Vorozcovs. 2004. High dynamic range display systems. In ACM SIGGRAPH 2004 Papers. 760–768.
[44]
Liang Shi, Beichen Li, Changil Kim, Petr Kellnhofer, and Wojciech Matusik. 2021. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 7849 (2021), 234–239.
[45]
Liang Shi, Beichen Li, and Wojciech Matusik. 2022. End-to-end learning of 3d phase-only holograms for holographic display. Light: Science & Applications 11, 1 (2022), 1–18.
[46]
Xin Yang, Ping Song, HongBo Zhang, and Qiong-Hua Wang. 2019. Full-color computer-generated holographic near-eye display based on white light illumination. Optics Express 27, 26 (2019), 38236–38249.
[47]
Alim Yolalmaz and Emre Yüce. 2022. Comprehensive deep learning model for 3D color holography. Scientific reports 12, 1 (2022), 1–9.
[48]
Ryosuke Yonesaka, Yonghee Lee, Peng Xia, Tatsuki Tahara, Yasuhiro Awatsuji, Kenzo Nishio, and Osamu Matoba. 2016. High dynamic range digital holography and its demonstration by off-axis configuration. IEEE Transactions on Industrial Informatics 12, 5 (2016), 1658–1663.
[49]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586–595.
[50]
Fangcheng Zhong, Akshay Jindal, Özgür Yöntem, Param Hanji, Simon Watt, and Rafal Mantiuk. 2021. Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Transactions on Graphics 40, 6 (2021), 241.
