“Holographic Parallax: 3D Holographic Near-eye Displays with Parallax Cues”
Conference:
Type(s):
Title:
- Holographic Parallax: 3D Holographic Near-eye Displays with Parallax Cues
Collaborator(s):
Description:
Recent advancements in holographic displays enable the creation of 4D light field holograms featuring view-dependent visual effects. Our holographic display prototype delivers an experience of state-of-the-art 4D light field holograms and highlights the significance of parallax cues in enhancing perceptual realism of 3D scenes.
References:
[1]
A. T. Bahill, M. R. Clark, and L. Stark. The main sequence, a tool for studying human eye movements. Mathematical biosciences, 24(3–4):191–204, 1975.
[2]
R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychological Review, 113(4):700, 2006.
[3]
O. Bryngdahl and A. Lohmann. Single-sideband holography. JOSA, 58(5):620–624, 1968.
[4]
P. Chakravarthula, E. Tseng, T. Srivastava, H. Fuchs, and F. Heide. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays. ACM Transactions on Graphics (TOG), 39(6):1–18, 2020.
[5]
P. Chakravarthula, S.-H. Baek, F. Schiffers, E. Tseng, G. Kuo, A. Maimone, N. Matsuda, O. Cossairt, D. Lanman, and F. Heide. Pupil-aware holography. ACM Transactions on Graphics (TOG), 41(6):1–15, 2022.
[6]
J.-H. R. Chang, A. Levin, B. V. Kumar, and A. C. Sankaranarayanan. Towards occlusion-aware multifocal displays. ACM Transactions on Graphics (TOG), 39(4):68–1, 2020.
[7]
C. Chen, B. Lee, N.-N. Li, M. Chae, D. Wang, Q.-H. Wang, and B. Lee. Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function. Opt. Express, 29(10):15089–15103, 2021.
[8]
S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein. Neural 3d holography: Learning accurate wave propagation models for 3d holographic virtual and augmented reality displays. ACM Trans. Graph. (SIGGRAPH Asia), 2021.
[9]
S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’Toole, and G. Wetzstein. Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–9, 2022.
[10]
V. R. Curtis, N. W. Caira, J. Xu, A. G. Sata, and N. C. P?gard. Dcgh: dynamic computer generated holography for speckle-free, high fidelity 3d displays. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pages 1–9. IEEE, 2021.
[11]
J. E. Cutting and P. M. Vishton. Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In Perception of space and motion, pages 69–117. Elsevier, 1995.
[12]
S. De Groot and J. Gebhard. Pupil size as determined by adapting luminance. JOSA, 42 (7):492–495, 1952.
[13]
A. Duane. Normal values of the accommodation at all ages. Journal of the American Medical Association, 59(12):1010–1013, 1912.
[14]
J. W. Goodman. Introduction to Fourier optics. Roberts and Company Publishers, 2005.
[15]
P. Guan, O. Mercier, M. Shvartsman, and D. Lanman. Perceptual requirements for eye-tracked distortion correction in vr. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1–8, 2022.
[16]
B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder. Foveated 3d graphics. ACM transactions on Graphics (tOG), 31(6):1–10, 2012.
[17]
D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. Journal of vision, 8(3): 33–33, 2008.
[18]
C. Jang, K. Bang, G. Li, and B. Lee. Holographic near-eye display with expanded eye-box. ACM Trans. Graph., 37(6), dec 2018.
[19]
C. Jang, K. Bang, M. Chae, B. Lee, and D. Lanman. Waveguide holography for 3d augmented reality glasses. Nature Communications, 15(1):66, 2024.
[20]
E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144, 2016.
[21]
H. Kang, E. Stoykova, and H. Yoshikawa. Fast phase-added stereogram algorithm for generation of photorealistic 3d content. Applied optics, 55(3):A135–A143, 2016.
[22]
K. Kavakl?, Y. Itoh, H. Urey, and K. Ak?it. Realistic defocus blur for multiplane computergenerated holography. In 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pages 418–426. IEEE, 2023.
[23]
D. Kim, S.-W. Nam, K. Bang, B. Lee, S. Lee, Y. Jeong, J.-M. Seo, and B. Lee. Vision-correcting holographic display: evaluation of aberration correcting hologram. Biomedical Optics Express, 12(8):5179–5195, 2021.
[24]
D. Kim, S.-W. Nam, B. Lee, J.-M. Seo, and B. Lee. Accommodative holography: improving accommodation response for perceptually realistic holographic displays. ACM Transactions on Graphics (TOG), 41(4):1–15, 2022.
[25]
V. Kiran Adhikarla, M. Vinkler, D. Sumin, R. K. Mantiuk, K. Myszkowski, H.-P. Seidel, and P. Didyk. Towards a quality metric for dense light fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 58–67, 2017.
[26]
R. Konrad, A. Angelopoulos, and G. Wetzstein. Gaze-contingent ocular parallax rendering for virtual reality. ACM Transactions on Graphics (TOG), 39(2):1–12, 2020.
[27]
G. Kuo, L. Waller, R. Ng, and A. Maimone. High resolution ?tendue expansion for holographic displays. ACM Transactions on Graphics (TOG), 39(4):66–1, 2020.
[28]
B. Lee, D. Yoo, J. Jeong, S. Lee, D. Lee, and B. Lee. Wide-angle speckleless dmd holographic display using structured illumination with temporal multiplexing. Optics Letters, 45(8):2148–2151, 2020.
[29]
B. Lee, D. Kim, S. Lee, C. Chen, and B. Lee. High-contrast, speckle-free, true 3d holography via binary cgh optimization. Scientific reports, 12(1):2811, 2022.
[30]
A. W. Lohmann and D. Paris. Binary fraunhofer holograms, generated by computer. Applied optics, 6(10):1739–1748, 1967.
[31]
A. Maimone, A. Georgiou, and J. S. Kollin. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. (SIGGRAPH), 36(4):85, 2017.
[32]
R. K. Mantiuk, G. Denes, A. Chapiro, A. Kaplanyan, G. Rufo, R. Bachy, T. Lian, and A. Patney. Fovvideovdp: A visible difference predictor for wide field-of-view video. ACM Transactions on Graphics (TOG), 40(4):1–19, 2021.
[33]
J. March, A. Krishnan, S. Watt, M. Wernikowski, H. Gao, A. ?. Y?ntem, and R. Mantiuk. Impact of correct and simulated focus cues on perceived realism. In SIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022.
[34]
K. Matsushima and S. Nakahara. Extremely high-definition full-parallax computergenerated hologram created by the polygon-based method. Applied optics, 48(34): H54–H63, 2009.
[35]
A. Mehrfard, J. Fotouhi, G. Taylor, T. Forster, N. Navab, and B. Fuerst. A comparative analysis of virtual reality head-mounted display systems. arXiv preprint arXiv:1912.02913, 2019.
[36]
O. Mercier, Y. Sulai, K. Mackenzie, M. Zannoli, J. Hillis, D. Nowrouzezahrai, and D. Lanman. Fast gaze-contingent optimal decompositions for multifocal displays. ACM Transactions on Graphics (TOG), 36(6):1–15, 2017.
[37]
S. Monin, A. C. Sankaranarayanan, and A. Levin. Analyzing phase masks for wide ?tendue holographic displays. In 2022 IEEE International Conference on Computational Photography (ICCP), pages 1–12. IEEE, 2022a.
[38]
S. Monin, A. C. Sankaranarayanan, and A. Levin. Exponentially-wide etendue displays using a tilting cascade. In 2022 IEEE International Conference on Computational Photography (ICCP), pages 1–12. IEEE, 2022b.
[39]
J. J. Naji and T. C. Freeman. Perceiving depth order during pursuit eye movement. Vision research, 44(26):3025–3034, 2004.
[40]
S.-W. Nam, Y. Kim, D. Kim, and Y. Jeong. Depolarized holography with polarization-multiplexing metasurface. ACM Transactions on Graphics (TOG), 42(6):1–16, 2023.
[41]
P. Napieralski and F. Rynkiewicz. Modeling human pupil dilation to decouple the pupillary light reflex. Open Physics, 17(1):458–467, 2019.
[42]
M. Nawrot. Eye movements provide the extra-retinal signal required for the perception of depth from motion parallax. Vision research, 43(14):1553–1562, 2003.
[43]
R. Ng, M. Levoy, M. Br?dif, G. Duval, M. Horowitz, and P. Hanrahan. Light field photography with a hand-held plenoptic camera. PhD thesis, Stanford University, 2005.
[44]
N. Padmanaban, Y. Peng, and G. Wetzstein. Holographic near-eye displays based on overlap-add stereograms. ACM Trans. Graph., 38(6), 2019a.
[45]
N. Padmanaban, Y. Peng, and G. Wetzstein. Holographic near-eye displays based on overlap-add stereograms. ACM Transactions on Graphics (TOG), 38(6):1–13, 2019b.
[46]
J. Park, K. Lee, and Y. Park. Ultrathin wide-angle large-area digital 3d holographic display using a non-periodic photon sieve. Nature communications, 10(1):1304, 2019.
[47]
J.-H. Park. Recent progress in computer-generated holography for three-dimensional scenes. Journal of Information Display, 18(1):1–12, 2017.
[48]
J.-H. Park. Efficient calculation scheme for high pixel resolution non-hogel-based computer generated hologram from light field. Optics Express, 28(5):6663–6683, 2020.
[49]
J.-H. Park and M. Askari. Non-hogel-based computer generated hologram from light field using complex field recovery technique from wigner distribution function. Optics express, 27(3):2562–2574, 2019.
[50]
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[51]
Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein. Neural holography with camera-in-the-loop training. ACM Trans. Graph., 39(6):1–14, 2020.
[52]
M. Perez-Ortiz and R. K. Mantiuk. A practical guide and software for analysing pairwise comparison experiments. arXiv preprint arXiv:1712.03686, 2017.
[53]
K. Ratnam, R. Konrad, D. Lanman, and M. Zannoli. Retinal image quality in near-eye pupil-steered systems. Optics Express, 27(26):38289–38311, 2019.
[54]
L. Shi, F.-C. Huang, W. Lopes, W. Matusik, and D. Luebke. Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3d computer graphics. ACM Trans. Graph., 36(6), 2017.
[55]
L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik. Towards real-time photorealistic 3d holography with deep neural networks. Nature, 591(7849):234–239, 2021.
[56]
L. Shi, B. Li, and W. Matusik. End-to-end learning of 3d phase-only holograms for holographic display. Light: Science & Applications, 11(1):247, 2022.
[57]
A. Symeonidou, D. Blinder, A. Munteanu, and P. Schelkens. Computer-generated holograms by multiple wavefront recording plane method with occlusion culling. Optics express, 23(17):22149–22161, 2015.
[58]
F. Wang, T. Ito, and T. Shimobaba. High-speed rendering pipeline for polygon-based holograms. Photonics Research, 11(2):313–328, 2023.
[59]
A. B. Watson and D. G. Pelli. Quest: A bayesian adaptive psychometric method. Perception & psychophysics, 33(2):113–120, 1983.
[60]
G. Westheimer. Directional sensitivity of the retina: 75 years of stiles-crawford effect. Proceedings of the Royal Society B: Biological Sciences, 275(1653):2777–2786, 2008.
[61]
G. Wetzstein and D. Lanman. Factored displays: improving resolution, dynamic range, color reproduction, and light field characteristics with advanced signal processing. IEEE Signal Processing Magazine, 33(5):119–129, 2016.
[62]
D. Yang, W. Seo, H. Yu, S. I. Kim, B. Shin, C.-K. Lee, S. Moon, J. An, J.-Y. Hong, G. Sung, et al. Diffraction-engineered holography: Beyond the depth representation limit of holographic displays. Nature Communications, 13(1):6012, 2022.
[63]
D. Yoo, Y. Jo, S.-W. Nam, C. Chen, and B. Lee. Optimization of computer-generated holograms featuring phase randomness control. Optics Letters, 46(19):4769–4772, 2021.
[64]
H. Zhang, Y. Zhao, L. Cao, and G. Jin. Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues. Optics express, 23(4):3901–3913, 2015.
[65]
H. Zhang, L. Cao, and G. Jin. Computer-generated hologram with occlusion effect using layer-based processing. Applied optics, 56(13), 2017.
[66]
Z. Zhang and M. Levoy. Wigner distributions and how they relate to the light field. In 2009 IEEE International Conference on Computational Photography (ICCP), pages 1–10. IEEE, 2009.
[67]
F. Zhong, A. Jindal, ?. Y?ntem, P. Hanji, S. Watt, and R. Mantiuk. Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Transactions on Graphics, 40(6):241, 2021.
