“Accommodation-invariant computational near-eye displays”

  • ©Robert K. Konrad, Nitish Padmanaban, Keenan Molner, Emily A. Cooper, and Gordon Wetzstein




    Accommodation-invariant computational near-eye displays


Session Title: Time to Focus



    Although emerging virtual and augmented reality (VR/AR) systems can produce highly immersive experiences, they can also cause visual discomfort, eyestrain, and nausea. One of the sources of these symptoms is a mismatch between vergence and focus cues. In current VR/AR near-eye displays, a stereoscopic image pair drives the vergence state of the human visual system to arbitrary distances, but the accommodation, or focus, state of the eyes is optically driven towards a fixed distance. In this work, we introduce a new display technology, dubbed accommodation-invariant (AI) near-eye displays, to improve the consistency of depth cues in near-eye displays. Rather than producing correct focus cues, AI displays are optically engineered to produce visual stimuli that are invariant to the accommodation state of the eye. The accommodation system can then be driven by stereoscopic cues, and the mismatch between vergence and accommodation state of the eyes is significantly reduced. We validate the principle of operation of AI displays using a prototype display that allows for the accommodation state of users to be measured while they view visual stimuli using multiple different display modes.


    1. Kurt Akeley, Simon J. Watt, Ahna Reza Girshick, and Martin S. Banks. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) 23, 3 (2004), 804–813. Google ScholarDigital Library
    2. Rosa Braga-Mele, David Chang, Steven Dewey, Gary Foster, Bonnie An Henderson, Warren Hill, Richard Hoffman, Brian Little, Nick Mamalis, Thomas Oetting, Donald Serafano, Audrey Talley-Rostov, Abhay Vasavada, and Sonia Yoo. 2014. Multifocal intraocular lenses: Relative indications and contraindications for implantation. J. Cataract Refract. Surg. 40, 2 (2014), 313–322. Google ScholarCross Ref
    3. F. W. Campbell. 1957. The depth of field of the human eye. Opt. Acta (Lond) 4 (1957), 157–164. Google ScholarCross Ref
    4. F. W. Campbell and G. Westheimer. 1960. Dynamics of accommodation responses of the human eye. J. Physiol. 151 (1960), 285–295. Google ScholarCross Ref
    5. Oliver Cossairt and Shree K. Nayar. 2010. Spectral focal sweep: Extended depth of field from chromatic aberrations. In Proc. ICCP. Google ScholarCross Ref
    6. Oliver Cossairt, Changyin Zhou, and Shree K. Nayar. 2010. Diffusion coded photography for extended depth of field. ACM Trans. Graph. (SIGGRAPH) 29, 4 (2010), 31:1–31:10.Google ScholarDigital Library
    7. James E. Cutting and Peter M. Vishton. 1995. Perceiving layout and knowing distances: The interaction, relative potency, and contextual use of different information about depth. W. Epstein and S. Rogers (Eds.), Percept. Space Motion (1995), 69–117.Google Scholar
    8. S. G. de Groot and J. W. Gebhard. 1952. Pupil size as determined by adapting luminance. J. Opt. Soc. Am. 42, 7 (1952), 492–495. Google ScholarCross Ref
    9. Eugene Dolgoff. 1997. Real-depth imaging: a new 3D imaging technology with inexpensive direct-view (no glasses) video and other applications. Proc. SPIE 3012 (1997), 282–288. Google ScholarCross Ref
    10. Edward R. Dowski and W. Thomas Cathey. 1995. Extended depth of field through wave-front coding. Appl. Opt. 34, 11 (1995), 1859–66. Google ScholarCross Ref
    11. E. F. Fincham. 1951. The accommodation reflex and its stimulus. Br. J. Ophthalmol. 35 (1951), 381–393. Google ScholarCross Ref
    12. E. F. Fincham and J. Walton. 1957. The reciprocal actions of accommodation and convergence. J. Physiol. 137, 3 (1957), 488–508. Google ScholarCross Ref
    13. Max Grosse, Gordon Wetzstein, Anselm Grundhöfer, and Oliver Bimber. 2010. Coded aperture projection. ACM Trans. Graph. 29, 3 (2010), 22:1–22:12.Google ScholarDigital Library
    14. G. Häusler. 1972. A method to increase the depth of focus by two step image processing. Opt. Commun. 6, 1 (1972), 38–42. Google ScholarCross Ref
    15. Gordon Heron, W. Neil Charman, and Clifton M. Schor. 2001. Age changes in the interactions between the accommodation and vergence systems. Optom. Vis. Sci. 10, 78 (2001), 754–62. Google ScholarCross Ref
    16. David M. Hoffman, Ahna Reza Girshick, Kurt Akeley, and Martin S. Banks. 2008. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. J. Vis. 8, 3 (2008). Google ScholarCross Ref
    17. Xinda Hu and Hong Hua. 2014. Design and assessment of a depth-fused multi-focal-plane display prototype. J. Disp. Technol. 10, 4 (2014), 308–316. Google ScholarCross Ref
    18. Hong Hua and Bahram Javidi. 2014. A 3D integral imaging optical see-through head-mounted display. Opt. Express 22, 11 (2014), 13484–13491. Google ScholarCross Ref
    19. Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein. 2015. The light field stereoscope: Immersive computer graphics via factored near-eye light field display with focus cues. ACM Trans. Graph. (SIGGRAPH) 34, 4 (2015). Google ScholarDigital Library
    20. Daisuke Iwai, Shoichiro Mihara, and Kosuke Sato. 2015. Extended depth-of-field projector by fast focal sweep projection. In Proc. VR. Google ScholarCross Ref
    21. Paul V. Johnson, Jared A. Q. Parnell, Joohwan Kim, Christopher D. Saunter, Gordon D. Love, and Martin S. Banks. 2016. Dynamic lens and monovision 3D displays to improve viewer comfort. Opt. Express 24 (2016), 11808–11827. Issue 11.Google ScholarCross Ref
    22. Robert Konrad, Emily A. Cooper, and Gordon Wetzstein. 2016. Novel optical configurations for virtual reality: Evaluating user preference and performance with focus-tunable and monovision near-eye displays. In Proc. SIGCHI. Google ScholarDigital Library
    23. Frank L. Kooi and Alexander Toet. 2004. Visual comfort of binocular and 3D displays. Displays 25 (2004), 99–108. Google ScholarCross Ref
    24. Gregory Kramida. 2015. Resolving the vergence-accommodation conflict in head-mounted displays. IEEE TVCG 22 (2015), 1912–1931. Issue 7.Google Scholar
    25. Marc Lambooij, Marten Fortuin, Ingrid Heynderickx, and Wijnand IJsselsteijn. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. J. Imaging Sci. Technol. 53, 3 (2009). Google ScholarCross Ref
    26. Douglas Lanman and David Luebke. 2013. Near-eye light field displays. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6 (2013), 220:1–220:10.Google Scholar
    27. Anat Levin, Samuel W. Hasinoff, Paul Green, Frédo Durand, and William T. Freeman. 2009. 4D frequency analysis of computational cameras for depth of field extension. ACM Trans. Graph. (SIGGRAPH) 28, 3 (2009), 97:1–97:14.Google ScholarDigital Library
    28. Sheng Liu, Dewen Cheng, and Hong Hua. 2008. An optical see-through head mounted display with addressable focal planes. In Proc. ISMAR. 33–42.Google Scholar
    29. Patrick Llull, Noah Bedard, Wanmin Wu, Ivana Tosic, Kathrin Berkner, and Nikhil Balram. 2015. Design and optimization of a near-eye multifocal display system for augmented reality. In OSA Imaging Appl. Opt. Google ScholarCross Ref
    30. Gordon D. Love, David M. Hoffman, Philip J. W. Hands, James Gao, Andrew K. Kirby, and Martin S. Banks. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Opt. Express 17, 18 (2009), 15716–25. Google ScholarCross Ref
    31. Kevin J. MacKenzie, David M. Hoffman, and Simon J. Watt. 2010. Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. J. Vis. 10, 8 (2010). Google ScholarCross Ref
    32. Susana Marcos, Esther Moreno, and Rafael Navarro. 1999. The depth-of-field of the human eye from objective and subjective measurements. Vision Res. 39, 12 (1999), 2039–2049. Google ScholarCross Ref
    33. Daniel Miau, Oliver Cossairt, and Shree K. Nayar. 2013. Focal sweep videography with deformable optics. In Proc. ICCP. 1–8. Google ScholarCross Ref
    34. Hajime Nagahara, Sujit Kuthirummal, Changyin Zhou, and Shree K. Nayar. 2008. Flexible depth of field photography. In Proc. ECCV. Google ScholarDigital Library
    35. Rahul Narain, Rachel A. Albert, Abdullah Bulbul, Gregory J. Ward, Martin S. Banks, and James F. O’Brien. 2015. Optimal presentation of imagery with focus cues on multi-plane displays. ACM Trans. Graph. (SIGGRAPH) 34, 4 (2015). Google ScholarDigital Library
    36. Nitish Padmanaban, Robert Konrad, Tal Stramer, Emily A. Cooper, and Gordon Wetzstein. 2017. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proc. Natl. Acad. Sci. U.S.A. 114 (2017), 2183–2188. Issue 9.Google ScholarCross Ref
    37. Harris Ripps, Newton B. Chin, Irwin M. Siegel, and Goodwin M. Breinin. 1962. The effect of pupil size on accommodation, convergence, and the AC/A ratio. Invest. Ophthalmol. 1 (1962), 127–135.Google Scholar
    38. Jannick P. Rolland, Myron W. Krueger, and Alexei Goon. 2000. Multifocal planes head-mounted displays. Appl. Opt. 39, 19 (2000), 3209–3215. Google ScholarCross Ref
    39. Clifton M. Schor. 1992. A dynamic model of cross-coupling between accommodation and convergence: simulations of step and frequency responses. Optom. Vis. Sci. 69, 4 (1992), 258–269. Google ScholarCross Ref
    40. Brian T. Schowengerdt and Eric J. Seibel. 2006. True 3-D scanned voxel displays using single or multiple light sources. J. SID 14, 2 (2006), 135–143. Google ScholarCross Ref
    41. Takashi Shibata, Joohwan Kim, David M. Hoffman, and Martin S. Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. J. Vis. 11, 8 (2011), 11. Google ScholarCross Ref
    42. T. Sugihara and T. Miyasato. 1998. 32.4: A lightweight 3-D HMD with accommodative compensation. SID Digest 29, 1 (1998), 927–930. Google ScholarCross Ref
    43. Laura E. Sweeney, Dirk Seidel, Mhairi Day, and Lyle S. Gray. 2014. Quantifying interactions between accommodation and vergence in a binocularly normal population. Vision Res. 105 (2014), 121–129. Google ScholarCross Ref
    44. F. M. Toates. 1972. Accommodation function of the human eye. Physiol. Rev. 52 (1972), 828–863.Google ScholarCross Ref
    45. Tracy K. Tsuetaki and Clifton M. Schor. 1987. Clinical method for measuring adaptation of tonic accommodation and vergence accommodation. Am. J. Optom. Physiol. Opt. 64, 6 (1987), 437–449. Google ScholarCross Ref
    46. Marc von Waldkirch, Paul Lukowicz, and Gerhard Tröster. 2004. Multiple imaging technique for extending depth of focus in retinal displays. Opt. Express 12, 25 (2004). Google ScholarCross Ref
    47. Marc von Waldkirch, Paul Lukowicz, and Gerhard Tröster. 2005. Oscillating fluid lens in coherent retinal projection displays for extending depth of focus. Optics Communications 253, 25 (2005). Google ScholarCross Ref
    48. P. A. Ward and W. N. Charman. 1987. On the use of small artificial pupils to open-loop the accommodation system. Ophthalmic Physiol. Opt. 7, 2 (1987), 191–193. Google ScholarCross Ref
    49. Gerald Westheimer. 1966. The Maxwellian view. Vision Res. 6 (1966), 669–682. Google ScholarCross Ref
    50. G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar. 2012. Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 4 (2012), 1–11. Google ScholarDigital Library
    51. Zhongsheng Zhai, Shanting Ding, QingHua Lv, Xuanze Wang, and Yuning Zhong. 2009. Extended depth of field through an axicon. J. Mod. Opt. 56, 11 (2009), 1304–1308. Google ScholarCross Ref

ACM Digital Library Publication: