“Chromablur: rendering chromatic eye aberration improves accommodation and realism” by Cholewiak, Love, Srinivasan, Ng and Banks
Conference:
Type(s):
Title:
- Chromablur: rendering chromatic eye aberration improves accommodation and realism
Session/Category Title: AR/VR
Presenter(s)/Author(s):
Abstract:
Computer-graphics engineers and vision scientists want to generate images that reproduce realistic depth-dependent blur. Current rendering algorithms take into account scene geometry, aperture size, and focal distance, and they produce photorealistic imagery as with a high-quality camera. But to create immersive experiences, rendering algorithms should aim instead for perceptual realism. In so doing, they should take into account the significant optical aberrations of the human eye. We developed a method that, by incorporating some of those aberrations, yields displayed images that produce retinal images much closer to the ones that occur in natural viewing. In particular, we create displayed images taking the eye’s chromatic aberration into account. This produces different chromatic effects in the retinal image for objects farther or nearer than current focus. We call the method ChromaBlur. We conducted two experiments that illustrate the benefits of ChromaBlur. One showed that accommodation (eye focusing) is driven quite effectively when ChromaBlur is used and that accommodation is not driven at all when conventional methods are used. The second showed that perceived depth and realism are greater with imagery created by ChromaBlur than in imagery created conventionally. ChromaBlur can be coupled with focus-adjustable lenses and gaze tracking to reproduce the natural relationship between accommodation and blur in HMDs and other immersive devices. It may thereby minimize the adverse effects of vergence-accommodation conflicts.
References:
1. Karan R Aggarwala, Ekaterina S Kruger, Steven Mathews, and Philip B Kruger. 1995. Spectral bandwidth and ocular accommodation. Journal of the Optical Society of America A 12, 3 (1995), 450–455. Cross Ref
2. Kurt Akeley, Simon J Watt, Ahna Reza Girshick, and Martin S Banks. 2004. A stereo display prototype with multiple focal distances. In ACM Transactions on Graphics (TOG), Vol. 23. ACM, 804–813.
3. Brian A Barsky. 2004. Vision-realistic rendering: simulation of the scanned foveal image from wavefront data of human subjects. In Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization. ACM, 73–81.
4. Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning 3, 1 (2011), 1–122.
5. Fergus W Campbell. 1960. Correlation of accommodation between the two eyes. Journal of the Optical Society of America 50, 7 (1960), 738–738. Cross Ref
6. Han Cheng, Justin K Barnett, Abhiram S Vilupuru, Jason D Marsack, Sanjeev Kasthurirangan, Raymond A Applegate, and Austin Roorda. 2004. A population study on changes in wave aberrations with accommodation. Journal of Vision 4, 4 (2004), 272–280. Cross Ref
7. Michael F Deering. 2005. A photon accurate model of the human eye. ACM Transactions on Graphics (TOG) 24, 3 (2005), 649–658.
8. S Kay Fisher, Kenneth J Ciuffreda, and Steven Hammer. 1987. Interocular equality of tonic accommodation and consensuality of accommodative hysteresis. Ophthalmic and Physiological Optics 7, 1 (1987), 17–20.
9. Horacio E Fortunato and Manuel M Oliveira. 2014. Fast high-quality non-blind deconvolution using sparse adaptive priors. The Visual Computer 30, 6–8 (2014), 661–671.
10. Robert T Held, Emily A Cooper, James F OfiBrien, and Martin S Banks. 2010. Using blur to affect perceived distance and size. ACM Transactions on Graphics 29, 2 (2010), 1–33.
11. David M Hoffman, Ahna R Girshick, Kurt Akeley, and Martin S Banks. 2008. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3 (2008), 1–30. Cross Ref
12. Miriam Hoyt. 2016. Standard (Stauton) Chess Set. (2016). Retrieved March 3, 2017 from http://www.blendswap.com/blends/view/86207
13. Xinda Hu and Hong Hua. 2015. Design and tolerance of a free-form optical system for an optical see-through multi-focal-plane display. Applied Optics 54, 33 (2015), 9990–9999. Cross Ref
14. Wenzel Jakob. 2010. Mitsuba renderer. (2010). Retrieved January 6, 2017 from http://www.mitsuba-renderer.org
15. Jay-Artist. 2012. Country-Kitchen Cycles. (2012). Retrieved January 6, 2017 from http://www.blendswap.com/blends/view/42851
16. Paul V Johnson, Jared AQ Parnell, Joohwan Kim, Christopher D Saunter, Gordon D Love, and Martin S Banks. 2016. Dynamic lens and monovision 3D displays to improve viewer comfort. Optics Express 24, 11 (2016), 11808–11827. Cross Ref
17. Masanori Kakimoto, Tomoaki Tatsukawa, Yukiteru Mukai, and Tomoyuki Nishita. 2007. Interactive simulation of the human eye depth of field and its correction by spectacle lenses. In Computer Graphics Forum, Vol. 26. Wiley Online Library, 627–636.
18. Craig Kolb, Don Mitchell, and Pat Hanrahan. 1995. A realistic camera model for computer graphics. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 317–324.
19. Robert Konrad, Emily A Cooper, and Gordon Wetzstein. 2016. Novel optical configurations for virtual reality: evaluating user preference and performance with focus-tunable and monovision near-eye displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1211–1220.
20. George-Alex Koulieris, Bee Bui, Martin S. Banks, and George Drettakis. 2017. Accommodation and Comfort in Head-Mounted Displays. ACM Transactions on Graphics (SIGGRAPH Conference Proceedings) 36, 4 (July 2017), 11. http://www-sop.inria.fr/reves/Basilic/2017/KBBD17
21. Philip B Kruger, Steven Mathews, Karan R Aggarwala, and Nivian Sanchez. 1993. Chromatic aberration and ocular focus: Fincham revisited. Vision Research 33, 10 (1993), 1397–1411. Cross Ref
22. Marc Lambooij, Marten Fortuin, Ingrid Heynderickx, and Wijnand IJsselsteijn. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of Imaging Science and Technology 53, 3 (2009), 30201-1-30201-14. Cross Ref
23. Tao Liu and Larry N Thibos. 2017. Variation of axial and oblique astigmatism with accommodation across the visual field. Journal of Vision 17, 3 (2017), 1–23. Cross Ref
24. Norberto López-Gil, Frances J Rucker, Lawrence R Stark, Mustanser Badar, Theodore Borgovan, Sean Burke, and Philip B Kruger. 2007. Effect of third-order aberrations on dynamic accommodation. Vision Research 47, 6 (2007), 755–765. Cross Ref
25. Gordon D Love, David M Hoffman, Philip JW Hands, James Gao, Andrew K Kirby, and Martin S Banks. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express 17, 18 (2009), 15716–15725. Cross Ref
26. Kevin J MacKenzie, David M Hoffman, and Simon J Watt. 2010. Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. Journal of Vision 10, 8 (2010), 1–20. Cross Ref
27. Guido Maiello, Manuela Chessa, Fabio Solari, and Peter J Bex. 2014. Simulated disparity and peripheral blur interact during binocular fusion. Journal of Vision 14, 8 (2014), 1–14. Cross Ref
28. David H Marimont and Brian A Wandell. 1994. Matching color images: the effects of axial chromatic aberration. Journal of the Optical Society of America A 11, 12 (1994), 3113–3122. Cross Ref
29. George Mather and David RR Smith. 2002. Blur discrimination and its relation to blur-mediated depth perception. Perception 31, 10 (2002), 1211–1219. Cross Ref
30. Michael Mauderer, Simone Conte, Miguel A Nacenta, and Dhanraj Vishwanath. 2014. Depth perception with gaze-contingent depth of field. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 217–226.
31. Sina Mostafawy, Omid Kermani, and Holger Lubatschowski. 1997. Virtual eye: retinal image visualization of the human eye. IEEE Computer Graphics and Applications 17, 1 (1997), 8–12.
32. Masashi Nakajima, Takahiro Hiraoka, Yoko Hirohara, Tetsuro Oshika, and Toshifumi Mihashi. 2015. Verification of the lack of correlation between age and longitudinal chromatic aberrations of the human eye from the visible to the infrared. Biomedical Optics Express 6, 7 (2015), 2676–2694. Cross Ref
33. Rafael Navarro, David R Williams, and Pablo Artal. 1993. Modulation transfer of the human eye as a function of retinal eccentricity. Journal of the Optical Society of America A 10, 2 (1993), 201–212. Cross Ref
34. Ren Ng and Pat Hanrahan. 2006. Digital correction of lens aberrations in light field photography. In Proceedings of the International Optical Design Conference. Optical Society of America. Cross Ref
35. Vincent A Nguyen, Ian P Howard, and Robert S Allison. 2005. Detection of the depth order of defocused images. Vision Research 45, 8 (2005), 1003–1011. Cross Ref
36. Matthias Nießner, Roman Sturm, and Günther Greiner. 2012. Real-time simulation and visualization of human vision through eyeglasses on the GPU. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 195–202.
37. D Alfred Owens. 1979. The Mandelbaum effect: Evidence for an accommodative bias toward intermediate viewing distances. Journal of the Optical Society of America 69, 5 (1979), 646–652. Cross Ref
38. Steven G Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, and others. 2010. Optix: a general purpose ray tracing engine. In ACM Transactions on Graphics (TOG), Vol. 29. ACM, 66.
39. Henry B Peters. 1961. The relationship between refractive error and visual acuity at three age levels. Optometry & Vision Science 38, 4 (1961), 194–198. Cross Ref
40. Matt Pharr and Greg Humphreys. 2014. Physically based rendering: From theory to implementation (3rd ed.). Morgan Kaufmann.
41. James Polans, BartJaeken, Ryan P McNabb, Pablo Artal, and Joseph A Izatt. 2015. Widefield optical model of the human eye with asymmetrically tilted and decentered lens that reproduces measured ocular aberrations. Optica 2, 2 (2015), 124–134. Cross Ref
42. Jason Porter, Antonio Guirao, Ian G Cox, and David R Williams. 2001. Monochromatic aberrations of the human eye in a large population. Journal of the Optical Society of America A 18, 8 (2001), 1793–1803. Cross Ref
43. Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. ACM SIGPLAN Notices 48, 6 (2013), 519–530.
44. Maurice C Rynders, Rafael Navarro, and M Angeles Losada. 1998. Objective measurement of the off-axis longitudinal chromatic aberration in the human eye. Vision Research 38, 4 (1998), 513–522. Cross Ref
45. Thomas O Salmon and Corina van de Pol. 2006. Normal-eye Zernike coefficients and root-mean-square wavefront errors. Journal of Cataract & Refractive Surgery 32, 12 (2006), 2064–2074. Cross Ref
46. Clifton M Schor. 1992. A dynamic model of cross-coupling between accommodation and convergence: simulations of step and frequency responses. Optometry & Vision Science 69, 4 (1992), 258–269. Cross Ref
47. Lindsay T Sharpe, Andrew Stockman, Herbert Jägle, and Jeremy Nathans. 1999. Opsin genes, cone photopigments, color vision, and color blindness. Color vision: From genes to perception (1999), 3–51.
48. Takashi Shibata, Joohwan Kim, David M Hoffman, and Martin S Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of Vision 11, 8 (2011), 1–29. Cross Ref
49. KH Spring and Walter S. Stiles. 1948. Variation of pupil size with change in the angle at which the light stimulus strikes the retina. The British Journal of Ophthalmology 32, 6 (1948), 340–346. Cross Ref
50. Benjamin Steinert, Holger Dammertz, Johannes Hanika, and Hendrik PA Lensch. 2011. General spectral camera lens simulation. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 1643–1654.
51. Mahesh M Subedar and Lina J Karam. 2016. 3D Blur Discrimination. ACM Transactions on Applied Perception (TAP) 13, 3 (2016), 1–13.
52. Ning Tang and ShuangJiu Xiao. 2015. Real-time human vision rendering using blur distribution function. In Proceedings of the 14th ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry. ACM, 39–42.
53. Larry N Thibos, Ming Ye, Xiaoxiao Zhang, and Arthur Bradley. 1992. The chromatic eye: a new reduced-eye model of ocular chromatic aberration in humans. Applied Optics 31, 19 (1992), 3594–3600. Cross Ref
54. Dhanraj Vishwanath and Erik Blaser. 2010. Retinal blur and the perception of egocentric distance. Journal of Vision 10, 10 (2010), 1–16. Cross Ref
55. Ingo Wald, Sven Woop, Carsten Benthin, Gregory S Johnson, and Manfred Ernst. 2014. Embree: a kernel framework for efficient CPU ray tracing. ACM Transactions on Graphics (TOG) 33, 4 (2014), 143.
56. Andrew B Watson and Albert J Ahumada. 2011. Blur clarified: A review and synthesis of blur discrimination. Journal of Vision 11, 5 (2011), 1–23. Cross Ref
57. Simon J Watt, Kurt Akeley, Marc O Ernst, and Martin S Banks. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10 (2005), 834–862. Cross Ref
58. James S Wolffsohn, Bernard Gilmartin, Edward AH Mallen, and Sei-ichi Tsujimura. 2001. Continuous recording of accommodation and pupil size using the Shin-Nippon SRW-5000 autorefractor. Ophthalmic and Physiological Optics 21, 2 (2001), 108–113. Cross Ref
59. James S Wolffsohn, Clare O’Donnell, W Neil Charman, and Bernard Gilmartin. 2004. Simultaneous continuous recording of accommodation and pupil size using the modified Shin-Nippon SRW-5000 autorefractor. Ophthalmic and Physiological Optics 24, 2 (2004), 142–147. Cross Ref
60. Jiaze Wu, Changwen Zheng, Xiaohui Hu, and Fanjiang Xu. 2011. Realistic simulation of peripheral vision using an aspherical eye model. Eurographics (2011).
61. Marina Zannoli, Gordon D Love, Rahul Narain, and Martin S Banks. 2016. Blur and the perception of depth at occlusions. Journal of Vision 16, 6 (2016), 1–25. Cross Ref
62. Tingting Zhang, Louise OfiHare, Paul B Hibbard, Harold T Nefs, and Ingrid Heynderickx. 2015. Depth of field affects perceived depth in stereographs. ACM Transactions on Applied Perception (TAP) 11, 4 (2015), 18.
