“Dark stereo: improving depth perception under low luminance” by Wolski, Zhong, Myszkowski and Mantiuk

It is often desirable or unavoidable to display Virtual Reality (VR) or stereoscopic content at low brightness. For example, a dimmer display reduces the flicker artefacts that are introduced by low-persistence VR headsets. It also saves power, prolongs battery life, and reduces the cost of a display or projection system. Additionally, stereo movies are usually displayed at relatively low luminance due to polarization filters or other optical elements necessary to separate two views. However, the binocular depth cues become less reliable at low luminance. In this paper, we propose a model of stereo constancy that predicts the precision of binocular depth cues for a given contrast and luminance. We use the model to design a novel contrast enhancement algorithm that compensates for the deteriorated depth perception to deliver good-quality stereoscopic images even for displays of very low brightness.

1. Reynold Bailey, Cindy Grimm, Christopher Davoli, and Richard Abrams. 2007. The effect of object color on depth ordering. (2007).Google Scholar
2. J. L. Barbur and A. Stockman. 2010. Photopic, mesopic and scotopic vision and changes in visual performance. Encyclopedia of the Eye 3 (2010), 323–331.Google ScholarCross Ref
3. P. Burt and E. Adelson. 1983. The Laplacian Pyramid as a Compact Image Code. IEEE Transactions on Communications 31, 4 (1983), 532–540.Google ScholarCross Ref
4. Dingcai Cao, Joel Pokorny, Vivianne C Smith, and Andrew J Zele. 2008. Rod contributions to color perception: linear with rod contrast. Vision Research 48, 26 (nov 2008), 2586–92. Google ScholarCross Ref
5. Alexandre Chapiro, Robin Atkins, and Scott Daly. 2019. A Luminance-Aware Model of Judder Perception. ACM Transactions on Graphics 38, 5, Article 142 (2019), 10 pages. Google ScholarDigital Library
6. J. Cutting and P. Vishton. 1995. Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In Perception of Space and Motion (Handbook Of Perception And Cognition), W. Epstein and S. Rogers (Eds.). Academic Press, 69–117.Google Scholar
7. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A Perceptual Model for Disparity. ACM Transactions on Graphics, Article 96 (2011), 10 pages. Google ScholarDigital Library
8. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2012a. Apparent stereo: the Cornsweet illusion can enhance perceived depth. In Human Vision and Electronic Imaging XVII, Bernice E. Rogowitz, Thrasyvoulos N. Pappas, and Huib de Ridder (Eds.), Vol. 8291. International Society for Optics and Photonics, SPIE, 180 — 191. Google ScholarCross Ref
9. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, Hans-Peter Seidel, and Wojciech Matusik. 2012b. A Luminance-Contrast-Aware Disparity Model and Applications. ACM Transactions on Graphics 31, 6 (2012).Google ScholarDigital Library
10. A. Erickson, K. Kim, G. Bruder, and G. F. Welch. 2020. Effects of Dark Mode Graphics on Visual Acuity and Fatigue with Virtual Reality Head-Mounted Displays. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 434–442.Google Scholar
11. John P Frisby and John E W Mayhew. 1978. Contrast Sensitivity Function for Stereopsis. Perception 7, 4 (1978), 423–429. arXiv:https://doi.org/10.1068/p070423 PMID: 704272. Google ScholarCross Ref
12. Chenyang Fu, Changjun Li, Guihua Cui, M. Ronnier Luo, Robert W. G. Hunt, and Michael R. Pointer. 2012. An investigation of colour appearance for unrelated colours under photopic and mesopic vision. Color Research & Application 37, 4 (2012), 238–254. Google ScholarCross Ref
13. B Y M A Georgeson and G D Sullivan. 1975. Contrast constancy: deblurring in human vision by spatial frequency channels. The Journal of Physiology 252, 3 (1975), 627–656.Google ScholarCross Ref
14. David M. Hoffman and Grace Lee. 2019. Temporal Requirements for VR Displays to Create a More Comfortable and Immersive Visual Experience. Information Display 35, 2 (mar 2019), 9–39. Google ScholarCross Ref
15. Jochen Jacobs, Xi Wang, and Marc Alexa. 2019. Keep It Simple: Depth-Based Dynamic Adjustment of Rendering for Head-Mounted Displays Decreases Visual Comfort. ACM Trans. Appl. Percept. 16, 3, Article 16 (2019), 16 pages.Google ScholarDigital Library
16. Petr Kellnhofer, Piotr Didyk, Karol Myszkowski, Mohamed M. Hefeeda, Hans-Peter Seidel, and Wojciech Matusik. 2016. GazeStereo3D: Seamless Disparity Manipulations. ACM Transactions on Graphics 35, 4 (2016). Google ScholarDigital Library
17. Petr Kellnhofer, Tobias Ritschel, Peter Vangorp, Karol Myszkowski, and Hans-Peter Seidel. 2014. Stereo Day-for-Night: Retargeting Disparity for Scotopic Vision. ACM Trans. Appl. Percept. 11, 3 (2014).Google ScholarDigital Library
18. Min H Kim, Tim Weyrich, and Jan Kautz. 2009. Modeling human color perception under extended luminance levels. In ACM SIGGRAPH 2009 papers. 1–9.Google ScholarDigital Library
19. John Krauskopf and Jason D Forte. 2002. Influence of chromaticity on vernier and stereo acuity. Journal of Vision 2, 9 (2002), 6–6.Google ScholarCross Ref
20. J.J. Kulikowski. 1976. Effective contrast constancy and linearity of contrast sensation. Vision Research 16, 12 (jan 1976), 1419–1431. Google ScholarCross Ref
21. Youngshin Kwak, Lindsay William MacDonald, and M. Ronnier Luo. 2003. Mesopic color appearance. In Human Vision and Electronic Imaging VIII, Bernice E. Rogowitz and Thrasyvoulos N. Pappas (Eds.), Vol. 5007. International Society for Optics and Photonics, SPIE, 161 — 169. Google ScholarCross Ref
22. Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear Disparity Mapping for Stereoscopic 3D. ACM Transactions on Graphics 29, 4, Article 75 (2010), 10 pages. Google ScholarDigital Library
23. Margaret S. Livingstone and David H. Hubel. 1994. Stereopsis and positional acuity under dark adaptation. Vision Research 34, 6 (1994), 799–802.Google ScholarCross Ref
24. Thomas Luft, Carsten Colditz, and Oliver Deussen. 2006. Image Enhancement by Unsharp Masking the Depth Buffer. ACM Transactions on Graphics 25, 3 (2006), 1206–1213. Google ScholarDigital Library
25. Ming Ronnier Luo and Changjun Li. 2013. CIECAM02 and Its Recent Developments. Springer New York, New York, NY, 19–58. Google ScholarCross Ref
26. Rafał Mantiuk, Allan G. Rempel, and Wolfgang Heidrich. 2009. Display considerations for night and low-illumination viewing. In Proc. of Symposium on Applied Perception in Graphics and Visualization – APGV ’09. 53–58. Google ScholarDigital Library
27. Thomas Oskam, Alexander Hornung, Huw Bowles, Kenny Mitchell, and Markus Gross. 2011. OSCAM – Optimized Stereoscopic Camera Control for Interactive 3D. ACM Transactions on Graphics 30, 6 (2011), 1–8. Google ScholarDigital Library
28. Eli Peli, Jian Yang, Robert Goldstein, and Adam Reeves. 1991. Effect of luminance on suprathreshold contrast perception. Journal of the Optical Society of America A 8, 8 (aug 1991), 1352. Google ScholarCross Ref
29. M. Schuchhardt, S. Jha, R. Ayoub, M. Kishinevsky, and G. Memik. 2015. Optimizing mobile display brightness by leveraging human visual perception. In 2015 International Conference on Compilers, Architecture and Synthesis for Embedded Systems (CASES). 11–20.Google Scholar
30. Jae Chul Shin, Hirohisa Yaguchi, and Satoshi Shioiri. 2004. Change of Color Appearance in Photopic, Mesopic and Scotopic Vision. Optical Review 11, 4 (2004), 265–271. Google ScholarCross Ref
31. David R Simmons and Frederick AA Kingdom. 1994. Contrast thresholds for stereoscopic depth identification with isoluminant and isochromatic stimuli. Vision Research 34, 22 (1994), 2971–2982.Google ScholarCross Ref
32. Gurjot Singh, Stephen R. Ellis, and J. Edward Swan. 2018. The Effect of Focal Distance, Age, and Brightness on Near-Field Augmented Reality Depth Matching. IEEE Transactions on Visualization and Computer Graphics 26, 2 (2018), 1385–1398. arXiv:1712.00088 Google ScholarCross Ref
33. Bjørn Stabell and Ulf Stabell. 1998. Chromatic rod-cone interaction during dark adaptation. J. Opt. Soc. Am. A 15, 11 (Nov 1998), 2809–2815. Google ScholarCross Ref
34. Tom Troscianko, Rachel Montagnon, Jacques Le Clerc, Emmanuelle Malbert, and Pierre-Louis Chanteau. 1991. The role of colour as a monocular depth cue. Vision research 31, 11 (1991), 1923–1929.Google Scholar
35. Khrystyna Vasylevska, Hyunjin Yoo, Tara Akhavan, and Hannes Kaufmann. 2019. Towards Eye-Friendly VR: How Bright Should It Be?. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 566–574. Google ScholarCross Ref
36. Robert Wanat and Rafał K. Mantiuk. 2014. Simulating and compensating changes in appearance between day and night vision. ACM Trans. Graph. 33 (2014), 147:1–147:12.Google ScholarDigital Library
37. John P Wann, Simon Rushton, and Mark Mon-Williams. 1995. Natural problems for stereoscopic depth perception in virtual environments. Vision research 35, 19 (1995), 2731–2736.Google Scholar
38. Simon J. Watt, Kurt Akeley, Marc O. Ernst, and Martin S. Banks. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10 (12 2005), 7–7. arXiv:https://arvojournals.org/arvo/content_public/journal/jov/932834/jov-5-10-7.pdf Google ScholarCross Ref
39. Felix A Wichmann and N Jeremy Hill. 2001. The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics 63, 8 (2001), 1293–1313.Google ScholarCross Ref
40. Sophie Wuerger, Maliha Ashraf, Minjung Kim, Jasna Martinovic, María Pérez-Ortiz, and Rafal K. Mantiuk. 2020. Spatio-chromatic contrast sensitivity under mesopic and photopic light levels. Journal of Vision 20, 4 (apr 2020), 23. Google ScholarCross Ref
41. Fangcheng Zhong, Akshay Jindal, Ali Özgür Yöntem, Param Hanji, Simon J. Watt, and Rafał K. Mantiuk. 2021. Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Transactions on Graphics 40, 6 (dec 2021), 1–14. Google ScholarDigital Library