“Perceptually Adaptive Real-Time Tone Mapping” by Tariq, Matsuda, Penner, Jia, Lanman, et al. …
Conference:
Type(s):
Title:
- Perceptually Adaptive Real-Time Tone Mapping
Session/Category Title: Technical Papers Fast-Forward
Presenter(s)/Author(s):
Abstract:
Tone mapping operators aim to remap content to a display’s dynamic range. Virtual reality is a popular new display modality that has significant differences from other media, making the use of traditional tone mapping techniques difficult. Moreover, real-time adaptive estimation of tone curves that faithfully maintain appearance remains a significant challenge. In this work, we propose a real-time perceptual contrast-matching framework, that allows us to optimally remap scenes for target displays. Our framework is optimized for efficiency and runs on a mobile Quest 2 headset in under 1ms per frame. A subjective study on an HDR-VR prototype demonstrates our method’s effectiveness across a wide range of display luminances, producing imagery that is preferred to alternatives tone mapped at peak luminances an order of magnitude higher. This result highlights the importance of good tone mapping for visual quality in VR.
References:
[1]
2016. Preparing for Real HDR. https://developer.nvidia.com/preparing-real-hdr.
[2]
Maliha Ashraf, Rafal Mantiuk, Jasna Martinovic, and Sophie Wuerger. 2022. Suprathreshold contrast matching between different luminance levels. In Color Imaging Conference (CIC).
[3]
Tunç Ozan Aydin, Nikolce Stefanoski, Simone Croci, Markus Gross, and Aljoscha Smolic. 2014. Temporally coherent local tone mapping of HDR video. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–13.
[4]
Peter G. J. Barten. 2003. Formula for the contrast sensitivity of the human eye. In Proc. SPIE 5294, Image Quality and System Performance, Yoichi Miyake and D. Rene Rasmussen (Eds.). 231–238. https://doi.org/10.1117/12.537476
[5]
Ronan Boitard, Rémi Cozot, Dominique Thoreau, and Kadi Bouatouch. 2014. Zonal brightness coherency for video tone mapping. Signal Process. Image Commun. 29 (2014), 229–246.
[6]
Yung-Yu Chuang, Alexei A. Efros, John Ruskin, and Frédo Durand. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics (2002).
[7]
Frédéric Drago, Karol Myszkowski, Thomas Annen, and Norishige Chiba. 2003. Adaptive Logarithmic Mapping For Displaying High Contrast Scenes. Computer Graphics Forum 22 (2003).
[8]
Gabriel Eilertsen, Rafał K Mantiuk, and Jonas Unger. 2015. Real-time noise-aware tone mapping. ACM Transactions on Graphics (TOG) 34, 6 (2015), 1–15.
[9]
Ific Goudé, Rémi Cozot, and Olivier Le Meur. 2020. A perceptually coherent TMO for visualization of 360 HDR images on HMD. In Transactions on Computational Science XXXVII: Special Issue on Computer Graphics. Springer, 109–128.
[10]
Mary M Hayhoe, Norma I Benimoff, and DC Hood. 1987. The time-course of multiplicative and subtractive adaptation process. Vision Research 27, 11 (1987), 1981–1996.
[11]
Chris Kiser, Erik Reinhard, Mike Tocci, and Nora Tocci. 2012. Real time automated tone mapping system for HDR video. In IEEE International Conference on Image Processing, Vol. 134. 2749–2752.
[12]
Mario Kleiner, David Brainard, and Denis Pelli. 2007. What’s new in Psychtoolbox-3? (2007).
[13]
Grzegorz Krawczyk, Karol Myszkowski, and Hans-Peter Seidel. 2005. Perceptual effects in real-time tone mapping. In Proceedings of the 21st spring conference on Computer graphics. 195–202.
[14]
JJ Kulikowski. 1976. Effective contrast constancy and linearity of contrast sensation. Vision research 16, 12 (1976), 1419–1431.
[15]
Rafał Mantiuk, Scott Daly, and Louis Kerofsky. 2008. Display adaptive tone mapping. In ACM SIGGRAPH 2008. 1–10.
[16]
Rafał K. Mantiuk, Maliha Ashraf, and Alexandre Chapiro. 2022. StelaCSF: A Unified Model of Contrast Sensitivity as the Function of Spatio-Temporal Frequency, Eccentricity, Luminance and Area. ACM Transactions on Graphics (TOG) 41, 4 (2022).
[17]
Rafał K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. 2021. FovVideoVDP: A Visible Difference Predictor for Wide Field-of-View Video. ACM Trans. Graph. 40, 4 (2021).
[18]
Nathan Matsuda, Alex Chapiro, Yang Zhao, Clinton Smith, Romain Bachy, and Douglas Lanman. 2022. Realistic Luminance in VR. In SIGGRAPH Asia 2022 Conference Papers. 1–8.
[19]
Arian Mehrfard, Javad Fotouhi, Giacomo Taylor, Tess Forster, Nassir Navab, and Bernhard Fuerst. 2019. A comparative analysis of virtual reality head-mounted display systems. arXiv preprint:1912.02913 (2019).
[20]
Miguel Melo, Kadi Bouatouch, Maximino Bessa, Hugo Coelho, Remi Cozot, and Alan Chalmers. 2018. Tone Mapping HDR Panoramas for Viewing in Head Mounted Displays. In VISIGRAPP (1: GRAPP). 232–239.
[21]
Hossein Najaf-Zadeh, Madhukar Budagavi, and Esmaeil Faramarzi. 2017. VR+ HDR: A system for view-dependent rendering of HDR video in virtual reality. In 2017 IEEE International Conference on Image Processing (ICIP). 1032–1036.
[22]
PolyHaven. 2023. HDRIs. https://www.polyhaven.com.
[23]
Erik Reinhard, Wolfgang Heidrich, Paul Debevec, Sumanta Pattanaik, Greg Ward, and Karol Myszkowski. 2010. High dynamic range imaging: acquisition, display, and image-based lighting. Morgan Kaufmann.
[24]
Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. 2002. Photographic tone reproduction for digital images. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques. 267–276.
[25]
Christophe Schlick. 1995. Quantization techniques for visualization of high dynamic range pictures. In Photorealistic rendering techniques. Springer, 7–20.
[26]
Robert Shapley, Ehud Kaplan, and Keith Purpura. 1993. Contrast sensitivity and light adaptation in photoreceptors or in the retinal network. Contrast sensitivity 5 (1993), 103–116.
[27]
Robert C Streijl, Stefan Winkler, and David S Hands. 2016. Mean opinion score (MOS) revisited: methods and applications, limitations and alternatives. Multimedia Systems 22, 2 (2016), 213–227.
[28]
Jack Tumblin, Jessica K. Hodgins, and Brian K. Guenter. 1999. Two methods for display of high contrast images. ACM Transactions on Graphics (TOG) 18 (1999), 56–94.
[29]
Okan Tarhan Tursun, Elena Arabadzhiyska-Koleva, Marek Wernikowski, Radosław Mantiuk, Hans-Peter Seidel, Karol Myszkowski, and Piotr Didyk. 2019. Luminance-contrast-aware foveated rendering. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1–14.
[30]
Peter Vangorp, Karol Myszkowski, Erich W Graf, and Rafał K Mantiuk. 2015. A model of local adaptation. ACM Transactions on Graphics (TOG) (2015).
[31]
David R Walton, Rafael Kuffner Dos Anjos, Sebastian Friston, David Swapp, Kaan Akşit, Anthony Steed, and Tobias Ritschel. 2021. Beyond blur: Real-time ventral metamers for foveated rendering. ACM Transactions on Graphics 40, 4 (2021), 1–14.
[32]
Robert Wanat and Rafał K Mantiuk. 2014. Simulating and compensating changes in appearance between day and night vision. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–12.
[33]
Minqi Wang and Emily A Cooper. 2021. A Re-examination of Dichoptic Tone Mapping. ACM Transactions on Graphics (TOG) 40, 2 (2021), 1–15.
[34]
Zhuming Zhang, Xinghong Hu, Xueting Liu, and Tien-Tsin Wong. 2018. Binocular tone mapping with improved overall contrast and local details. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 433–442.
