“The effect of display capabilities on the gloss consistency between real and virtual objects” by Chen, Jindal, Piovarči, Wang, Seidel, et al. …
Conference:
Type(s):
Title:
- The effect of display capabilities on the gloss consistency between real and virtual objects
Session/Category Title: Technical Papers Fast-Forward
Presenter(s)/Author(s):
Abstract:
A faithful reproduction of gloss is inherently difficult because of the limited dynamic range, peak luminance, and 3D capabilities of display devices. This work investigates how the display capabilities affect gloss appearance with respect to a real-world reference object. To this end, we employ an accurate imaging pipeline to achieve perceptual gloss match between a virtual and real object presented side-by-side on an augmented-reality high-dynamic-range (HDR) stereoscopic display, which has not been previously attained to this extent. Based on this precise gloss reproduction, we conduct a series of gloss matching experiments to study how gloss perception degrades based on individual factors: object albedo, display luminance, dynamic range, stereopsis, and tone-mapping operator. We support the study with a detailed analysis of individual factors, following by an in-depth discussion on the observed perceptual effects. Our experiments demonstrate that stereoscopic presentation has a limited effect on gloss matching task on our HDR display. However, both reduced peak luminance and dynamic range of the display reduce the perceived gloss. This means that the visual system cannot compensate for the changes in gloss appearance across luminance (lack of gloss constancy), and the tone mapping operator should be carefully selected when reproducing gloss on low dynamic range (LDR) display.
References:
[1]
Wendy J Adams, Gizem Kucukoglu, Michael S Landy, and Rafał K Mantiuk. 2018. Naturally glossy: Gloss perception, illumination statistics, and tone mapping. J. Vis. 18, 13 (2018), 4–4.
[2]
Barton L Anderson. 2011. Visual perception of materials and surfaces. Curr. Biol. 21, 24 (2011), R978–R983.
[3]
Maliha Ashraf, Rafal Mantiuk, Jasna Martinovic, and Sophie Wuerger. 2022. Suprathreshold contrast matching between different luminance levels. In Color and Imaging Conference, Vol. 30. Society for Imaging Science & Technology, 219–224.
[4]
A. Blake and G. Brelstaff. 1988. Geometry from Specularities. In Proc. Int. Conf. on Computer Vision. 394–403.
[5]
A. Blake and H. Bülthoff. 1990. Does the brain know the physics of specular reflection?Nature 343, 6254 (1990), 165–168.
[6]
Peter D Burns and Don Williams. 2018. Camera resolution and distortion: Advanced edge fitting. Electronic Imaging 30 (2018), 1–5.
[7]
Patrizio Campisi and Karen Egiazarian. 2017. Blind image deconvolution: theory and applications. CRC press.
[8]
Bin Chen, Michal Piovarči, Chao Wang, Hans-Peter Seidel, Piotr Didyk, Karol Myszkowski, and Ana Serrano. 2022. Gloss management for consistent reproduction of real and virtual objects. In SIGGRAPH Asia 2022 Conference Papers. 1–9.
[9]
Bin Chen, Chao Wang, Michal Piovarči, Hans-Peter Seidel, Piotr Didyk, Karol Myszkowski, and Ana Serrano. 2021. The effect of geometry and illumination on appearance perception of different material categories. Visual Comput. 37, 12 (2021), 2975–2987.
[10]
K. Doerschner, R.W. Fleming, O. Yilmaz, P.R. Schrater, B. Hartung, and D. Kersten. 2011. Visual motion and the perception of surface material. Curr. Biol. 21, 23 (2011), 2010–2016.
[11]
Katja Doerschner, Laurence T Maloney, and Huseyin Boyaci. 2010. Perceived glossiness in high dynamic range scenes. Journal of Vision 10, 9 (2010), 11–11.
[12]
Frédéric Drago and Karol Myszkowski. 2001. Validation proposal for global illumination and rendering techniques. Comput. Graph. 25, 3 (2001), 511–518.
[13]
Ron O. Dror, Alan S. Willsky, and Edward H. Adelson. 2004. Statistical characterization of real-world illumination. J. Vis. 4 (2004), 821–837.
[14]
James A Ferwerda, Fabio Pellacini, and Donald P Greenberg. 2001. Psychophysically based model of surface gloss perception. In Human vision and electronic imaging vi, Vol. 4299. SPIE, 291–301.
[15]
Jiří Filip, Martina Kolafová, Michal Havlíček, Radomír Vávra, Michal Haindl, and Holly Rushmeier. 2018. Evaluating physical and rendered material appearance. Visual Comput. 34, 6-8 (2018), 805–816.
[16]
Graham D. Finlayson, Michal Mackiewicz, and Anya Hurlbert. 2015. Color Correction Using Root-Polynomial Regression. IEEE Transactions on Image Processing 24, 5 (may 2015), 1460–1470. https://doi.org/10.1109/TIP.2015.2405336
[17]
Roland W Fleming. 2017. Material perception. Annu. Rev. Vis. Sci. 3 (2017), 365–388.
[18]
Roland W. Fleming, Ron O. Dror, and Edward H. Adelson. 2003. Real-world illumination and the perception of surface reflectance properties. J. Vis. 3, 5 (2003), 3–3.
[19]
Adria Fores, Mark D Fairchild, and Ingeborg Tastl. 2014. Improving the perceptual uniformity of a gloss space. In Color and Imaging Conference, Vol. 2014. Society for Imaging Science and Technology, 7–13.
[20]
Adria Fores, James Ferwerda, Ingeborg Tastl, and John Recker. 2013. Perceiving gloss in surfaces and images. In Color and Imaging Conference, Vol. 2013. Society for Imaging Science and Technology, 44–51.
[21]
Davit Gigilashvili, Weiqi Shi, Zeyu Wang, Marius Pedersen, Jon Yngve Hardeberg, and Holly Rushmeier. 2021. The role of subsurface scattering in glossiness perception. ACM Transactions on Applied Perception (TAP) 18, 3 (2021), 1–26.
[22]
Param Hanji, Fangcheng Zhong, and Rafał K Mantiuk. 2020. Noise-aware merging of high dynamic range image stacks without camera calibration. In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer, 376–391.
[23]
Bruce Hartung and Daniel Kersten. 2003. How does the perception of shape interact with the perception of shiny material?Journal of Vision 3, 9 (2003), 59–59.
[24]
R.F. Hess, F.A.A. Kingdom, and L.R. Ziegler. 1999. On the relationship between the spatial channels for luminance and disparity processing. Vis. Res. 39, 3 (1999), 559–568.
[25]
Y-X. Ho, M.S. Landy, and L.T. Maloney. 2008. Conjoint measurement of gloss and surface texture. Psychol. Sci. 19, 2 (2008), 196–204.
[26]
AC Hurlbert, BG Cumming, and AJ Parker. 1991. Recognition and perceptual use of specular reflections. Investigative Ophthalmology & Visual Science 32 (1991), 105.
[27]
Aaron Isaksen, Leonard McMillan, and Steven J Gortler. 2000. Dynamically reparameterized light fields. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 297–306.
[28]
Iona S Kerrigan and Wendy J Adams. 2013. Highlights, disparity, and perceived gloss with convex and concave surfaces. Journal of Vision 13, 1 (2013), 9–9.
[29]
J.J. Kulikowski. 1976. Effective contrast constancy and linearity of contrast sensation. Vision Research 16, 12 (jan 1976), 1419–1431. https://doi.org/10.1016/0042-6989(76)90161-9
[30]
Matthias Scheller Lichtenauer, Philipp Schuetz, and Peter Zolliker. 2013. Interaction improves perception of gloss. Journal of Vision 13, 14 (2013), 14–14.
[31]
Phillip J Marlow and Barton L Anderson. 2013. Generative constraints on image cues for perceived gloss. J. Vis. 13, 14 (2013), 2–2.
[32]
Phillip J Marlow, Juno Kim, and Barton L Anderson. 2012. The perception and misperception of specular surface reflectance. Curr. Biol. 22, 20 (2012), 1909–1913.
[33]
Kenichiro Masaoka, Yukihiro Nishida, Masayuki Sugawara, Eisuke Nakasu, and Yuji Nojiri. 2013. Sensation of Realness From High-Resolution Images of Real Objects. IEEE Trans. Broadcast. 59, 1 (2013), 72–83.
[34]
Rainer Mausfeld, Gunnar Wendt, and Jürgen Golz. 2014. Lustrous material appearances: Internal and external constraints on triggering conditions for binocular lustre. i-Perception 5, 1 (2014), 1–19.
[35]
Ann McNamara. 2006. Exploring Visual and Automatic Measures of Perceptual Fidelity in Real and Simulated Imagery. ACM Trans. Appl. Percept. 3, 3 (2006), 217–238.
[36]
Daniele Menon, Stefano Andriani, and Giancarlo Calvagno. 2006. Demosaicing with directional filtering and a posteriori decision. IEEE Transactions on Image Processing 16, 1 (2006), 132–141.
[37]
Gary W. Meyer, Holly E. Rushmeier, Michael F. Cohen, Donald P. Greenberg, and Kenneth E. Torrance. 1986. An experimental evaluation of computer graphics imagery. ACM Trans. Graph. 5, 1 (1986), 30–50.
[38]
Isamu Motoyoshi, Shin’ya Nishida, Lavanya Sharan, and Edward H Adelson. 2007. Image statistics and the perception of surface qualities. Nature 447, 7141 (2007), 206–209.
[39]
Alexander A Muryy, Roland W Fleming, and Andrew E Welchman. 2016. ‘Proto-rivalry’: how the binocular brain identifies gloss. Proceedings of the Royal Society B: Biological Sciences 283, 1830 (2016), 20160383.
[40]
Gaë Obein, Kenneth Knoblauch, and Françise Viéot. 2004. Difference scaling of gloss: Nonlinearity, binocularity, and constancy. J. Vis. 4, 9 (2004), 4–4.
[41]
Edwin Olson. 2011. AprilTag: A robust and flexible visual fiducial system. In 2011 IEEE international conference on robotics and automation. IEEE, 3400–3407.
[42]
Eli Peli, Jian Yang, Robert Goldstein, and Adam Reeves. 1991. Effect of luminance on suprathreshold contrast perception. Journal of the Optical Society of America A 8, 8 (aug 1991), 1352. https://doi.org/10.1364/JOSAA.8.001352
[43]
Fabio Pellacini, James A Ferwerda, and Donald P Greenberg. 2000. Toward a psychophysically-based light reflection model for image synthesis. In Proc. ACM SIGGRAPH. 55–64.
[44]
Maria Perez-Ortiz and Rafal K Mantiuk. 2017. A practical guide and software for analysing pairwise comparison experiments. arXiv preprint arXiv:1712.03686 (2017).
[45]
Jonathan B Phillips, James A Ferwerda, and Stefan Luka. 2009. Effects of image dynamic range on apparent surface gloss. In Color and imaging conference, Vol. 2009. Society for Imaging Science and Technology, 193–197.
[46]
Sylvia C Pont and Susan F te Pas. 2006. Material — Illumination Ambiguities and the Perception of Solid Objects. Perception 35, 10 (2006), 1331–1350.
[47]
Yuichi Sakano and Hiroshi Ando. 2010. Effects of head motion and stereo viewing on perceived glossiness. J. Vis. 10, 9 (2010), 15–15.
[48]
Yuichi Sakano and Hiroshi Ando. 2012. Psychophysical evaluations of a current multi-view 3-D display: Its advantages in glossiness reproduction. Journal of the Society for Information Display 20, 5 (2012), 286–292.
[49]
Yuichi Sakano and Hiroshi Ando. 2021. Conditions of a Multi-View 3D Display for Accurate Reproduction of Perceived Glossiness. IEEE Transactions on Visualization and Computer Graphics 28, 10 (2021), 3336–3350.
[50]
A. C. Schmid, P. Barla, and K. Doerschner. 2020. Material category determined by specular reflection structure mediates the processing of image features for perceived gloss. bioRxiv (2020).
[51]
Ana Serrano, Bin Chen, Chao Wang, Michal Piovarci, Hans-Peter Seidel, Piotr Didyk, and Karol Myszkowski. 2021. The effect of shape and illumination on material perception: model and applications. ACM Trans. on Graph. 40, 4 (2021).
[52]
Midori Tanaka and Takahiko Horiuchi. 2015. Investigating perceptual qualities of static surface appearance using real materials and displayed images. Vis. Res. 115 (2015), 246–258.
[53]
Krzysztof Templin, Piotr Didyk, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2012. Highlight microdisparity for improved gloss depiction. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1–5.
[54]
Jan Jaap R. van Assen, Maarten W. A. Wijntjes, and Sylvia C. Pont. 2016. Highlight shapes and perception of gloss for real and photographed objects. J. Vis. 16, 6 (2016), 1–14.
[55]
Gunnar Wendt and Franz Faul. 2019. Differences in stereoscopic luster evoked by static and dynamic stimuli. i-Perception 10, 3 (2019), 2041669519846133.
[56]
Gunnar Wendt, Franz Faul, Vebjørn Ekroll, and Rainer Mausfeld. 2010. Disparity, motion, and color information improve gloss constancy performance. Journal of vision 10, 9 (2010), 7–7.
[57]
Gunnar Wendt, Franz Faul, and Rainer Mausfeld. 2008. Highlight disparity contributes to the authenticity and strength of perceived glossiness. Journal of Vision 8, 1 (2008), 14–14.
[58]
Josh Wills, Sameer Agarwal, David Kriegman, and Serge Belongie. 2009. Toward a perceptual space for gloss. ACM Trans. Graph. 28, 4 (2009), 1–15.
[59]
Weiye Xiao, Taoyong Tan, and Chengqi Xue. 2023. Gloss-Bridge: A Method to Reduce the Visual Perception Gap between Real-World Glossiness and PBR Reflectance Properties in Virtual Reality. Applied Sciences 13, 8 (2023), 4722.
[60]
Shoji Yamamoto, Masashi Sawabe, Mayu Yokoya, and Norimichi Tsumura. 2012. Enhancement of Gloss Perception by using Binocular Disparity. In Conference on Colour in Graphics, Imaging, and Vision, Vol. 2012. Society for Imaging Science and Technology, 226–230.
[61]
Akiko Yoshida, Matthias Ihrke, Rafał Mantiuk, and Hans-Peter Seidel. 2008. Brightness of the glare illusion. In Proceedings of the 5th symposium on Applied perception in graphics and visualization. 83–90.
[62]
Fan Zhang, Huib de Ridder, Pascal Barla, and Sylvia Pont. 2020. A systematic approach to testing and predicting light-material interactions. J. Vis. 19, 4, Article 11 (2020), 22 pages.
[63]
Fangcheng Zhong, Akshay Jindal, Özgür Yöntem, Param Hanji, Simon Watt, and Rafal Mantiuk. 2021. Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Trans. Graph. 40, 6 (2021), 241.
