“Saccade-contingent Rendering”
Conference:
Type(s):
Title:
- Saccade-contingent Rendering
Presenter(s)/Author(s):
Abstract:
In this work we introduce saccade-contingent rendering, a perceptually optimized rendering technique which takes advantage of reductions in visual acuity after saccades.
References:
[1]
Rachel Albert, Anjul Patney, David Luebke, and Joohwan Kim. 2017. Latency Requirements for Foveated Rendering in Virtual Reality. ACM Trans. Appl. Percept. 14, 4, Article 25 (sep 2017), 13 pages. https://doi.org/10.1145/3127589
[2]
Elena Arabadzhiyska, Okan Tarhan Tursun, Karol Myszkowski, Hans-Peter Seidel, and Piotr Didyk. 2017. Saccade landing position prediction for gaze-contingent rendering. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1?12.
[3]
Michael Bach and Kerstin Sch?fer. 2016. Visual Acuity Testing: Feedback Affects Neither Outcome nor Reproducibility, but Leaves Participants Happier. PLOS ONE 11 (01 2016), 1?11. https://doi.org/10.1371/journal.pone.0147803
[4]
A Terry Bahill. 1975. Most naturally occurring human saccades have magnitudes of 15 deg or less. Invest. Ophthalmol 14 (1975), 468?469.
[5]
Shrikant R Bharadwaj and Clifton M Schor. 2005. Acceleration characteristics of human ocular accommodation. Vision Research 45, 1 (2005), 17?28.
[6]
Marco Boi, Martina Poletti, Jonathan D. Victor, and Michele Rucci. 2017. Consequences of the Oculomotor Cycle for the Dynamics of Perception. Current Biology 27, 9 (2017), 1268?1277.
[7]
James K Bowmaker and HJk Dartnall. 1980. Visual pigments of rods and cones in a human retina.The Journal of physiology 298, 1 (1980), 501?511.
[8]
David H Brainard. 1997. The psychophysics toolbox. Spatial vision 10, 4 (1997), 433?436.
[9]
David C Burr, M Concetta Morrone, and John Ross. 1994. Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature 371, 6497 (1994), 511?513.
[10]
Ashley M Clark, Janis Intoy, Michele Rucci, and Martina Poletti. 2022. Eye drift during fixation predicts visual acuity. Proceedings of the National Academy of Sciences 119, 49 (2022), e2200256119.
[11]
Michele A. Cox, Janis Intoy, Yuanhao H. Li, Scott Murdison, Bin Yang, Zhetuo Zhao, and Michele Rucci. 2022. Oculomotor influences on the dynamics of visual sensitivity. Journal of Vision 22, 14 (2022), 4085.
[12]
Scott Daly. 2001. Engineering observations from spatiovelocity and spatiotemporal visual models. In Vision Models and Applications to Image and Video Processing. Springer, 179?200.
[13]
Gyorgy Denes, Akshay Jindal, Aliaksei Mikhailiuk, and Rafa? K Mantiuk. 2020. A perceptual model of motion quality for rendering with adaptive refresh-rate and resolution. ACM Transactions on Graphics (TOG) 39, 4 (2020), 133?1.
[14]
Gyorgy Denes, Kuba Maruszczyk, George Ash, and Rafa? K Mantiuk. 2019. Temporal Resolution Multiplexing: Exploiting the limitations of spatio-temporal vision for more efficient VR rendering. IEEE transactions on visualization and computer graphics 25, 5 (2019), 2072?2082.
[15]
Mark R Diamond, John Ross, and Maria C Morrone. 2000. Extraretinal control of saccadic suppression. Journal of Neuroscience 20, 9 (2000), 3449?3455.
[16]
Budmonde Duinkharjav, Kenneth Chen, Abhishek Tyagi, Jiayi He, Yuhao Zhu, and Qi Sun. 2022. Color-perception-guided display power reduction for virtual reality. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1?16.
[17]
Jasper H Fabius, Alessio Fracasso, Tanja CW Nijboer, and Stefan Van der Stigchel. 2019. Time course of spatiotopic updating across saccades. Proceedings of the National Academy of Sciences 116, 6 (2019), 2027?2032.
[18]
Agostino Gibaldi and Martin S Banks. 2019. Binocular eye movements are adapted to the natural environment. Journal of Neuroscience 39, 15 (2019), 2877?2888.
[19]
Alexander Goettker, Kevin J MacKenzie, and T Scott Murdison. 2020. Differences between oculomotor and perceptual artifacts for temporally limited head mounted displays. Journal of the Society for Information Display 28, 6 (2020), 509?519.
[20]
Michael J Gray, Annabelle Blangero, James P Herman, Josh Wallman, and Mark R Harwood. 2014. Adaptation of naturally paced saccades. Journal of neurophysiology 111, 11 (2014), 2343?2354.
[21]
Phillip Guan, Olivier Mercier, Michael Shvartsman, and Douglas Lanman. 2022. Perceptual Requirements for Eye-Tracked Distortion Correction in VR. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH ?22). Association for Computing Machinery, New York, NY, USA, Article 51, 8 pages. https://doi.org/10.1145/3528233.3530699
[22]
Phillip Guan, Eric Penner, Joel Hegland, Benjamin Letham, and Douglas Lanman. 2023. Perceptual Requirements for World-Locked Rendering in AR and VR. In SIGGRAPH Asia 2023 Conference Papers (, Sydney, NSW, Australia, ) (SA ?23). Association for Computing Machinery, New York, NY, USA, Article 35, 10 pages. https://doi.org/10.1145/3610548.3618134
[23]
Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. 2012. Foveated 3D graphics. ACM transactions on Graphics (tOG) 31, 6 (2012), 1?10.
[24]
Saad Idrees, Matthias P Baumann, Felix Franke, Thomas A M?nch, and Ziad M Hafed. 2020. Perceptual saccadic suppression starts in the retina. Nature communications 11, 1 (2020), 1977.
[25]
Akshay Jindal, Krzysztof Wolski, Karol Myszkowski, and Rafa? K Mantiuk. 2021. Perceptual model for adaptive local shading and refresh rate. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1?18.
[26]
D. H. Kelly. 1979. Motion and vision. II. Stabilized spatio-temporal threshold surface. J. Opt. Soc. Am. 69, 10 (Oct 1979), 1340?1349. https://doi.org/10.1364/JOSA.69.001340
[27]
Mario Kleiner, David Brainard, and Denis Pelli. 2007. What?s new in Psychtoolbox-3? (2007).
[28]
JJ Koenderink and AJ Van Doorn. 1978. Visual detection of spatial contrast; influence of location in the visual field, target extent and illuminance level. Biological Cybernetics 30, 3 (1978), 157?167.
[29]
Eileen Kowler. 2011. Eye movements: The past 25 years. Vision Research 51, 19 (2011), 1457?1483.
[30]
Brooke Krajancich, Petr Kellnhofer, and Gordon Wetzstein. 2023. Towards Attention?aware Foveated Rendering. ACM Transactions on Graphics (TOG) 42, 4 (2023), 1?10.
[31]
Lisa M Kroell and Martin Rolfs. 2022. Foveal vision anticipates defining features of eye movement targets. Elife 11 (2022), e78106.
[32]
Xutao Kuang, Martina Poletti, Jonathan D. Victor, and Michele Rucci. 2012. Temporal Encoding of Spatial Information during Active Visual Fixation. Current Biology 22 (2012), 510?514.
[33]
Michael Land, Neil Mennie, and Jennifer Rusted. 1999. The roles of vision and eye movements in the control of activities of daily living. Perception 28, 11 (1999), 1311?1328.
[34]
Yuanhao Li, Michele A Cox, T Scott Murdison, Bin Yang, Janis Intoy, Zhetuo Zhao, and Michele Rucci. 2021. Post-saccadic dynamics of visual sensitivity across the visual field. Journal of Vision 21, 9 (2021), 1930?1930.
[35]
Thurmon E Lockhart and Wen Shi. 2010. Effects of age on dynamic accommodation. Ergonomics 53, 7 (2010), 892?903.
[36]
Lester C. Loschky and Gary S. Wolverton. 2007. How late can you update gaze-contingent multiresolutional displays without detection?ACM Trans. Multimedia Comput. Commun. Appl. 3, 4, Article 7 (dec 2007), 10 pages. https://doi.org/10.1145/1314303.1314310
[37]
Rafa? Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Trans. Graph. 30, 4, Article 40 (jul 2011), 14 pages. https://doi.org/10.1145/2010324.1964935
[38]
Rafa? K. Mantiuk, Maliha Ashraf, and Alexandre Chapiro. 2022. StelaCSF: A Unified Model of Contrast Sensitivity as the Function of Spatio-Temporal Frequency, Eccentricity, Luminance and Area. ACM Trans. Graph. 41, 4, Article 145 (jul 2022). https://doi.org/10.1145/3528223.3530115
[39]
Rafa? K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. 2021. FovVideoVDP: A visible difference predictor for wide field-of-view video. ACM Transactions on Graphics (TOG) 40, 4 (2021).
[40]
Ethel Matin. 1974. Saccadic suppression: a review and an analysis.Psychological bulletin 81, 12 (1974), 899.
[41]
Sage L Matthews, Alvaro Uribe-Quevedo, and Alexander Theodorou. 2020. Rendering optimizations for virtual reality using eye-tracking. In 2020 22nd symposium on virtual and augmented reality (SVR). IEEE, 398?405.
[42]
Naghmeh Mostofi, Zhetuo Zhao, Janis Intoy, Marco Boi, Jonathan D. Victor, and Michele Rucci. 2020. Spatiotemporal Content of Saccade Transients. Current Biology 30, 9 (2020), 3999?4008.
[43]
T. Scott Murdison, Gunnar Blohm, and Frank Bremmer. 2019. Saccade-induced changes in ocular torsion reveal predictive orientation perception. Journal of Vision 19, 11 (2019), 10?10.
[44]
Matthias Niemeier, J Douglas Crawford, and Douglas B Tweed. 2003. Optimal transsaccadic integration explains distorted spatial perception. Nature 422, 6927 (2003), 76?80.
[45]
Lucy Owen, Jonathan Browder, Benjamin Letham, Gideon Stocek, Chase Tymms, and Michael Shvartsman. 2021. Adaptive Nonparametric Psychophysics. arxiv:2104.09549
[46]
Nitish Padmanaban, Robert Konrad, Tal Stramer, Emily A Cooper, and Gordon Wetzstein. 2017. Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays. Proceedings of the National Academy of Sciences 114, 9 (2017), 2183?2188.
[47]
Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards Foveated Rendering for Gaze-Tracked Virtual Reality. ACM Trans. Graph. 35, 6, Article 179 (dec 2016), 12 pages.
[48]
Denis G Pelli. 1997. The VideoToolbox software for visual psychophysics: transforming numbers into movies.Spatial vision 10, 4 (1997), 437?442.
[49]
DA Robinson. 1964. The mechanics of human saccadic eye movement. The Journal of physiology 174, 2 (1964), 245.
[50]
John G Robson. 1966. Spatial and temporal contrast-sensitivity functions of the visual system. Josa 56, 8 (1966), 1141?1142.
[51]
Martin Rolfs and Richard Schweitzer. 2022. Coupling perception to action through incidental sensory consequences of motor behaviour. Nature Reviews Psychology 1, 2 (2022), 112?123.
[52]
John Ross, M Concetta Morrone, Michael E Goldberg, and David C Burr. 2001. Changes in visual perception at the time of saccades. Trends in neurosciences 24, 2 (2001), 113?121.
[53]
Michele Rucci and Martina Poletti. 2015. Control and Functions of Fixational Eye Movements. Annual Review of Vision Science 1 (2015), 499?518.
[54]
Richard Schweitzer and Martin Rolfs. 2020. Intra-saccadic motion streaks as cues to linking object locations across saccades. Journal of Vision 20, 4 (2020), 17?17.
[55]
Nicholas T. Swafford, Jos? A. Iglesias-Guitian, Charalampos Koniaris, Bochang Moon, Darren Cosker, and Kenny Mitchell. 2016. User, Metric, and Computational Evaluation of Foveated Rendering Methods. In Proceedings of the ACM Symposium on Applied Perception (Anaheim, California) (SAP ?16). Association for Computing Machinery, New York, NY, USA, 7?14. https://doi.org/10.1145/2931002.2931011
[56]
Taimoor Tariq, Cara Tursun, and Piotr Didyk. 2022. Noise-based enhancement for foveated rendering. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1?14.
[57]
Cara Tursun and Piotr Didyk. 2022. Perceptual Visibility Model for Temporal Contrast Changes in Periphery. ACM Transactions on Graphics 42, 2 (2022), 1?16.
[58]
Okan Tarhan Tursun, Elena Arabadzhiyska-Koleva, Marek Wernikowski, Rados?aw Mantiuk, Hans-Peter Seidel, Karol Myszkowski, and Piotr Didyk. 2019. Luminance-Contrast-Aware Foveated Rendering. ACM Trans. Graph. 38, 4, Article 98 (jul 2019), 14 pages. https://doi.org/10.1145/3306346.3322985
[59]
V Virsu and J Rovamo. 1979. Visual resolution, contrast sensitivity, and the cortical magnification factor. Experimental brain research 37 (1979), 475?494.
[60]
Frances C. Volkmann, Lorrin A. Riggs, Keith D. White, and Robert K. Moore. 1978. Contrast sensitivity during saccadic eye movements. Vision Research 18, 9 (1978), 1193?1199.
[61]
Andrew B Watson and Albert J Ahumada. 2016. The pyramid of visibility. Electronic Imaging 28, 16 (2016), 1?1.
[62]
Andrew B Watson, Albert J Ahumada, and Joyce E Farrell. 1986. Window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays. JOSA A 3, 3 (1986), 300?307.
[63]
Benjamin Watson, Neff Walker, and Larry F Hodges. 1996. Effectiveness of spatial level of detail degradation in the periphery of head-mounted displays. In Conference Companion on Human Factors in Computing Systems. 227?228.
[64]
Benjamin Watson, Neff Walker, Larry F. Hodges, and Aileen Worden. 1997. Managing Level of Detail through Peripheral Degradation: Effects on Search Performance with a Head-Mounted Display. ACM Trans. Comput.-Hum. Interact. 4, 4 (dec 1997), 323?346. https://doi.org/10.1145/267135.267137
[65]
Yang Zhao, Dave Lindberg, Bruce Cleary, Olivier Mercier, Ryan Mcclelland, Eric Penner, Yu-Jen Lin, Julia Majors, and Douglas Lanman. 2023. Retinal-Resolution Varifocal VR. In ACM SIGGRAPH 2023 Emerging Technologies. 1?3.
