“Foveated 3D graphics” by Guenter, Finch, Drucker, Tan and Snyder
Conference:
Type(s):
Title:
- Foveated 3D graphics
Session/Category Title: GPU's and Rendering
Presenter(s)/Author(s):
Abstract:
We exploit the falloff of acuity in the visual periphery to accelerate graphics computation by a factor of 5-6 on a desktop HD display (1920×1080). Our method tracks the user’s gaze point and renders three image layers around it at progressively higher angular size but lower sampling rate. The three layers are then magnified to display resolution and smoothly composited. We develop a general and efficient antialiasing algorithm easily retrofitted into existing graphics code to minimize “twinkling” artifacts in the lower-resolution layers. A standard psychophysical model for acuity falloff assumes that minimum detectable angular size increases linearly as a function of eccentricity. Given the slope characterizing this falloff, we automatically compute layer sizes and sampling rates. The result looks like a full-resolution image but reduces the number of pixels shaded by a factor of 10-15.We performed a user study to validate these results. It identifies two levels of foveation quality: a more conservative one in which users reported foveated rendering quality as equivalent to or better than non-foveated when directly shown both, and a more aggressive one in which users were unable to correctly label as increasing or decreasing a short quality progression relative to a high-quality foveated reference. Based on this user study, we obtain a slope value for the model of 1.32-1.65 arc minutes per degree of eccentricity. This allows us to predict two future advantages of foveated rendering: (1) bigger savings with larger, sharper displays than exist currently (e.g. 100 times speedup at a field of view of 70° and resolution matching foveal acuity), and (2) a roughly linear (rather than quadratic or worse) increase in rendering cost with increasing display field of view, for planar displays at a constant sharpness.
References:
1. Baudisch, P., DeCarlo, D., Duchowski, A. T., and Geisler, W. S. 2003. Focusing on the essential: considering attention in display design. Commun. ACM 46 (March), 60–66.
2. Bergström, P. 2003. Eye-movement Controlled Image Coding. PhD thesis, Linköping University, Linköping, Sweden.
3. Cheng, I. 2003. Foveated 3D model simplification. In Proceedings, Signal Processing and its Applications, 241–244.
4. Colenbrander, A. 2001. Vision and vision loss. In Duane’s Clinical Opthalmology. Lippincott, Williams and Wilkins, ch. 51.
5. Cook, R. L., Porter, T., and Carpenter, L. 1984. Distributed ray tracing. SIGGRAPH Comput. Graph. 18 (January), 137–145.
6. Duchowski, A. T., and Çöltekin, A. 2007. Foveated gaze-coningent displays for periperal LOD management, 3D visualization, and stereo imaging. ACM Trans. on Multimedia, Communications and Applications 3, 4 (Dec.).
7. Duchowski, A. T. 2002. A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, and Computers 34, 455–70.
8. Funkhouser, T., and Sequin, C. 1993. Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments. In Computer Graphics (SIGGRAPH ’93 Proceedings), vol. 27, 247–254.
9. Geisler, W. S., and Perry, J. S. 2002. Real-time simulation of arbitrary visual fields. In Proceedings of the symposum on eye tracking research applications ETRA.
10. Guenter, B. 1994. Motion compensated noise reduction. Tech. rep., Microsoft Research.
11. Han, C., Sun, B., Ramamoorthi, R., and Grinspun, E. 2007. Frequency domain normal map filtering. ACM Trans. Graph. 26 (July).
12. Horvitz, E., and Lengyel, J. 1997. Perception, attention, and resources: a decision-theoretic approach to graphics rendering. In Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, UAI’97, 238–249.
13. Levoy, M., and Whitaker, R. 1989. Gaze-directed volume rendering. Tech. rep., University of North Carolina.
14. Loschky, L. C., and McConkie, G. W. 2000. User performance with gaze contingent multiresolutional displays. In Proceedings of the 2000 symposium on Eye tracking research & applications, ETRA ’00, 97–103.
15. Luebke, D., Halen, B., Newfield, D., and Watson, B. 2000. Perceptually driven simplification using gaze-directed rendering. Tech. rep., University of Virginia.
16. Murphy, H., and Duchowski, A. T. 2001. Gaze-contingent level of detail rendering. In Eurographics 2001 (Short Presentations).
17. Murphy, H. A., and Duchowski, A. T. 2007. Hybrid image/model based gaze-contingent rendering. Applied Perception in Graphics and Visualization.
18. Myszkowski, K. 2002. Perception-based global illumination, rendering, and animation techniques. In SCCG ’02: Proceedings of the 18th Spring Conference on Computer Graphics, 13–24.
19. Nehab, D., Sander, P. V., Lawrence, J. D., Tatarchuk, N., and Isidoro, J. R. 2007. Accelerating real-time shading with reverse reprojection caching. In ACM SIGGRAPH/Eurographics Symposium on Graphics Hardware, 25–35.
20. Ohshima, T., Yamamoto, H., and Tamura, H. 1996. Gaze-directed adaptive rendering for interacting with virtual space. In Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), IEEE Computer Society, Washington, DC, USA, VRAIS ’96, 103–110.
21. Olano, M., and Baker, D. 2010. LEAN mapping. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D ’10, 181–188.
22. O’Sullivan, C., Dingliana, J., and Howlett, S. 2002. Gaze-contingent algorithms for interactive graphics. In The Mind’s Eyes: Cognitive and Applied Aspects of Eye Movement Research, J. Hyönä, R. Radach, and H. Deubel, Eds. Elsevier Science, Oxford.
23. Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: towards a new standard for image fidelity. ACM Trans. Graph. 26 (July).
24. Reddy, M. 1998. Specification and evaluation of level of detail selection criteria. Virtual Reality 3, 2, 132–143.
25. Reddy, M. 2001. Perceptually optimized 3D graphics. Computer Graphics and Applications, IEEE 21, 5 (sep/oct), 68–75.
26. Reingold, E., Loschky, L. C., McConkie, G. W., and Stampe, D. M. 2003. Gaze-contingent multiresolution displays: an integrative review. Human Factors 45, 2, 307–28.
27. Strasburger, H., Rentschler, I., and Jüttner, M. 2011. Peripheral vision and pattern recognition: a review. Journal of Vision 11, 5, 1–82.
28. Thibos, L. N. 1989. Image processing by the human eye. In Visual Communications and Image Processing IV, W. A. Pearlman, Ed., Proc. SPIE 1199, 1148–1153.
29. Virsu, V., and Romano, J. 1979. Visual resolution, contrast sensitivity, and the cortical magnification factor. Experimental Brain Research 37, 475–494.
30. Watson, B., Walker, N., Hodges, L. F., and Worden, A. 1997. Managing level of detail through peripheral degradation: effects on search performance with a head-mounted display. ACM Trans. Comput.-Hum. Interact. 4 (December), 323–346.
31. Watson, B., Walker, N., and Hodges, L. F. 2004. Supra-threshold control of peripheral LOD. ACM Trans. Graph. 23 (Aug.), 750–759.
32. Weaver, K. A. 2007. Design and evaluation of a perceptually adaptive rendering system for immersive virtual reality environments. Master’s thesis, Iowa State University.
33. Weymouth, F. W. 1958. Visual sensory units and the minimum angle of resolution. American Journal of Opthalmology 46, 102–113.
34. Yee, H., Pattanaik, S., and Greenberg, D. P. 2001. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Trans. Graph. 20 (January), 39–65.
