“Apparent display resolution enhancement for moving images” by Didyk, Eisemann, Ritschel, Myszkowski and Seidel

Limited spatial resolution of current displays makes the depiction of very fine spatial details difficult. This work proposes a novel method applied to moving images that takes into account the human visual system and leads to an improved perception of such details. To this end, we display images rapidly varying over time along a given trajectory on a high refresh rate display. Due to the retinal integration time the information is fused and yields apparent super-resolution pixels on a conventional-resolution display. We discuss how to find optimal temporal pixel variations based on linear eye-movement and image content and extend our solution to arbitrary trajectories. This step involves an efficient method to predict and successfully treat potentially visible flickering. Finally, we evaluate the resolution enhancement in a perceptual study that shows that significant improvements can be achieved both for computer generated images and photographs.

1. Allen, W., and Ulichney, R. 2005. Wobulation: Doubling the addressed resolution of projection displays. In Proceedings of the Symposium Digest of Technical Papers (SID), vol. 47.4 of The Society for Information Display, 1514–1517.Google Scholar
2. Bijl, P., Schutte, K., and Hogervorst, M. A. 2006. Applicability of TOD, MTDP, MRT and DMRT for dynamic image enhancement techniques. In Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 6207.Google Scholar
3. Burr, D. C. 1979. Acuity for apparent vernier offset. Vision Research 19, 7, 835–837.Google ScholarCross Ref
4. Coleman, T. F., and Li, Y. 1996. A reflective newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J. on Optimization 6, 4, 1040–1058. Google ScholarDigital Library
5. Curcio, C. A., Sloan, K. R., Kalina, R. E., and Hendrickson, A. E. 1990. Human photoreceptor topography. The Journal of Comparative Neurology 292, 4, 497–523.Google ScholarCross Ref
6. Damera-Venkata, N., and Chang, N. L. 2009. Display supersampling. ACM Trans. Graph. 28, 1, 9:1–9:19. Google ScholarDigital Library
7. de Lange, H. 1958. Research into the dynamic nature of the human fovea – Cortex systems with intermittent and modulated light. I. Attenuation characteristics with white and colored light. J. Opt. Soc. Am. 48, 11, 777–783.Google ScholarCross Ref
8. Deering, M. F. 2005. A photon accurate model of the human eye. ACM Trans. Graph. (Proc. SIGRAPH 2005) 24, 3, 649–658. Google ScholarDigital Library
9. Fahle, M., and Poggio, T. 1981. Visual hyperacuity: Spatiotemporal interpolation in human vision. Proceedings of the Royal Society of London. Series B, Biological Sciences 213, 1193, 451–477.Google Scholar
10. Hara, Z., and Shiramatsu, N. 2000. Improvement in the picture quality of moving pictures for matrix displays. J. SID 8, 2, 129–137.Google Scholar
11. Kalloniatis, M., and Luu, C. 2009. Temporal resolution. http://webvision.med.utah.edu/temporal.html.Google Scholar
12. Klompenhouwer, M. A., and de Haan, G. 2003. Subpixel image scaling for color-matrix displays. J. SID 11, 1, 99–108.Google Scholar
13. Kopf, J., Uyttendaele, M., Deussen, O., and Cohen, M. 2007. Capturing and viewing gigapixel images. ACM Trans. Graph. (Proc. SIGGRAPH 2007) 26, 3. Google ScholarDigital Library
14. Krapels, K., Driggers, R. G., and Teaney, B. 2005. Target-acquisition performance in undersampled infrared imagers: static imagery to motion video. Applied Optics 44, 33, 7055–7061.Google ScholarCross Ref
15. Krauzlis, R., and Lisberger, S. 1994. Temporal properties of visual motion signals for the initiation of smooth pursuit eye movements in monkeys. J Neurophysiol. 72, 1, 150–162.Google ScholarCross Ref
16. Laird, J., Rosen, M., Pelz, J., Montag, E., and Daly, S. 2006. Spatio-velocity CSF as a function of retinal velocity using unstabilized stimuli. In Human Vision and Electronic Imaging XI, vol. 6057 of SPIE Proceedings Series, 32–43.Google Scholar
17. Mäkelä, P., Rovamo, J., and Whitaker, D. 1994. Effects of luminance and external temporal noise on flicker sensitivity as a function of stimulus size at various eccentricities. Vision Research 34, 15, 1981–91.Google ScholarCross Ref
18. Martinez-Conde, S., Macknik, S. L., and Hubel, D. H. 2004. The role of fixational eye movements in visual perception. Nature Reviews Neuroscience 5, 3, 229–239.Google ScholarCross Ref
19. McKee, S. P., and Taylor, D. G. 1984. Discrimination of time: comparison of foveal and peripheral sensitivity. J. Opt. Soc. Am. A 1, 6, 620–628.Google ScholarCross Ref
20. Messing, D. S., and Kerofsky, L. J. 2006. Using optimal rendering to visually mask defective subpixels. In Human Vision and Electr. Imaging XI, vol. 6057 of SPIE Proc. Series, 236–247.Google Scholar
21. Mitchell, D. P., and Netravali, A. N. 1988. Reconstruction filters in computer-graphics. Proc. SIGGRAPH 22, 4, 221–228. Google ScholarDigital Library
22. Park, S., Park, M., and Kang, M. 2003. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine 20, 3, 21–36.Google ScholarCross Ref
23. Platt, J. 2000. Optimal filtering for patterned displays. Signal Processing Letters, IEEE 7, 7, 179–181.Google ScholarCross Ref
24. Purves, D., Shimpi, A., and Lotto, B. R. 1999. An empirical explanation of the Cornsweet effect. J. Neuroscience 19, 19, 8542–8551.Google ScholarCross Ref
25. Schütz, A. C., Braun, D. I., Kerzel, D., and Gegenfurtner, K. R. 2008. Improved visual sensitivity during smooth pursuit eye movements. Nat. Neuroscience 11, 10, 1211–1216.Google ScholarCross Ref
26. Tekalp, A. 1995. Digital Video Processing. Prentice Hall. Google ScholarDigital Library
27. Van Hateren, J. H. 2005. A cellular and molecular model of response kinetics and adaptation in primate cones and horizontal cells. J. Vision 5, 4, 331–347.Google ScholarCross Ref
28. Wandell, B. 1995. Foundations of Vision. Sinauer Associates.Google Scholar
29. Zavagno, D., and Caputo, G. 2001. The glare effect and the perception of luminosity. Perception 30, 2, 209–222.Google ScholarCross Ref