“Visual equivalence: towards a new standard for image fidelity” by Ramanarayanan, Ferwerda, Walter and Bala

Efficient, realistic rendering of complex scenes is one of the grand challenges in computer graphics. Perceptually based rendering addresses this challenge by taking advantage of the limits of human vision. However, existing methods, based on predicting visible image differences, are too conservative because some kinds of image differences do not matter to human observers. In this paper, we introduce the concept of visual equivalence, a new standard for image fidelity in graphics. Images are visually equivalent if they convey the same impressions of scene appearance, even if they are visibly different. To understand this phenomenon, we conduct a series of experiments that explore how object geometry, material, and illumination interact to provide information about appearance, and we characterize how two kinds of transformations on illumination maps (blurring and warping) affect these appearance attributes. We then derive visual equivalence predictors (VEPs): metrics for predicting when images rendered with transformed illumination maps will be visually equivalent to images rendered with reference maps. We also run a confirmatory study to validate the effectiveness of these VEPs for general scenes. Finally, we show how VEPs can be used to improve the efficiency of two rendering algorithms: Light-cuts and precomputed radiance transfer. This work represents some promising first steps towards developing perceptual metrics based on higher order aspects of visual coding.

1. Adelson, E. H. 2000. Lightness perception and lightness illusions, 2nd ed. The MIT Press, Cambridge, MA, 339–351.Google Scholar
2. Anson, O., Sundstedt, V., Gutierrez, D., and Chalmers, A. 2006. Efficient selective rendering of participating media. In APGV ’06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization, 135–142. Google ScholarDigital Library
3. Beck, J. 1972. Surface Color Perception. Cornell University Press, Ithaca, NY.Google Scholar
4. Belhumeur, P. N., Kriegman, D. J., and Yuille, A. L. 1997. The bas-relief ambiguity. CVPR ’97: IEEE conference on Computer vision and pattern recognition 00, 1060–1066. Google ScholarDigital Library
5. Blake, A., and Bülthoff, H. H. 1990. Does the brain know the physics of specular reflection? Nature 394, 165–168.Google ScholarCross Ref
6. Brainard, D. H., and Maloney, L. T. 2004. Perception of color and material properties in complex scenes. Journal of Vision 4, 9, 2–4.Google ScholarCross Ref
7. Brown, R., Cooper, L., and Pham, B. 2003. Visual attention-based polygon level of detail management. In GRAPHITE ’03: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, 55–62. Google ScholarDigital Library
8. Cabral, B., Olano, M., and Nemec, P. 1999. Reflection space image based rendering. In SIGGRAPH ’99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 165–170. Google ScholarDigital Library
9. Cater, K., Chalmers, A., and Ledda, P. 2002. Selective quality rendering by exploiting human inattentional blindness: looking but not seeing. In VRST ’02: Proceedings of the ACM symposium on Virtual reality software and technology, 17–24. Google ScholarDigital Library
10. Cater, K., Chalmers, A., and Dalton, C. 2003. Varying rendering fidelity by exploiting human change blindness. In GRAPHITE ’03: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, 39–46. Google ScholarDigital Library
11. Cavanagh, P., and Leclerc, Y. G. 1989. Shape from shadows. Journal of Experimental Psychology: Human Perception and Performance 15, 1, 3–27.Google ScholarCross Ref
12. Chalmers, A., Cater, K., and Maflioli, D. 2003. Visual attention models for producing high fidelity graphics efficiently. In SCCG ’03: Proceedings of the 19th spring conference on Computer graphics, ACM Press, New York, NY, USA, 39–45. Google ScholarDigital Library
13. Cleju, I., and Saupe, D. 2006. Evaluation of supra-threshold perceptual metrics for 3D models. In APGV ’06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization, 41–44. Google ScholarDigital Library
14. Cutting, J. E., and Millard, R. T. 1984. Three gradients and the perception of flat and curved surfaces. Journal of Experimental Psychology: General 113, 2, 198–216.Google ScholarCross Ref
15. Daly, S. 1993. The visible differences predictor: an algorithm for the assessment of image fidelity. In Digital Images and Human Vision, A. B. Watson, Ed. MIT Press, 179–206. Google ScholarDigital Library
16. Dana, K. J., van Ginneken, B., Nayar, S. K., and Koenderink, J. J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 18, 1, 1–34. Google ScholarDigital Library
17. Debattista, K., Sundstedt, V., Santos, L. P., and Chalmers, A. 2005. Selective component-based rendering. In GRAPHITE ’05: Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, 13–22. Google ScholarDigital Library
18. Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In SIGGRAPH ’97, 369–378. Google ScholarDigital Library
19. DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. In ACM Transactions on Graphics. Google ScholarDigital Library
20. Dror, R. O., Willsky, A. S., and Adelson, E. H. 2004. Statistical characterization of real-world illumination. Journal of Vision 4, 9, 821–837.Google ScholarCross Ref
21. Ferwerda, J., and Pellacini, F. 2003. Functional difference predictors (FDPs): Measuring meaningful image differences. Asilomar Conference on Signals, Systems, and Computers, 1388–1392.Google Scholar
22. Ferwerda, J. A., Pellacini, F., and Greenberg, D. P. 2001. A psychophysically-based model of surface gloss perception. In Proceedings of the SPIE: Human Vision and Electronic Imaging VI, vol. 4299, 291–301.Google Scholar
23. Fleming, R. W., and Bülthoff, H. H. 2005. Low-level image cues in the perception of translucent materials. ACM Transactions on Applied Perception 2, 3, 346–382. Google ScholarDigital Library
24. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. Journal of Vision 3, 5, 347–368.Google ScholarCross Ref
25. Fleming, R. W., Torralba, A., and Adelson, E. H. 2004. Specular reflections and the perception of shape. Journal of Vision 4, 9, 798–820.Google ScholarCross Ref
26. Funkhouser, T., and Shilane, P. 2006. Partial matching of 3D shapes with priority-driven search. In Fourth Eurographics Symposium on Geometry Processing, 131–142. Google ScholarDigital Library
27. Gescheider, G. 1997. Psychophysics: The Fundamentals. Lawrence Erlbaum Associates, Mahwah, NJ.Google Scholar
28. Gibson, J. 1979. The Ecological Approach to Visual Perception. Houghton Mifflin, Boston, MA.Google Scholar
29. Gilchrist, A., Kossyfidis, C., Bonato, F., Agostini, T., Cataliotti, J., Li, X., Spehar, B., Annan, V., and Economou, E. 1999. An anchoring theory of lightness perception. Psychological Review 106, 4, 795–834.Google ScholarCross Ref
30. Hartung, B., and Kersten, D. 2002. Distinguishing shiny from matte. Journal of Vision 2, 7, 551–551.Google ScholarCross Ref
31. Ho, Y.-X., Landy, M. S., and Maloney, L. T. 2006. How direction of illumination affects visually perceived surface roughness. Journal of Vision 6, 5, 634–648.Google ScholarCross Ref
32. Howlett, S., Hamill, J., and O’Sullivan, C. 2004. An experimental approach to predicting saliency for simplified polygonal models. In APGV ’04: Proceedings of the 1st Symposium on Applied perception in graphics and visualization, 57–64. Google ScholarDigital Library
33. Joachims, T. 1999. Making large-scale support vector machine learning practical. In Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, 169–184. Google ScholarDigital Library
34. Kautz, J., Vázquez, P.-P., Heidrich, W., and Seidel, H.-P. 2000. A unified approach to prefiltered environment maps. In Proceedings of the Eurographics Workshop on Rendering Techniques 2000, Springer-Verlag, London, UK, 185–196. Google ScholarDigital Library
35. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Transactions on Graphics 25, 3, 654–663. Google ScholarDigital Library
36. Khang, B.-G., Koenderink, J. J., and Kappers, A. M. L. 2006. Perception of illumination direction in images of 3-D convex objects: Influence of surface materials and light fields. Perception 35, 5, 625–645.Google ScholarCross Ref
37. Knill, D. C., and Richards, W. 1996. Perception as Bayesian Inference. Cambridge University Press, New York, NY. Google ScholarDigital Library
38. Koenderink, J. J., van Doorn, A. J., Kappers, A. M. L., te Pas, S. F., and Pont, S. C. 2003. Illumination direction from texture shading. Journal of the Optical Society of America A 20, 6, 987–995.Google ScholarCross Ref
39. Koffka, K. 1935. Principles of Gestalt Psychology. Harcourt, Brace and World, New York, NY.Google Scholar
40. Lee, C. H., Varshney, A., and Jacobs, D. W. 2005. Mesh saliency. ACM Trans. Graph. 24, 3, 659–666. Google ScholarDigital Library
41. Luebke, D. P., and Hallen, B. 2001. Perceptually driven simplification for interactive rendering. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques, 223–234. Google ScholarDigital Library
42. Mantiuk, R., Daly, S., Myszkowski, K., and Seidel, H.-P. 2005. Predicting visible differences in high dynamic range images – model and its calibration. In Proceedings of the SPIE: Human Vision and Electronic Imaging X, vol. 5666, 204–214.Google Scholar
43. Meseth, J., Müller, G., Klein, R., Röder, F., and Arnold, M. 2006. Verification of rendering quality from measured BTFs. In APGV ’06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization, 127–134. Google ScholarDigital Library
44. Myszkowski, K. 2002. Perception-based global illumination, rendering, and animation techniques. In SCCG ’02: Proceedings of the 18th spring conference on Computer graphics, 13–24. Google ScholarDigital Library
45. Ng, R., Ramamoorthi, R., and Hanrahan, P. 2004. Triple product wavelet integrals for all-frequency relighting. ACM Transactions on Graphics, 477–487. Google ScholarDigital Library
46. Ostrovsky, Y., Cavanagh, P., and Sinha, P. 2005. Perceiving illumination inconsistencies in scenes. Perception 34, 11, 1301–1314.Google ScholarCross Ref
47. Palmer, S. E. 1999. Vision science: From Photons to Phenomenology. Bradford Books/MIT Press, Cambridge, MA.Google Scholar
48. Parkhurst, D., and Niebur, E. 2004. A feasibility test for perceptually adaptive level of detail rendering on desktop systems. In APGV ’04: Proceedings of the 1st Symposium on Applied perception in graphics and visualization, 49–56. Google ScholarDigital Library
49. Pellacini, F., Ferwerda, J. A., and Greenberg, D. P. 2000. Toward a psychophysically-based light reflection model for image synthesis. In SIGGRAPH ’00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 55–64. Google ScholarDigital Library
50. Perlin, K. 2002. Improving noise. ACM Transactions on Graphics 21, 3 (July), 681–682. Google ScholarDigital Library
51. Pont, S. C., and Koenderink, J. J. 2004. Surface illuminance flow. In 3DPVT ’04: Proceedings of the 2nd international symposium on 3D data processing, visualization and transmission, 2–9. Google ScholarCross Ref
52. Qu, L., and Meyer, G. W. 2006. Perceptually driven interactive geometry remeshing. In SI3D ’06: Proceedings of the 2006 symposium on Interactive 3D graphics and games, 199–206. Google ScholarDigital Library
53. Ramamoorthi, R., and Hanrahan, P. 2002. Frequency space environment map rendering. In ACM Transactions on Graphics, 517–526. Google ScholarDigital Library
54. Reddy, M. 2001. Perceptually optimized 3D graphics. IEEE Computer Graphics and Applications 21, 5, 68–75. Google ScholarDigital Library
55. Reinhard, E., Shirley, P., Ashikhmin, M., and Troscianko, T. 2004. Second order image statistics in computer graphics. In APGV ’04: Proceedings of the 1st Symposium on Applied perception in graphics and visualization, 99–106. Google ScholarDigital Library
56. Robilotto, R., Khang, B.-G., and Zaidi, Q. 2002. Sensory and physical determinants of perceived achromatic transparency. Journal of Vision 2, 5, 388–403.Google ScholarCross Ref
57. Rock, I. 1983. The Logic of Perception. MIT Press, Cambridge, MA.Google Scholar
58. Rogowitz, B. E., and Rushmeier, H. E. 2001. Are image quality metrics adequate to evaluate the quality of geometric objects? In Proceedings of the SPIE: Human Vision and Electronic Imaging VI, vol. 4299, 340–348.Google Scholar
59. Stokes, W. A., Ferwerda, J. A., Walter, B., and Green-berg, D. P. 2004. Perceptual illumination components: A new approach to efficient, high quality global illumination rendering. ACM Transactions on Graphics 23, 3 (Aug.), 742–749. Google ScholarDigital Library
60. Sundstedt, V., Debattista, K., and Chalmers, A. 2004. Selective rendering using task-importance maps. In APGV ’04: Proceedings of the 1st symposium on Applied perception in graphics and visualization, 175–175. Google ScholarDigital Library
61. te Pas, S. F., and Pont, S. C. 2005. A comparison of material and illumination discrimination performance for real rough, real smooth and computer generated smooth spheres. In APGV ’05: Proceedings of the 2nd symposium on Applied perception in graphics and visualization, 75–81. Google ScholarDigital Library
62. Todd, J. T., and Mingolla, E. 1983. Perception of surface curvature and direction of illumination from patterns of shading. Journal of Experimental Psychology: Human Perception and Performance 9, 4, 583–595.Google ScholarCross Ref
63. Todd, J. T., Norman, J. F., and Mingolla, E. 2004. Lightness constancy in the presence of specular highlights. Psychological Science 15, 1, 33–39.Google ScholarCross Ref
64. Tumblin, J., and Rushmeier, H. 1993. Tone reproduction for realistic images. IEEE Computer Graphics and Applications 13, 6, 42–48. Google ScholarDigital Library
65. Vapnik, V. 1995. The Nature of Statistical Learning Theory. Springer-Verlag, Berlin. Google ScholarDigital Library
66. Walter, B., Fernandez, S., Arbree, A., Bala, K., Donikian, M., and Greenberg, D. P. 2005. Lightcuts: a scalable approach to illumination. ACM Transactions on Graphics 24, 3, 1098–1107. Google ScholarDigital Library
67. Wandell, B. A. 1993. Color appearance: The effects of illumination and spatial pattern. In Proceedings of the National Academy of Sciences, vol. 90, 9778–9784.Google ScholarCross Ref
68. Ward, G. J. 1992. Measuring and modeling anisotropic reflection. In SIGGRAPH ’92: Proceedings of the 19th annual conference on Computer graphics and interactive techniques, vol. 26, 265–272. Google ScholarDigital Library
69. Watson, B., Friedman, A., and McGaffey, A. 2001. Measuring and predicting visual fidelity. In SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 213–220. Google ScholarDigital Library
70. Westlund, H. B., and Meyer, G. W. 2001. Applying appearance standards to light reflection models. In SIGGRAPH ’01: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 501–51. Google ScholarDigital Library
71. Williams, N., Luebke, D., Cohen, J. D., Kelley, M., and Schubert, B. 2003. Perceptually guided simplification of lit, textured meshes. In SI3D ’03: Proceedings of the 2003 symposium on Interactive 3D graphics, 113–121. Google ScholarDigital Library
72. Xiao, B., and Brainard, D. H. 2006. Color perception of 3D objects: constancy with respect to variation of surface gloss. In APGV ’06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization, 63–68. Google ScholarDigital Library
73. Yee, H., Pattanaik, S., and Greenberg, D. P. 2001. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. ACM Transactions on Graphics 20, 1, 39–65. Google ScholarDigital Library