“Perception of perspective distortions in image-based rendering” by Vangorp, Richardt, Cooper, Chaurasia, Banks, et al. …

  • ©Peter Vangorp, Christian Richardt, Emily A. Cooper, Gaurav Chaurasia, Martin (Marty) S. Banks, and George Drettakis

Conference:


Type:


Title:

    Perception of perspective distortions in image-based rendering

Session/Category Title: Perception

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Image-based rendering (IBR) creates realistic images by enriching simple geometries with photographs, e.g., mapping the photograph of a building façade onto a plane. However, as soon as the viewer moves away from the correct viewpoint, the image in the retina becomes distorted, sometimes leading to gross misperceptions of the original geometry. Two hypotheses from vision science state how viewers perceive such image distortions, one claiming that they can compensate for them (and therefore perceive scene geometry reasonably correctly), and one claiming that they cannot compensate (and therefore can perceive rather significant distortions). We modified the latter hypothesis so that it extends to street-level IBR. We then conducted a rigorous experiment that measured the magnitude of perceptual distortions that occur with IBR for façade viewing. We also conducted a rating experiment that assessed the acceptability of the distortions. The results of the two experiments were consistent with one another. They showed that viewers’ percepts are indeed distorted, but not as severely as predicted by the modified vision science hypothesis. From our experimental results, we develop a predictive model of distortion for street-level IBR, which we use to provide guidelines for acceptability of virtual views and for capture camera density. We perform a confirmatory study to validate our predictions, and illustrate their use with an application that guides users in IBR navigation to stay in regions where virtual views yield acceptable perceptual distortions.

References:


    1. Adams, K. R. 1972. Perspective and the viewpoint. Leonardo 5, 3, 209–217.Google ScholarCross Ref
    2. Bakeman, R. 2005. Recommended effect size statistics for repeated measures designs. Behavior Research Methods 37, 3, 379–384.Google ScholarCross Ref
    3. Banks, M. S., Held, R. T., and Girshick, A. R. 2009. Perception of 3-D layout in stereo displays. Information Display 25, 1, 12–16.Google ScholarCross Ref
    4. Buehler, C., Bosse, M., McMillan, L., Gortler, S., and Cohen, M. 2001. Unstructured lumigraph rendering. In Proceedings of ACM SIGGRAPH 2001, 425–432. Google ScholarDigital Library
    5. Cooper, E. A., Piazza, E. A., and Banks, M. S. 2012. The perceptual basis of common photographic practice. Journal of Vision 12, 5, 8:1–14.Google ScholarCross Ref
    6. Debevec, P., Yu, Y., and Borshukov, G. 1998. Efficient view-dependent image-based rendering with projective texture-mapping. In Proceedings of EGWR ’98, 105–116.Google Scholar
    7. Ernst, M. O., and Banks, M. S. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 6870, 429–433.Google Scholar
    8. Goldstein, E. B. 1987. Spatial layout, orientation relative to the observer, and perceived projection in pictures viewed at an angle. Journal of Experimental Psychology: Human Perception and Performance 13, 2, 256.Google ScholarCross Ref
    9. Kopf, J., Chen, B., Szeliski, R., and Cohen, M. 2010. Street slide: browsing street level imagery. ACM Transactions on Graphics 29, 3, 96:1–8. Google ScholarDigital Library
    10. Lumsden, E. A. 1983. Perception of radial distance as a function of magnification and truncation of depicted spatial layout. Attention, Perception, & Psychophysics 33, 2, 177–182.Google ScholarCross Ref
    11. Morvan, Y., and O’Sullivan, C. 2009. Handling occluders in transitions from panoramic images: A perceptual study. ACM Transactions on Applied Perception 6, 4, 25:1–15. Google ScholarDigital Library
    12. Perkins, D. N. 1972. Visual discrimination between rectangular and nonrectangular parallelopipeds. Attention, Perception, & Psychophysics 12, 5, 396–400.Google ScholarCross Ref
    13. Pharr, M., and Humphreys, G. 2010. Physically Based Rendering: From Theory to Implementation, 2nd ed. Morgan Kaufmann. Google ScholarDigital Library
    14. Pirenne, M. H. 1970. Optics, Painting and Photography. Cambridge University Press.Google Scholar
    15. Revelle, W. 2008. psych: Procedures for Psychological, Psychometric, and Personality Research. R package version 1.0-42+.Google Scholar
    16. Rosinski, R. R., Mulholland, T., Degelman, D., and Farber, J. 1980. Picture perception: An analysis of visual compensation. Attention, Perception, & Psychophysics 28, 6, 521–526.Google ScholarCross Ref
    17. Sedgwick, H. A. 1991. The effects of viewpoint on the virtual space of pictures. In Pictorial Communication in Virtual and Real Environments, S. R. Ellis, Ed. Taylor & Francis, 460–479. Google ScholarDigital Library
    18. Shum, H. Y., Chan, S. C., and Kang, S. B. 2006. Image-based rendering, vol. 2. Springer. Google ScholarDigital Library
    19. Smith, P. C., and Smith, O. W. 1961. Ball throwing responses to photographically portrayed targets. Journal of Experimental Psychology 62, 3, 223.Google ScholarCross Ref
    20. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics 25, 3, 835–846. Google ScholarDigital Library
    21. Steinicke, F., Bruder, G., and Kuhl, S. 2011. Realistic perspective projections for virtual objects and environments. ACM Transactions on Graphics 30, 5, 112:1–10. Google ScholarDigital Library
    22. Stich, T., Linz, C., Wallraven, C., Cunningham, D., and Magnor, M. 2011. Perception-motivated interpolation of image sequences. ACM Transactions on Applied Perception 8, 2, 11:1–25. Google ScholarDigital Library
    23. Todorović, D. 2008. Is pictorial perception robust? the effect of the observer vantage point on the perceived depth structure of linear-perspective images. Perception 37, 1, 106.Google ScholarCross Ref
    24. Todorović, D. 2009. The effect of the observer vantage point on perceived distortions in linear perspective images. Attention, Perception, & Psychophysics 71, 1, 183–193.Google ScholarCross Ref
    25. Vangorp, P., Chaurasia, G., Laffont, P.-Y., Fleming, R. W., and Drettakis, G. 2011. Perception of visual artifacts in image-based rendering of façades. Computer Graphics Forum 30, 4 (Proceedings of EGSR 2011), 1241–1250. Google ScholarDigital Library
    26. Vincent, L. 2007. Taking online maps down to street level. Computer 40, 118–120. Google ScholarDigital Library
    27. Vishwanath, D., Girshick, A. R., and Banks, M. S. 2005. Why pictures look right when viewed from the wrong place. Nature Neuroscience 8, 10, 1401–1410.Google ScholarCross Ref
    28. Wallach, H., and Marshall, F. 1986. Shape constancy in pictorial representation. Attention, Perception, & Psychophysics 39, 233–235.Google ScholarCross Ref
    29. Watt, S. J., Akeley, K., Ernst, M. O., and Banks, M. S. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10, 7:834–862.Google ScholarCross Ref
    30. Yang, T., and Kubovy, M. 1999. Weakening the robustness of perspective: Evidence for a modified theory of compensation in picture perception. Attention, Perception, & Psychophysics 61, 3, 456–467.Google ScholarCross Ref
    31. Yu, J., McMillan, L., and Sturm, P. 2010. Multiperspective modeling, rendering and imaging. Computer Graphics Forum 29, 1, 227–246.Google ScholarCross Ref
    32. Zar, J. H. 2010. Biostatistical Analysis, 5th ed. Prentice Hall. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: