“Stylization and abstraction of photographs”

  • ©Doug DeCarlo and Anthony Santella




    Stylization and abstraction of photographs



    Good information design depends on clarifying the meaningful structure in an image. We describe a computational approach to stylizing and abstracting photographs that explicitly responds to this design goal. Our system transforms images into a line-drawing style using bold edges and large regions of constant color. To do this, it represents images as a hierarchical structure of parts and boundaries computed using state-of-the-art computer vision. Our system identifies the meaningful elements of this structure using a model of human perception and a record of a user’s eye movements in looking at the photo; the system renders a new image using transformations that preserve and highlight these visual elements. Our method thus represents a new alternative for non-photorealistic rendering both in its visual style, in its approach to visual form, and in its techniques for interaction.


    1. AGRAWALA, M., AND STOLTE, C. 2001. Rendering effective route maps: improving usability through generalization. In Proc. of ACM SIGGRAPH 2001, 241-249. Google Scholar
    2. AHUJA, N. 1996. A transform for multiscale image segmentation by integrated edge and region detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 18, 12, 1211-1235. Google Scholar
    3. BURT, P., AND ADELSON, E. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Communications 31, 4, 532-540.Google Scholar
    4. CAMPBELL, F., AND ROBSON, J. 1968. Application of Fourier analysis to the visibility of gratings. Journal of Physiology 197, 551-566.Google Scholar
    5. CHRISTOUDIAS, C., GEORGESCU, B., AND MEER, P. 2002. Synergism in low level vision. In Proc. ICPR 2002. Google Scholar
    6. COMANICIU, D., AND MEER, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 5. Google Scholar
    7. CURTIS, C. 1999. Non-photorealistic animation. In ACM SIGGRAPH 1999 Course Notes #17 (Section 9).Google Scholar
    8. DEUSSEN, O., AND STROTHOTTE, T. 2000. Computer-generated pen-and-ink illustration of trees. In Proc. of ACM SIGGRAPH 2000, 13-18. Google Scholar
    9. DUCHOWSKI, A., AND VERTEGAAL, R. 2000. Eye-based interaction in graphical systems: Theory and practice. In ACM SIGGRAPH 2000 Course Notes #5.Google Scholar
    10. DUCHOWSKI, A. 2000. Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. on Image Processing 9, 8 (Aug.), 1437-1440. Google Scholar
    11. DURAND, F., OSTROMOUKHOV, V., MILLER, M., DURANLEAU, F., AND DORSEY, J. 2001. Decoupling strokes and high-level attributes for interactive traditional drawing. In Proceedings of the 12th Eurographics Workshop on Rendering, 71-82. Google Scholar
    12. FINKELSTEIN, A., AND SALESIN, D. 1994. Multiresolution curves. In Proc. of ACM SIGGRAPH 94, 261-268. Google Scholar
    13. FOLEY, J., VAN DAM, A., FEINER, S., AND HUGHES, J. 1997. Computer Graphics: Principles and Practice, 2nd edition. Addison Wesley. Google Scholar
    14. GOOCH, B., AND GOOCH, A. 2001. Non-Photorealistic Rendering. A K Peters. Google Scholar
    15. GOOCH, A. A., GOOCH, B., SHIRLEY, P., AND COHEN, E. 1998. A non-photorealistic lighting model for automatic technical illustration. In Proc. of ACM SIGGRAPH 98, 447-452. Google Scholar
    16. HAEBERLI, P. 1990. Paint by numbers: Abstract image representations. In Proc. of ACM SIGGRAPH 90, 207-214. Google Scholar
    17. HANDFORD, M. 1987. Where’s Waldo? Little, Brown and Company.Google Scholar
    18. HENDERSON, J. M., AND HOLLINGWORTH, A. 1998. Eye movements during scene viewing: An overview. In Eye Guidance in Reading and Scene Perception, G. Underwood, Ed. Elsevier Science Ltd., 269-293.Google Scholar
    19. HERMAN, I., AND DUKE, D. 2001. Minimal graphics. IEEE Computer Graphics and Applications 21, 6, 18-21. Google Scholar
    20. HERTZMANN, A., AND ZORIN, D. 2000. Illustrating smooth surfaces. In Proc. of ACM SIGGRAPH 2000, 517-526. Google Scholar
    21. HERTZMANN, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proc. of ACM SIGGRAPH 98, 453-460. Google Scholar
    22. HERTZMANN, A. 2001. Paint by relaxation. In Computer Graphics International, 47-54. Google Scholar
    23. HOFFMAN, D. D. 1998. Visual intelligence: how we create what we see. Norton.Google Scholar
    24. JUST, M. A., AND CARPENTER, P. A. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 441-480.Google Scholar
    25. KELLY, D. 1984. Retinal inhomogenity: I. spatiotemporal contrast sensitivity. Journal of the Optical Society of America A 74, 1, 107-113.Google Scholar
    26. KOENDERINK, J. J., M. A. BOUMAN, A. B. D. M., AND SLAPPENDEL, S. 1978. Perimetry of contrast detection thresholds of moving spatial sine wave patterns. II. the far peripheral visual field (eccentricity 0-50). Journal of the Optical Society of America A 68, 6, 850-854.Google Scholar
    27. KOENDERINK, J. J. 1984. The structure of images. Biological Cybernetics 50, 363-370.Google Scholar
    28. KOENDERINK, J. J. 1984. What does the occluding contour tell us about solid shape? Perception 13, 321-330.Google Scholar
    29. KOWALSKI, M. A., MARKOSIAN, L., NORTHRUP, J. D., BOURDEV, L., BARZEL, R., HOLDEN, L. S., AND HUGHES, J. 1999. Art-based rendering of fur, grass, and trees. In Proc. of ACM SIGGRAPH 99, 433-438. Google Scholar
    30. LEYTON, M. 1992. Symmetry, causality, mind. MIT Press.Google Scholar
    31. LINDEBERG, T. 1994. Scale-Space Theory in Computer Vision. Kluwer Academic Publishers. Google Scholar
    32. LITWINOWICZ, P. 1997. Processing images and video for an impressionist effect. In Proc. of ACM SIGGRAPH 97, 407-414. Google Scholar
    33. MACKWORTH, N., AND MORANDI, A. 1967. The gaze selects informative details within pictures. Perception and Psychophysics 2, 547-552.Google Scholar
    34. MANNOS, J. L., AND SAKRISON, D. J. 1974. The effects of a visual fidelity criterion on the encoding of images. IEEE Trans. on Information Theory 20, 4, 525-536.Google Scholar
    35. MARKOSIAN, L., KOWALSKI, M. A., TRYCHIN, S. J., BOURDEV, L. D., GOLDSTEIN, D., AND HUGHES, J. F. 1997. Real-time nonphotorealistic rendering. In Proc. of ACM SIGGRAPH 97, 415-420. Google Scholar
    36. MARR, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman, San Francisco. Google Scholar
    37. MEER, P., AND GEORGESCU, B. 2001. Edge detection with embedded confidence. IEEE Trans. on Pattern Analysis and Machine Intelligence 23, 12, 1351-1365. Google Scholar
    38. PATTANAIK, S. N., FERWERDA, J. A., FAIRCHILD, M. D., AND GREENBERG, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. of ACM SIGGRAPH 98, 287-298. Google Scholar
    39. REDDY, M. 2001. Perceptually optimized 3D graphics. IEEE Computer Graphics and Applications 21, 5 (September/October), 68-75. Google Scholar
    40. REGAN, D. 2000. Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance, Color, Texture, Motion and Binocular Disparity. Sinauer.Google Scholar
    41. ROVAMO, J., AND VIRSU, V. 1979. An estimation and application of the human cortical magnification factor. Experimental Brain Research 37, 495-510.Google Scholar
    42. SAITO, T., AND TAKAHASHI, T. 1990. Comprehensible rendering of 3-D shapes. In Proc. of ACM SlGGRAPH 90, 197-206. Google Scholar
    43. SANTELLA, A., AND DECARLO, D. 2002. Abstracted painterly renderings using eye-tracking data. In Proc. of the Second International Symp. on Non-photorealistic Animation and Rendering (NPAR). Google Scholar
    44. SHIRAISHI, M., AND YAMAGUCHI, Y. 2000. An algorithm for automatic painterly rendering based on local source image approximation. In Proc. of the First International Symp. on Non-photorealistic Animation and Rendering (NPAR), 53-58. Google Scholar
    45. SIBERT, L. E., AND JACOB, R. J. K. 2000. Evaluation of eye gaze interaction. In Proc. CHI 2000, 281-288. Google Scholar
    46. TRUCCO, E., AND VERRI, A. 1998. Introductory Techniques for 3-D Computer Vision. Prentice-Hall. Google Scholar
    47. TUFTE, E. R. 1990. Envisioning Information. Graphics Press. Google Scholar
    48. VERTEGAAL, R. 1999. The gaze groupware system: Mediating joint attention in mutiparty communication and collaboration. In Proc. CHI ’99, 294-301. Google Scholar
    49. WINKENBACH, G., AND SALESIN, D. H. 1994. Computer-generated pen-and-ink illustration. In Proc. of ACM SIGGRAPH 94, 91-100. Google Scholar
    50. YARBUS, A. L. 1967. Eye Movements and Vision. Plenum Press.Google Scholar
    51. ZEKI, S. 1999. Inner Vision: An Exploration of Art and the Brain. Oxford Univ. Press.Google Scholar

ACM Digital Library Publication:

Overview Page: