“Non-photorealistic virtual environments” by Klein, Li, Kazhdan, Correa, Finkelstein, et al. …

We describe a system for non-photorealistic rendering (NPR) of virtual environments. In real time, it synthesizes imagery of architectural interiors using stroke-based textures. We address the four main challenges of such a system — interactivity, visual detail, controlled stroke size, and frame-to-frame coherence — through image based rendering (IBR) methods. In a preprocessing stage, we capture photos of a real or synthetic environment, map the photos to a coarse model of the environment, and run a series of NPR filters to generate textures. At runtime, the system re-renders the NPR textures over the geometry of the coarse model, and it adds dark lines that emphasize creases and silhouettes. We provide a method for constructing non-photorealistic textures from photographs that largely avoids seams in the resulting imagery. We also offer a new construction, art-maps, to control stroke size across the images. Finally, we show a working system that provides an immersive experience rendered in a variety of NPR styles.

1. Buck, I., Finkelstein, A., Jacobs, C., Klein, A., Salesin, D. H., Seims, J., Szeliski, R., and Toyama, K. Performance-driven hand-drawn animation. Proceedings of NPAR 2000 (June 2000).
2. Chen, S. E. Quicktime VR – An image-based approach to virtual en-vironment navigation. Computer Graphics (SIGGRAPH 95), 29-38.
3. Chen, S. E., and Williams, L. View interpolation for image synthesis. Computer Graphics (SIGGRAPH 93), 279-288.
4. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. Computer-generated watercolor. Computer Graphics (SIGGRAPH 97), 421-430.
5. Debevec, P. Image-based modeling, rendering, and lighting. Course #35, SIGGRAPH 2000 Course Notes (July 2000).
6. Debevec, P. E., Taylor, C. J., and Malik, J. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Computer Graphics (SIGGRAPH 96), 11-20.
7. Debevec, P. E., Yu, Y., and Borshukov, G. D. Efficient view-dependent image-based rendering with projective texture-mapping. Eurographics Rendering Workshop (June 1998), 105-116.
8. Efros, A. A., and Leung, T. K. Texture synthesis by non-parametric sampling. IEEE International Conference on Computer Vision (1999).
9. Funkhouser, T. A. A visibility algorithm for hybrid geometry- and image-based modeling and rendering. Computers and Graphics, Special Issue on Visibility (1999).
10. Funkhouser, T. A., Teller, S. J., Sequin, C. H., and Khorramabadi, D. The UC Berkeley system for interactive visualization of large architectural models. Presence 5, 1 (January 1996).
11. Gooch, A., Gooch, B., Shirley, P., and Cohen, E. A non-photorealistic lighting model for automatic technical illustration. Computer Graph-ics (SIGGRAPH 98), 447-452.
12. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. The lumigraph. Computer Graphics (SIGGRAPH 96), 43-54.
13. Haeberli, P. E. Paint by numbers: Abstract image representations. Computer Graphics (SIGGRAPH 90), 207-214.
14. Heckbert, P. Survey of texture mapping. IEEE Computer Graphics and Applications (Nov. 1986).
15. Hertzmann, A. Painterly rendering with curved brush strokes of multiple sizes. Computer Graphics (SIGGRAPH 98), 453-460.
16. Hewlett Packard. HP PEX Texture Mapping, www.hp.com/mhm/WhitePapers/PEXtureMapping/PEXtureMapping.html.
17. Horry, Y., ichi Anjyo, K., and Arai, K. Tour into the picture: Using a spidery mesh interface to make animation from a single image. Computer Graphics (SIGGRAPH 97), 225-232.
18. Kowalski, M. A., Markosian, L., Northrup, J. D., Bourdev, L., Barzel, R., Holden, L. S., and Hughes, J. Art-based rendering of fur, grass, and trees. Computer Graphics (SIGGRAPH 99), 433-438.
19. Levoy, M., and Hanrahan, P. Light field rendering. Computer Graphics (SIGGRAPH 96), 31-42.
20. Litwinowicz, P. Processing images and video for an impressionist effect. Computer Graphics (SIGGRAPH 97), 407-414.
21. Manocha, D. Interactive walkthroughs of large geometric databases. Course #18, SIGGRAPH 2000 Course Notes (July 2000).
22. Markosian, L., Kowalski, M. A., Trychin, S. J., Bourdev, L. D., Gold-stein, D., and Hughes, J. F. Real-time nonphotorealistic rendering. Computer Graphics (SIGGRAPH 97), 415-420.
23. McMillan, L., and Bishop, G. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH 95), 39-46.
24. Meier, B. J. Painterly rendering for animation. Computer Graphics (SIGGRAPH 96), 477-484.
25. Mizuno, S., Okada, M., and ichiro Toriwaki, J. Virtual sculpting and virtual woodcut printing. The Visual Computer 14, 2 (1998), 39-51.
26. Ostromoukhov, V. Digital facial engraving. Computer Graphics (SIGGRAPH 99), 417-424.
27. Saito, T., and Takahashi, T. NC machining with G-buffer method. Computer Graphics (SIGGRAPH 91), 207-216.
28. Salisbury, M. P., Wong, M. T., Hughes, J. F., and Salesin, D. H. Ori-entable textures for image-based pen-and-ink illustration. Computer Graphics (SIGGRAPH 97), 401-406.
29. Shade, J. W., Gortler, S. J., wei He, L., and Szeliski, R. Layered depth images. Computer Graphics (SIGGRAPH 98), 231-242.
30. Tsai, R. Y. A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses. IEEE Journal of Robotics and Automation 3, 4 (Aug. 1987), 323-344.
31. Williams, L. Pyramidal parametrics. Computer Graphics (SIG-GRAPH 83), 1-11.
32. Winkenbach, G., and Salesin, D. H. Computer-generated pen-and-ink illustration. Computer Graphics (SIGGRAPH 94), 91-100.
33. Winkenbach, G., and Salesin, D. H. Rendering parametric surfaces in pen and ink. Computer Graphics (SIGGRAPH 96), 469-476.
34. Wong, M. T., Zongker, D. E., and Salesin, D. H. Computer-generated floral ornament. Computer Graphics (SIGGRAPH 98), 423-434.
35. Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. Multiperspective panoramas for cel animation. Computer Graphics (SIGGRAPH 97), 243-250.