“Rendering with coherent layers” by Lengyel and Snyder

For decades, animated cartoons and movie special effects have factored the rendering of a scene into layers that are updated independently and composed in the final display. We apply layer factorization to real-time computer graphics. The layers allow targeting of resources, whether the ink and paint artists of cartoons or the graphics pipeline as described here, to those parts of the scene that are most important. To take advantage of frame-to-frame coherence, we generalize layer factorization to apply to both dynamic geometric objects and terms of the shading model, introduce new ways to trade off fidelity for resource use in individual layers, and show how to compute warps that reuse renderings for multiple frames. We describe quantities, called fiducials, that measure the fidelity of approximations to the original image. Layer update rates, spatial resolution, and other quality parameters are determined by geometric, photometric, visibility, and sampling fiducials weighted by the content author’s preferences. We also compare the fidelity of various types of reuse warps and demonstrate the suitability of the affine warp. Using Talisman, a hardware architecture with an efficient layer primitive, the work presented here dramatically improves the geometric complexity and shading quality of scenes rendered in real-time.

1. Consider the Lowly 2×2 Matrix, Jim Blinn, IEEE Computer Graphics and Applications, March 1996, pp. 82-88.
2. View Interpolation for Image Synthesis, Shenchang Eric Chen and Lance Williams, SIGGRAPH 93, pp. 279-288.
3. QuickTime VR – An Image-Based Approach to Virtual Environment Navigation, Shenchang Eric Chen, SIGGRAPH 95, pp. 29-38.
4. Shade Trees, Robert L. Cook, SIGGRAPH 94, pp. 223-232.
5. Interactive Design of Complex Time Dependent Lighting, Julie Dorsey, Jim Arvo, Donald P. Greenberg, IEEE Computer Graphics and Application, March 1995, Volume 15, Number 2, pp. 26-36.
6. Adaptive Display Algorithm for Interactive Frame Rates During Visualization of Complex Virtual Environments, Thomas A. Funkhouser and Carlo H. S6quin, SIGGRAPH 93, pp. 247-254.
7. Practical Optimization, Philip E. Gill, Walter Murray, and Margaret H. Wright, Academic Press, London, 1981.
8. Hierarchical Z-Buffer Visibility, Ned Greene, Michael Kass, Gavin Miller, SIGGRAPH 93, pp. 231-238.
9. Error-Bounded Antialiased Rendering of Complex Environments, Ned Greene and Michael Kass, SIGGRAPH 94, pp. 59-66.
10. Specializing Shaders, Brian Guenter, Todd B. Knoblock, and Erik Rut, SIGGRAPH 95, pp. 343-350.
11. A Language for Shading and Lighting Calculations, Pat Hanrahan and Jim Lawson, SIGGRAPH 90, pp. 289-298.
12. The Calculus of the Non-Exact Perspective Projection, Scene-Shitting for Computer Animation. Georg Rainer Hofmann. Tutorial Notes for Computer Animation, Eurographics ’89.
13. Flexible Rendering of 3D Graphics Under Varying Resources: Issues and Directions, Eric Horvitz and Jed Lengyel, In Symposium on Flexible Computation, AAAI Notes FS-96-06, pp. 81-88. Also available as Microsoft Technical Report MSR-TR-96-18.
14. Decision-Theoretic Regulation of Graphics Rendering, Eric Horvitz and Jed Lengyel, In Thirteenth Conference on Uncertainty in Artificial Intelligence, D. Geiger and P. Shenoy, eds., August 1997.
15. Frame to Frame Coherence and the Hidden Surface Computation: Constraints for a Convex World, Harold Hubschman, and Steven W. Zucker, SIGGRAPH 81, pp. 45-54.
16. Ray Tracing Complex Scenes, Timothy L. Kay and James T. Kajiya, SIGGRAPH 86, pp. 269-278.
17. Visual Navigation of Large Environments Using Textured Clusters, Paolo W. C. Maciel and Peter Shirley, Proceedings 1995 Symposium on Interactive 3D Graphics, April 1995, pp. 95-102.
18. Post-Rendering 3D Warping, William R. Mark, Leonard McMillan, and Gary Bishop, Proceedings 1997 Symposium on Interactive 3D Graphics, April 1997, pp. 7-16.
19. Painterly Rendering for Animation, Barbara J. Meier, SIGGRAPH 96, pp. 477-484.
20. PixelFlow: High-Speed Rendering Using Image Composition, Steve Molnar, John Eyles, and John Poulton, SIGGRAPH 92, pp. 231-240.
21. Plenoptic Modeling: An Image-Based Rendering System, Leonard McMillan, Gary Bishop, SIGGRAPH 95, pp. 39-46.
22. Compositing Digital Images, Thomas Porter and Tom Duff, SIG- GRAPH 84, pp. 253-259.
23. Priority Rendering with a Virtual Reality Address Recalculation Pipeline, Matthew Regan and Ronald Pose, SIGGRAPH 94, pp. 155-162.
24. Fast Shadows and Lighting Effects Using Texture Mapping, Mark Segal, Carl Korobkin, Rolf van Widenfelt, Jim Foran, Paul Haeberli, SIG- GRAPH 92, pp. 249-252.
25. Exploiting Frame to Frame Coherence in a Virtual Reality System, Gernot Schaufler, Proceedings of VRAIS ’96, April 1996, pp. 95-102.
26. A Three Dimensional Image Cache for Virtual Reality, Gernot Schaufler and Wolfgang Sttirzlinger, Proceedings of Eurographics ’96, August 1996, pp. 227-235.
27. Hierarchical Image Caching for Accelerated Walkthroughs of Complex Environments, Jonathan Shade, Dani Lischinski, David Salesin, Tony DeRose, John Snyder, SIGGRAPH 96, pp. 75-82.
28. Path Specification and Path Coherence, Kim L. Shelley and Donald P. Greenberg, SIGGRAPH 82, pp. 157-166.
29. Visibility Sorting for Dynamic, Aggregate Geometry, John Snyder, Microsoft Technical Report, MSR-TR-97-11.
30. Talisman: Commodity Real-time 3D Graphics for the PC, Jay Torborg, Jim Kajiya, SIGGRAPH 96, pp. 353-364.
31. Feature Matching and Affine Transformation for 2D Cell Animation, Ming Xie, Visual Computer, Volume 11, 1995, pp. 419-428.