“Layered depth images” by Shade, Gortler, He and Szeliski

  • ©Jonathan Shade, Steven J. Gortler, Li-Wei He, and Richard Szeliski

Conference:


Type:


Title:

    Layered depth images

Presenter(s)/Author(s):


Abstract:


    In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan’s warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alpha-compositing can be done efficiently without depth sorting. This makes splatting an efficient solution to the resampling problem.

References:


    1. S. Baker, R. Szeliski, and P. Anandan. A Layered Approach to Stereo Reconstruction. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’98). Santa Barbara, June 1998.
    2. Shenchang Eric Chen and Lance Williams. View Interpolation for Image Synthesis. In James T. Kajiya, editor, Computer Graphics (SIG- GRAPH’93 Proceedings), volume 27, pages 279-288. August 1993.
    3. William Dally, Leonard McMillan, Gary Bishop, and Henry Fuchs. The Delta Tree: An Object Centered Approach to Image Based Rendering. AI technical Memo 1604, MIT, 1996.
    4. Lucia Darsa, Bruno Costa Silva, and Amitabh Varshney. Navigating Static Environments Using Image-Space Simplification and Morphing. In Proc. 1997 Symposium on Interactive 3D Graphics, pages 25-34. 1997.
    5. Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. Modeling and Rendering Architecture from Photographs: A Hybrid Geometryand Image-Based Approach. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 11-20. ACM SIGGRAPH, Addison Wesley, August 1996.
    6. O. Faugeras. Three-dimensional computer vision: A geometric viewpoint. MIT Press, Cambridge, Massachusetts, 1993.
    7. Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael E Cohen. The Lumigraph. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 43-54. ACM SIGGRAPH, Addison Wesley, August 1996.
    8. Paul S. Heckbert. Survey of Texture Mapping. IEEE Computer Graphics and Applications, 6(11):56-67, November 1986.
    9. Paul S. Heckbert and Henry P. Moreton. Interpolation for Polygon Texture Mapping and Shading. In David Rogers and Rae Earnshaw, editors, State of the Art in Computer Graphics: Visualization and Modeling, pages 101-111. Springer-Verlag, 1991.
    10. Youichi Horry, Ken ichi Anjyo, and Kiyoshi Arai. Tour Into the Picture: Using a Spidery Mesh Interface to Make Animation from a Single Image. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, pages 225-232. ACM SIGGRAPH, Addison Wesley, August 1997.
    11. R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: A parallax based approach. In Twelfth International Conference on Pattern Recognition (ICPR’94), volume A, pages 685- 688. IEEE Computer Society Press, Jerusalem, Israel, October 1994.
    12. Anthony G. LaMarca. Caches and Algorithms. Ph.D. thesis, University of Washington, 1996.
    13. S. Laveau and O. D. Faugeras. 3-D Scene Representation as a Collection of Images. In Twelfth International Conference on Pattern Recognition (ICPR’94), volume A, pages 689-691. IEEE Computer Society Press, Jerusalem, Israel, October 1994.
    14. Jed Lengyel and John Snyder. Rendering with Coherent Layers. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, pages 233-242. ACM SIGGRAPH, Addison Wesley, August 1997.
    15. Marc Levoy and Pat Hanrahan. Light Field Rendering. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 31-42. ACM SIGGRAPH, Addison Wesley, August 1996.
    16. Mark Levoy and Turner Whitted. The Use of Points as a Display Primitive. Technical Report 85-022, University of North Carolina, 1985.
    17. William R. Mark, Leonard McMilland, and Gary Bishop. Post- Rendering 3D Warping. In Proc. 1997 Symposium on Interactive 3D Graphics, pages 7-16. 1997.
    18. Nelson Max. Hierarchical Rendering of Trees from Precomputed Multi-Layer Z-Buffers. In Xavier Pueyo and Peter Schr6der, editors, Eurographics Rendering Workshop 1996, pages 165-174. Eurographics, Springer Wein, New York City, NY, June 1996.
    19. Leonard McMillan. Computing Visibility Without Depth. Technical Report 95-047, University of North Carolina, 1995.
    20. Leonard McMillan. A List-Priority Rendering Algorithm for Redisplaying Projected Surfaces. Technical Report 95-005, University of North Carolina, 1995.
    21. Leonard McMillan and Gary Bishop. Plenoptic Modeling: An Image- Based Rendering System. In Robert Cook, editor, SIGGRAPH 95 Conference Proceedings, Annual Conference Series, pages 39-46. ACM SIGGRAPH, Addison Wesley, August 1995.
    22. Leonard McMillan and Gary Bishop. Shape as a Pertebation to Projective Mapping. Technical Report 95-046, University of North Carolina, 1995.
    23. Don E Mitchell. personal communication. 1997.
    24. Matt Pharr, Craig Kolb, Reid Gershbein, and Pat Hanrahan. Rendering Complex Scenes with Memory-Coherent Ray Tracing. In Turner Whitted, editor, SIGGRAPH 97 Conference Proceedings, Annual Conference Series, pages 101-108. ACM SIGGRAPH, Addison Wesley, August 1997.
    25. H. S. Sawhney. 3D Geometry from Planar Parallax. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’94), pages 929-934. IEEE Computer Society, Seattle, Washington, June 1994.
    26. Gernot Schaufler and Wolfgang Stfirzlinger. A Three-Dimensional Image Cache for Virtual Reality. In Proceedings of Eurographics ’96, pages 227-236. August 1996.
    27. Mark Segal, Carl Korobkin, Roll van Widenfelt, Jim Foran, and Paul E. Haeberli. Fast shadows and lighting effects using texture mapping. In Edwin E. Catmull, editor, Computer Graphics (SIGGRAPH ’92 Proceedings), volume 26, pages 249-252. July 1992.
    28. StevenM. Seitz and Charles R. Dyer. View Morphing: Synthesizing 3D Metamorphoses Using Image Transforms. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 21-30. ACM SIGGRAPH, Addison Wesley, August 1996.
    29. Steven M. seitz and Charles R. Dyer. Photorealistic Scene Reconstruction by Voxel Coloring. In Proc. Computer Vision and Pattern Recognition Conf., pages 1067-1073. 1997.
    30. Jonathan Shade, Dani Lischinski, David Salesin, Tony DeRose, and John Snyder. Hierarchical Image Caching for Accelerated Walkthroughs of Complex Environments. In Holly Rushmeier, editor, SIG- GRAPH 96 Conference Proceedings, Annual Conference Series, pages 75-82. ACM SIGGRAPH, Addison Wesley, August 1996.
    31. Jay Torborg and Jim Kajiya. Talisman: Commodity Real-time 3D Graphics for the PC. In Holly Rushmeier, editor, SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 353-364. ACM SIGGRAPH, Addison Wesley, August 1996.
    32. J.Y.A. Wang and E. H. Adelson. Layered Representation for Motion Analysis. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’ 93), pages 361-366. New York, New York, June 1993.
    33. Lee Westover. Footprint Evaluation for Volume Rendering. In Forest Baskett, editor, Computer Graphics (SIGGRAPH ’90 Proceedings), volume 24, pages 367-376. August 1990.
    34. G. Wolberg. Digital Image Warping. IEEE Computer Society Press, Los Alamitos, California, 1990.

ACM Digital Library Publication:


Overview Page: