“Plenoptic sampling” by Chai, Tong, Chan and Shum

This paper studies the problem of plenoptic sampling in image-based rendering (IBR). From a spectral analysis of light field signals and using the sampling theorem, we mathematically derive the analytical functions to determine the minimum sampling rate for light field rendering. The spectral support of a light field signal is bounded by the minimum and maximum depths only, no matter how complicated the spectral support might be because of depth variations in the scene. The minimum sampling rate for light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval. Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering.
Plenoptic sampling goes beyond the minimum number of images needed for anti-aliased light field rendering. More significantly, it utilizes the scene depth information to determine the minimum sampling curve in the joint image and geometry space. The minimum sampling curve quantitatively describes the relationship among three key elements in IBR systems: scene complexity (geometrical and textural information), the number of image samples, and the output resolution. Therefore, plenoptic sampling bridges the gap between image-based rendering and traditional geometry-based rendering. Experimental results demonstrate the effectiveness of our approach.

1. E.H. Adelson and J. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, MA, 1991.
2. M. Bass, editor. Handbook of Optics. McGraw-Hill, New York, 1995.
3. R.C. Bolles, H. H. Baker, and D. H. Marimont. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7-55, 1987.
4. E. Camahort, A. Lerios, and D. Fussell. Uniformly sampled light fields. In Proc. 9th Eurographics Workshop on Rendering, pages 117-130, 1998.
5. J.-X. Chai and H.-Y. Shum. Parallel projections for stereo reconstruction. In Proc. CVPR 2000, 2000.
6. C. Chang, G. Bishop, and A. Lastra. Ldi tree: A hierarchical representation for image-based rendering. SIGGRAPH’99, pages 291-298, August 1999.
7. S. Chen and L. Williams. View interpolation for image synthesis. Computer Graphics (SIGGRAPH’93), pages 279-288, August 1993.
8. T. Feng and H.-Y. Shum. An optical analysis of light field rendering. Technical report, Microsoft Research, MSR-TR-2000-38, May 2000.
9. B. Girod. Motion compensation: visual aspects, accuracy, and fundamental limits. In Motion Analysis and Image Sequence Processing. Kluwer, 1995.
10. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, pages 43-54, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
11. M. Halle. Holographic stereograms as discrete imaging systems. In Proc. SPIE Vol.2176, Practical Holography VIII, pages 73-84, May 1994.
12. V. Hlavac, A. Leonardis, and T. Werner. Automatic selection of reference views for image-based scene representations. In Proc. ECCV, pages 526-535, 1996.
13. A. Isaksen, L. McMillan, and S. Gortler. Dynamically reparameterized light fields. Technical report, Technical Report MIT-LCS-TR-778, May 1999.
14. S. Kang. A survey of image-based rendering techniques. In VideoMetrics, SPIE Vol. 3641, pages 2-16, 1999.
15. J. Lengyel. The convergence of graphics and vision. Technical report, IEEE Computer, July 1998.
16. M. Levoy and R Hanrahan. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, pages 31-42, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
17. Z.-C. Lin and H.-Y. Shum. On the numbers of samples needed in light field rendering with constant-depth assumption. In Proc. CVPR 2000, 2000.
18. W. Mark, L. McMillan, and G. Bishop. Post-rendering 3d warping. In Proc. Symposium on I3D Graphics, pages 7-16, 1997.
19. L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH’95), pages 39-46, August 1995.
20. H. Schirmacher, W. Heidrich, and H. Seidel. Adaptive acquisition of lumigraphs from synthetic scenes. In Eurographics ’99, pages 151-159, Sept 1999.
21. S. M. Seitz and C. M. Dyer. View morphing. In Computer Graphics Proceedings, Annual Conference Series, pages 21-30, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
22. J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Computer Graphics (SIGGRAPH’98) Proceedings, pages 231-242, Orlando, July 1998. ACM SIGGRAPH.
23. H.-Y. Shum and L.-W. He. Rendering with concentric mosaics. In Proc. SIG- GRAPH 99, pages 299-306, 1999.
24. P. R Sloan, M. F. Cohen, and S. J. Gortler. Time critical lumigraph rendering. In Symposium on Interactive 3D Graphics, pages 17-23, Providence, RI, USA, 1997.
25. A. Tekal. Digital Video Processing. Prentice Hall, 1996.