“Rendering with concentric mosaics” by Shum and He

This paper presents a novel 3D plenoptic function, which we call concentric mosaics. We constrain camera motion to planar concentric circles, and create concentric mosaics using a manifold mosaic for each circle (i.e., composing slit images taken at different locations). Concentric mosaics index all input image rays naturally in 3 parameters: radius, rotation angle and vertical elevation. Novel views are rendered by combining the appropriate captured rays in an efficient manner at rendering time. Although vertical distortions exist in the rendered images, they can be alleviated by depth correction. Like panoramas, concentric mosaics do not require recovering geometric and photometric scene models. Moreover, concentric mosaics provide a much richer user experience by allowing the user to move freely in a circular region and observe significant parallax and lighting changes. Compared with a Lightfield or Lumigraph, concentric mosaics have much smaller file size because only a 3D plenoptic function is constructed. Concentric mosaics have good space and computational efficiency, and are very easy to capture. This paper discusses a complete working system from capturing, construction, compression, to rendering of concentric mosaics from synthetic and real environments.

1. E.H. Adelson and J. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, MA, 1991.
2. S.A. Benton. Survey of holographic stereograms. In Proc. SPIE Vol. 367Int. Soc. Eng., pages 15-19, 1983.
3. R Beardsley, R Tom and A. Zisserman. 3D model acquisition from extended image sequences. In Fourth European Conference on Computer Vision (ECCV’96), volume 2, pages 683-695, Cambridge, England, April 1996. Springer-Verlag.
4. S.E. Chen. QuickTime VR – an image-based approach to virtual environment navigation. Computer Graphics (SIGGRAPIt’95), pages 29-38, August 1995.
5. M.F. Cohen. Personal email communication, September 1997.
6. S. Chen and L. Williams. View interpolation for image synthesis. Computer Graphics (SIGGRAPIt’93), pages 279-288, August 1993.
7. R E. Debevec, C. J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Computer Graphics (SIGGRAPH’96), pages 11-20, August 1996.
8. O. Faugeras. Three-dimensional computer vision: A geometric viewpoint. MIT Press, Cambridge, Massachusetts, 1993.
9. O.D. Faugeras, Laveau S., Robert L., Csurka G., and Zeller C. 3-D reconstruction of urban scenes from sequences of images. Computer Vision and Image Understanding, 69(3):292-309, March 1998.
10. S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, pages 43-54, Proc. SIGGRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
11. R. Gupta and R.I. Hartley. Linear pushbroom cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(9):963-975, September 1997.
12. S.B. Kang and R. Szeliski. 3-D scene data recovery using omnidirectional multibaseline stereo. In IEEE Computer Society ConJerence on Computer Vision and Pattern Recognition (CVPR’96), pages 364-370, San Francisco, California, June 1996.
13. S. Laveau and O. Faugeras. 3-D scene representation as a collection of images and fundamental matrices. Technical Report 2205,1NRIA- Sophia Antipolis, February 1994.
14. M. Levoy and R Hanrahan. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, pages 31-42, Proc. SIG- GRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
15. L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPIt’95), pages 39-46, August 1995.
16. S. Mann and R. W. Picard. Virtual bellows: Constructing high-quality images from video. In First IEEE International Conference on Image Processing (ICIP-94), volume I, pages 363-367, Austin, Texas, November 1994.
17. V.S. Nalwa. A true omnidirecional viewer. Technical report, Bell Laboratories, Holmdel, NJ, USA, February 1996.
18. S. Nayar. Catadioptric omnidirectional camera. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), pages 482-488, San Juan, Puerto Rico, June 1997.
19. S. Peleg and M. Ben-Ezra. Stereo panorama with a single camera. In Proc. Computer Vision and Pattern Recognition Conf., 1999.
20. S. Peleg and J. Herman. Panoramic mosaics by manifold projection. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’97), pages 338-343, San Juan, Puerto Rico, June 1997.
21. R Rademacher and Bishop G. Multiple-center-of-projection images. In Computer Graphics Proceedings, Annual Conference Series, pages 199-206, Proc. SIGGRAPH’98 (Orlando), July 1998. ACM SIG- GRAPH.
22. B. Rousso, S. Peleg, Finci I., and Rav-Acha A. Universal mosaicing using pipe projection. In Sixth International Conference on Computer Vision (ICCV’98), pages 945-952, Bombay, January 1998.
23. R R Sloan, M. F. Cohen, and S. J. Gortler. Time critical lumigraph rendering. In Symposium on Interactive 3D Graphics, pages 17-23, Providence, RI, USA, 1997.
24. S.M. Seitz and C. M. Dyer. View morphing. In Computer Graphics Proceedings, Annual Conference Series, pages 21-30, Proc. SIG- GRAPH’96 (New Orleans), August 1996. ACM SIGGRAPH.
25. J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Computer Graphics (SIGGRAPH’98) Proceedings, pages 231-242, Orlando, July 1998. ACM SIGGRAPH.
26. R. Szeliski and H.-Y. Shum. Creating full view panoramic image mosaics and texture-mapped models. Computer Graphics (SIG- GRAPIt’97), pages 251-258, August 1997.
27. Y. Sato, M. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. In Computer Graphics Proceedings, Annual Conference Series, pages 379-387, Proc. SIGGRAPH’97 (Los Angeles), August 1997. ACM SIGGRAPH.
28. R. Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, pages 22-30, March 1996.
29. D.N. Wood et al. Multiperspective panoramas for cel animation. In Computer Graphics Proceedings, Annual Conference Series, pages 243-250, Proc. SIGGRAPH’97 (Los Angeles), August 1997. ACM SIGGRAPH.
30. T. Wong, P. Heng, S. Or, and W. Ng. Image-based rendering with controllable illumination. In Proceedings of the 8-th Eurographics Workshop on Rendering, pages 13-22, St. Etienne, France, June 1997.
31. Y. Yu and J. Malik. Recovering photometric properties of architectural scenes from photographs. Computer Graphics (SIGGRAPIt’96), pages 207-218, July 1998.
32. J.Y. Zheng and S. Tsuji. Panoramic representation of scenes for route understanding. In Proc. of the lOth Int. Conf. Pattern Recognition, pages 161-167, June 1990.