“Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences” by Sawhney, Guo, Hanna, Kumar, Adkins, et al. …

This paper introduces a novel application of IBR technology for efficient rendering of high quality CG and live action stereoscopic sequences. Traditionally, IBR has been applied to render novel views using image and depth based representations of the plenoptic functions. In this work, we present a restricted form of IBR in which lower resolution images for the views to be generated at a very high resolution are assumed to be available. Specifically, the paper addresses the problem of synthesizing stereo IMAX(R)1 3D motion picture images at a standard resolution of 4-6K. At such high resolutions, producing CG content is extremely time consuming and capturing live action requires bulky cameras. We propose a Hybrid Stereo Camera concept in which one view is rendered at the target high resolution but the other is rendered at a much lower resolution. Methods for synthesizing the second view sequence at the target resolution using image analysis and IBR techniques are the focus of this work. The high quality results from the techniques presented in this paper have been visually evaluated in the IMAX 3D large screen projection environment. The paper also highlights generalizations and extensions of the hybrid stereo camera concept.

1. G. Adiv. Determining 3D motion and structure from optical flows generated by several moving objects. IEEE TPAMI, 7(4):384-401, 1985.
2. J. R. Bergen et al. Hierarchical model-based motion estimation. In Euro. Conf. on Comp. Vision, pages 237-252, 1992.
3. P. J. Burt and E. H. Adelson. The laplacian pyramid as a compact image code. IEEE TCOMM, 31(4):532-540, 1983.
4. K. J. Hanna and Neil E. Okamoto. Combining stereo and motion analysis for direct estimation of scene structure. In Proc. Intl. Conf. on Computer Vision, pages 357-365, 1993.
5. Phil Harman. Home based 3d entertainment – an overview. In IEEE Intl. Conf. on Image Processing, pages 1-4, 2000.
6. B.K.P. Horn. Robot Vision. MIT Press, 1986.
7. M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions. IJCV, 12:5-16, 1994.
8. B. Julesz. Foundations of Cyclopean Perception. The University of Chicago Press, 1971.
9. R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: A parallax based approach. In ICPR, pages 685-688, 1994.
10. B. D. Lucas and T. Kanade. An iterative image registration technique with an application in stereo vision. In IJCAI, 1981.
11. W. R. Mark, L. McMillan, and G. Bishop. Post-rendering 3d warping. In Symp. on Interactive 3D Graphics (Providence, RI), pages 7-16, 1997.
12. Y. Matsumoto et al. Conversion system of monocular image sequence to stereo using motion parallax. In SPIE Stereo. Disp. and VR Sys.(Vol 3012), 1997.
13. L. McMillan and G. Bishop. Plenoptic modeling: An imagebased rendering system. In SIGGRAPH, pages 39-46, 1995.
14. P.J. Narayanan, P.W. Rander, and T. Kanade. Constructing virtual worlds using dense stereo. In ICCV, pages 3-10, 1998.
15. Michael G. Perkins. Data compression of stereo pairs. IEEE Trans. on Comm., 40(4):684-696, 1992.
16. W. K. Pratt. Digital Image Processing (Second Edition).Wiley, 1991.
17. H. S. Sawhney. 3D geometry from planar parallax. In CVPR, pages 929-934, 1994.
18. A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In CVPR, pages 483-489, 1994.
19. L. Stelmach et al. Human perception of mismatched stereoscopic 3d stereo inputs. In ICIP, 2000.
20. R. Szeliski. Prediction error as a quality metric for motion and stereo. In Proc. Intl. Conf. on Computer Vision, 1999.
21. R. Szeliski and J. Coughlan. Spline-based image registration. IJCV, 22(3):199-218, 1997.
22. W.H.Press, B.P.Flannery, S.A.Teukolsky, and W.T.Vetterling. Numerical Recipes in C. Cambridge University Press, 1986.
23. G. Wolberg. Digital Image Warping. IEEE Computer Society Press, CA, 1990.