“Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach” by Debevec, Taylor and Malik

We present a new approach for modeling and rendering existing architectural scenes from a sparse set of still photographs. Our modeling approach, which combines both geometry-based and image-based techniques, has two components. The first component is a photogrammetric modeling method which facilitates the recovery of the basic geometry of the photographed scene. Our photogrammetric modeling approach is effective, convenient, and robust because it exploits the constraints that are characteristic of architectural scenes. The second component is a model-based stereo algorithm, which recovers how the real scene deviates from the basic model. By making use of the model, our stereo technique robustly recovers accurate depth from widely-spaced image pairs. Consequently, our approach can model large architectural environments with far fewer photographs than current image-based modeling approaches. For producing renderings, we present view-dependent texture mapping, a method of compositing multiple views of a scene that better simulates geometric detail on basic models. Our approach can be used to recover models for use in either geometry-based or image-based rendering systems. We present results that demonstrate our approach’s ability to create realistic renderings of architectural scenes from viewpoints far from the original photographs.

1. Ali Azarbayejani and Alex Pentland. Recursive estimation of motion, structure, and focal length. IEEE Trans. Pattern Anal. Machine Intell., 17 (6):562-575, June 1995.
2. H.H. Baker and T. O. Binford. Depth from edge and intensity based stereo. In Proceedings of the Seventh IJCAI, Vancouvel; BC, pages 631-636,1981.
3. Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Technical Report UCB//CSD-96-893, U.C. Berkeley, CS Division, January 1996.
4. D.J.Fleet, A.D.Jepson, and M.R.M. Jenkin. Phase-based disparity measurement. CVGIP: Image Understanding, 53 (2): 198-210,1991.
5. Oliver Faugeras and Giorgio Toscani. The calibration problem for stereo. In P1vceedings lEEE CVPR 86, pages 15-20,1986.
6. Olivier Faugeras. Three-Dimensional Computer Vision. MIT Press, 1993.
7. Olivier Faugeras, Stephane Laveau, Luc Robert, Gabriella Csurka, and Cyril Zeller. 3-d reconstruction of urban scenes from sequences of images. Technical Report 2572, INRIA, June 1995.
8. W.E.L. Grimson. Firm Images to Sulface. MIT Press, 1981.
9. D. Jones and J. Malik. Computational framework for determining stereo correspondence from a set of linear spatial filters. Image and Vision Computing, 10(10) :699-708, December 1992.
10. E. Kruppa. Zur ermittlung eines objectes aus zwei perspektiven mit innerer orientierung. Sitz.-Be~: Akad. Wiss., Wien, Math. Naturw. Kl., Abt. Ila., 122:1939- 1948, 1913.
11. H.C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133-135, September 1981.
12. D. Marr and T. Poggio. A computationaltheory of human stereo vision. P~vceedings of the Royal Society of London, 204:301-328,1979.
13. Leonard McMillan and Gary Bishop. Plenoptic modeling: An image-based rendering system. In SIGGRAPH’ 95, 1995.
14. Eric N. Mortensen and William A. Barrett. Intelligent scissors for image composition. In SIGGRAPH’ 95, 1995.
15. S.B. Pollard, J. E. W. Mayhew, and J. R Frisby. A stereo correspondence algorithm using a disparity gradient limit. Perception, 14:449-470,1985.
16. R. Szeliski. Image mosaicing for tele-reality applications. In IEEE Computer Graphics and Applications, 1996.
17. Camillo J. Taylor and David J. Kriegman. Structure and motion from line segments in multiple images. IEEE Trans. Pattern Anal. Machine Intell., 17(11), November 1995.
18. S.J. Teller, Celeste Fowler, Thomas Funkhouser, and Pat Hanrahan. Partitioning and ordering large radiosity computations. In SIGGRAPH ’94, pages 443-450, 1994.
19. Carlo Tomasi and Takeo Kanade. Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9(2):137-154, November 1992.
20. Roger Tsai. A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal of Robotics and Automation, 3 (4):323-344, August 1987.
21. S. Ullman. The Interpretation of Visual Motion. The MIT Press, Cambridge, MA, 1979.
22. L Williams. Casting curved shadows on curved surfaces. In SIGGRAPH’78, pages 270-274,1978.
23. Lance Williams and Eric Chen. View interpolation for image synthesis. In SIG- GRAPH ’93, 1993.
24. Mourad Zerroug and Ramakant Nevatia. Segmentation and recovery of shgcs from a real intensity image. In Eu~vpean Conference on Computer Vision, pages 319-330,1994.