“Image-based procedural modeling of facades” by Müller, Zeng, Wonka and Gool

This paper describes algorithms to automatically derive 3D models of high visual quality from single facade images of arbitrary resolutions. We combine the procedural modeling pipeline of shape grammars with image analysis to derive a meaningful hierarchical facade subdivision. Our system gives rise to three exciting applications: urban reconstruction based on low resolution oblique aerial imagery, reconstruction of facades based on higher resolution ground-based imagery, and the automatic derivation of shape grammar rules from facade images to build a rule base for procedural modeling technology.

1. Alegre, F., and Dellaert, F. 2004. A probabilistic approach to the semantic interpretation of building facades. In International Workshop on Vision Techniques Applied to the Rehabilitation of City Centres.Google Scholar
2. Bekins, D., and Aliaga, D. 2005. Build-by-number: Rearranging the real world to visualize novel architectural spaces. In IEEE Visualization.Google Scholar
3. Brenner, C., and Ripperda, N. 2006. Extraction of facades using rjM-CMC and constraint equations. In Photogrammetric Computer Vision, 155–160.Google Scholar
4. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings of ACM SIGGRAPH 96, ACM Press, H. Rushmeier, Ed., 11–20. Google ScholarDigital Library
5. Dick, A., Torr, P., Ruffle, S., and Cipolla, R. 2001. Combining single view recognition and multiple view stereo for architectural scenes. In ICCV, IEEE Computer Society, Los Alamitos, CA, 268–274.Google Scholar
6. Dick, A. R., Torr, P. H. S., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. International Journal of Computer Vision 60, 2, 111–134. Google ScholarDigital Library
7. Duarte, J. 2002. Malagueira Grammar — towards a tool for customizing Alvaro Siza’s mass houses at Malagueira. PhD thesis, MIT School of Architecture and Planning.Google Scholar
8. Flemming, U. 1987. More than the sum of its parts: the grammar of queen anne houses. Environment and Planning B 14, 323–350.Google ScholarCross Ref
9. Früh, C., and Zakhor, A. 2001. 3D model generation for cities using aerial photographs and ground level laser scans. In CVPR, IEEE Computer Society, 31–38.Google Scholar
10. Früh, C., and Zakhor, A. 2003. Constructing 3d city models by merging ground-based and airborne views. Computer Graphics and Applications (Nov.), 52–61. Google ScholarDigital Library
11. Han, F., and Zhu, S.-C. 2005. Bottom-up/top-down image parsing by attribute graph grammar. In ICCV, IEEE Computer Society, Washington, DC, USA, 1778–1785. Google ScholarDigital Library
12. Havemann, S. 2005. Generative Mesh Modeling. PhD thesis, TU Braunschweig.Google Scholar
13. Hu, J., You, S., and Neumann, U. 2006. Integrating lidar, aerial image and ground images for complete urban building modeling. In 3DPVT. Google ScholarDigital Library
14. Jepson, W., Liggett, R., and Friedman, S. 1996. Virtual modeling of urban environments. PRESENCES 5, 1, 72–86.Google ScholarDigital Library
15. Jojic, N., Frey, B., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In ICCV, IEEE Computer Society, 34–41. Google ScholarDigital Library
16. Karner, K., Bauer, J., Klaus, A., Leberl, F., and Grabner, M. 2001. Virtual habitat: Models of the urban outdoors. In Third International Workshop on Automatic Extraction of Man-Made Objects from Aerial and Space Imaging, 393–402.Google Scholar
17. Koning, H., and Eizenberg, J. 1981. The language of the prairie: Frank lloyd wrights prairie houses. Environment and Planning B 8, 295–323.Google ScholarCross Ref
18. Kullback, S. 1959. Information theory and statistics. John Wiley and Sons., New York.Google Scholar
19. Lee, S. C., and Nevatia, R. 2003. Interactive 3D building modeling using a hierarchical representation. In International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, IEEE Computer Society, 58–65. Google ScholarDigital Library
20. Lee, S. C., and Nevatia, R. 2004. Extraction and integration of window in a 3D building model from ground view image. In CVPR, IEEE Computer Society, 113–120. Google ScholarDigital Library
21. Lee, S. C., Jung, S. K., and Nevatia, R. 2002. Automatic integration of facade textures into 3D building models with a projective geometry based line clustering. Computer Graphics Forum 21, 3 (Sept.), 511–519.Google ScholarCross Ref
22. Legakis, J., Dorsey, J., and Gortler, S. J. 2001. Feature-based cellular texturing for architectural models. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 309–316. Google ScholarDigital Library
23. Liebowitz, D., and Zisserman, A. 1998. Metric rectification for perspective images of planes. In CVPR, IEEE Computer Society, 482–488. Google ScholarDigital Library
24. Marvie, J.-E., Perret, J., and Bouatouch, K. 2005. The FL-system: a functional L-system for procedural geometric modeling. The Visual Computer 21, 5, 329–339.Google ScholarDigital Library
25. Měch, R., and Prusinkiewicz, P. 1996. Visual models of plants interacting with their environment. In Proceedings of ACM SIGGRAPH 96, ACM Press, H. Rushmeier, Ed., 397–410. Google ScholarDigital Library
26. Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. 2006. Procedural Modeling of Buildings. In Proceedings of ACM SIGGRAPH 2006 / ACM Transactions on Graphics, ACM Press, New York, NY, USA, vol. 25, 614–623. Google ScholarDigital Library
27. Parish, Y. I. H., and Müller, P. 2001. Procedural modeling of cities. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 301–308. Google ScholarDigital Library
28. Prusinkiewicz, P., and Lindenmayer, A. 1991. The Algorithmic Beauty of Plants. Springer Verlag. Google ScholarDigital Library
29. Prusinkiewicz, P., James, M., and Měch, R. 1994. Synthetic topiary. In Proceedings of ACM SIGGRAPH 94, ACM Press, A. Glassner, Ed., 351–358. Google ScholarDigital Library
30. Prusinkiewicz, P., Mündermann, P., Karwowski, R., and Lane, B. 2001. The use of positional information in the modeling of plants. In Proceedings of ACM SIGGRAPH 2001, ACM Press, E. Fiume, Ed., 289–300. Google ScholarDigital Library
31. Ramamoorthi, R., and Arvo, J. 1999. Creating generative models from range images. In SIGGRAPH ’99: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 195–204. Google ScholarDigital Library
32. Realviz, 2007. Realviz ImageModeler V4.0 product information. http://www.realviz.com.Google Scholar
33. Ribarsky, W., Wasilewski, T., and Faust, N. 2002. From urban terrain models to visible cities. IEEE Computer Graphics & Applications 22, 4, 231–238. Google ScholarDigital Library
34. Schindler, K., and Bauer, J. 2003. A model-based method for building reconstruction. In International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, IEEE Computer Society, 74–82. Google ScholarDigital Library
35. Stiny, G., and Mitchell, W. J. 1978. The palladian grammar. Environment and Planning B 5, 5–18.Google ScholarCross Ref
36. Stiny, G. 1975. Pictorial and Formal Aspects of Shape and Shape Grammars. Birkhauser Verlag, Basel.Google Scholar
37. Stiny, G. 1982. Spatial relations and grammars. Environment and Planning B 9, 313–314.Google ScholarCross Ref
38. Takase, Y., Sho, N., Sone, A., and Shimiya, K. 2003. Automatic generation of 3d city models and related applications. In International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 113–120.Google Scholar
39. Wang, X., Totaro, S., Taillandier, F., Hanson, A., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Proc. ISPRS Commission III Symposium on Photogrammetric Computer Vision, 381–386.Google Scholar
40. Wang, L., You, S., and Neumann, U. 2006. Large-scale urban modeling by combining ground level panoramic and aerial imagery. In 3DPVT. Google ScholarDigital Library
41. Wells, W., Viola, P., Atsumi, H., Nakajima, S., and Kikinis, R., 1996. Multi-modal volume registration by maximization of mutual information.Google Scholar
42. Wonka, P., Wimmer, M., Sillion, F., and Ribarsky, W. 2003. Instant architecture. ACM Transactions on Graphics 22, 3, 669–677. Google ScholarDigital Library