“Image-based façade modeling”
Conference:
Type(s):
Title:
- Image-based façade modeling
Session/Category Title: Urban modelling
Presenter(s)/Author(s):
Abstract:
We propose in this paper a semi-automatic image-based approach to façade modeling that uses images captured along streets and relies on structure from motion to recover camera positions and point clouds automatically as the initial stage for modeling. We start by considering a building façade as a flat rectangular plane or a developable surface with an associated texture image composited from the multiple visible images. A façade is then decomposed and structured into a Directed Acyclic Graph of rectilinear elementary patches. The decomposition is carried out top-down by a recursive subdivision, and followed by a bottom-up merging with the detection of the architectural bilateral symmetry and repetitive patterns. Each subdivided patch of the flat façade is augmented with a depth optimized using the 3D points cloud. Our system also allows for an easy user feedback in the 2D image space for the proposed decomposition and augmentation. Finally, our approach is demonstrated on a large number of façades from a variety of street-side images.
References:
1. Agarwala, A., Agrawala, M., Cohen, M., Salesin, D., and Szeliski, R. 2006. Photographing long scenes with multi-viewpoint panoramas. ACM Transactions on Graphics(SIGGRAPH) 25, 3, 853–861. Google ScholarDigital Library
2. Berg, A. C., Grabler, F., and Malik, J. 2007. Parsing images of architectural scenes. In Proceedings of IEEE International Conference on Computer Vision, 1–8.Google Scholar
3. Canny, J. F. 1986. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 8, 679–714. Google ScholarDigital Library
4. Cornelis, N., Leibe, B., Cornelis, K., and Gool, L. V. 2008. 3d urban scene modeling integrating recognition and reconstruction. International Journal of Computer Vision 78, 2–3, 121–141. Google ScholarDigital Library
5. Criminisi, A., Perez, P., and Toyama, K. 2003. Object removal by exemplar-based inpainting. In Proceedings of IEEE Computer Vision and Pattern Recognition, vol. 2, 721–728.Google Scholar
6. Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: a hybrid geometry-and image-based approach. In Proceedings of ACM SIGGRAPH, 11–20. Google Scholar
7. Dick, A., Torr, P., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. International Journal of Computer Vision 2, 111–134. Google ScholarCross Ref
8. Duda, R. O., and Hart, P. E. 1972. Use of the hough transformation to detect lines and curves in pictures. Communications of the ACM 15, 1, 11–15. Google ScholarDigital Library
9. Früh, C., and Zakhor, A. 2003. Constructing 3d city models by merging ground-based and airborne views. In Proceedings of IEEE Computer Vision and Pattern Recognition, vol. 2, 562–569.Google Scholar
10. Furukawa, Y., and Ponce, J. 2007. Accurate, dense, and robust multi-view stereopsis. In Proceedings of IEEE Conference Computer Vision and Pattern Recognition, 1–8.Google Scholar
11. Geman, S., and Geman, D. 1984. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6, 6, 721–741.Google ScholarDigital Library
12. Goesele, M., Snavely, N., Curless, B., Seitz, S. M., and Hoppe, H. 2007. Multi-view stereo for community photo collections. In Proceeding of IEEE International Conference in Computer Vision, 1–8.Google Scholar
13. Hartley, R. I., and Zisserman, A. 2004. Multiple View Geometry in Computer Vision, second ed. Cambridge University Press, ISBN: 0521540518. Google Scholar
14. Kullback, S., and Leibler, R. A. 1951. On information and sufficiency. Annals of Mathematical Statistics 22, 79–86.Google ScholarCross Ref
15. Lhuillier, M., and Quan, L. 2005. A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 3, 418–433. Google ScholarDigital Library
16. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Transactions on Graphics 23, 303–308. Google ScholarDigital Library
17. Lukas, Z., Andreas, K., Barbara, G.-G., and Konrad, K. 2006. Towards 3d map generation from digital aerial images. International Journal of Photogrammetry and Remote Sensing 60, 413–427.Google ScholarCross Ref
18. Lukas, Z., Joachim, B., Konrad, K., and Horst, B. 2008. Fusion of feature- and area-based information for urban buildings modeling from aerial imagery. In Proceedings of the European Conference on Computer Vision. Google Scholar
19. Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Gool, L. V. 2006. Procedural modeling of buildings. ACM Transactions on Graphics 3, 614–623. Google ScholarDigital Library
20. Müller, P., Zeng, G., Wonka, P., and Gool, L. V. 2007. Image-based procedural modeling of façades. ACM Transactions on Graphics 26, 3, 85. Google ScholarDigital Library
21. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. ACM Transactions on Graphics 1, 433–442.Google Scholar
22. Pollefeys, M., Nistér, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.-J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., and Towles, H. 2007. Detailed real-time urban 3d reconstruction from video. International Journal of Comptuer Vision 78, 2–3, 143–167. Google Scholar
23. Scharstein, D., and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47, 1/2/3, 7–42. Google ScholarCross Ref
24. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: Exploring photo collections in 3d. ACM Transactions on Graphics 3, 835–846. Google ScholarDigital Library
25. Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Transactions on Graphics 24, 861–868. Google ScholarDigital Library
26. van den Hengel, A., Dick, A., Thormählen, T., Ward, B., and Torr, P. H. S. 2007. Videotrace: rapid interactive scene modelling from video. ACM Transactions on Graphics 3, 86. Google ScholarDigital Library
27. Vogiatzis, G., Torr, P. H. S., Seitz, S. M., and Cipolla, R. 2008. Reconstructing relief surfaces. Image and Vision Computing 26, 3, 397–404. Google ScholarDigital Library
28. Weiss, Y., and Freeman, W. T. 2001. On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Transactions on Information Theory 47, 2, 723–735. Google ScholarDigital Library
29. Werner, T., and Zisserman, A. 2002. New techniques for automated architectural reconstruction from photographs. In Proceedings of the European Conference on Computer Vision, vol. 2, 541–555. Google ScholarDigital Library
30. Wonka, P., Wimmer, M., Sillion, F., and Ribarsky, W. 2003. Instant architecture. ACM Transactions on Graphics 4, 669–677. Google ScholarDigital Library
