“SmartBoxes for interactive urban reconstruction” by Nan, Sharf, Zhang, Cohen-Or and Chen

We introduce an interactive tool which enables a user to quickly assemble an architectural model directly over a 3D point cloud acquired from large-scale scanning of an urban scene. The user loosely defines and manipulates simple building blocks, which we call SmartBoxes, over the point samples. These boxes quickly snap to their proper locations to conform to common architectural structures. The key idea is that the building blocks are smart in the sense that their locations and sizes are automatically adjusted on-the-fly to fit well to the point data, while at the same time respecting contextual relations with nearby similar blocks. SmartBoxes are assembled through a discrete optimization to balance between two snapping forces defined respectively by a data-fitting term and a contextual term, which together assist the user in reconstructing the architectural model from a sparse and noisy point cloud. We show that a combination of the user’s interactive guidance and high-level knowledge about the semantics of the underlying model, together with the snapping forces, allows the reconstruction of structures which are partially or even completely missing from the input.

1. Bokeloh, M., Berner, A., Wand, M., Seidel, H.-P., and Schilling, A. 2009. Symmetry detection using feature lines. Computer Graphics Forum (Proceedings of Eurographics) 28, 2, 697–706.Google ScholarCross Ref
2. Chen, X., Kang, S. B., Xu, Y.-Q., Dorsey, J., and Shum, H.-Y. 2008. Sketching reality: Realistic interpretation of architectural designs. ACM Trans. on Graphics 27, 2, 1–15. Google ScholarDigital Library
3. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. Proc. of ACM SIGGRAPH, 11–20. Google ScholarDigital Library
4. Dick, A. R., Torr, P. H. S., and Cipolla, R. 2004. Modelling and interpretation of architecture from several images. Int. J. Comput. Vision 60, 2, 111–134. Google ScholarDigital Library
5. Früh, C., Jain, S., and Zakhor, A. 2005. Data processing algorithms for generating textured 3d building facade meshes from laser scans and camera images. Int. J. Comput. Vision 61, 2. Google ScholarDigital Library
6. Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2009. Reconstructing building interiors from images.Google Scholar
7. Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2009. Manhattan-world stereo. In Proc. of IEEE Conf. on Comp. Vis. and Pat. Rec., 1422–1429.Google Scholar
8. Gal, R., Shamir, A., Hassner, T., Pauly, M., and Cohen-Or, D. 2007. Surface reconstruction using local shape priors. In Proc. of Eurographics Symp. on Geometry Processing, 253–262. Google ScholarDigital Library
9. Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S. 2007. Multi-view stereo for community photo collections. In Proc. of Int. Conf. on Comp. Vis., 1–8.Google Scholar
10. Hohmann, B., Krispel, U., Havemann, S., and Fellner, D. W. 2009. Cityfit: High-quality urban recontrustion by fitting shape grammars to images and derived textured point clouds. In Proceedings of the 3rd ISPRS Workshop.Google Scholar
11. Jiang, N., Tan, P., and Cheong, L.-F. 2009. Symmetric architecture modeling with a single image. ACM Trans. on Graphics 28, 5, 1–8. Google ScholarDigital Library
12. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital michelangelo project: 3D scanning of large statues. In Proc. of ACM SIGGRAPH, 131–144. Google ScholarDigital Library
13. Müller, P., Wonka, P., Haegler, S., Ulmer, A., and Van Gool, L. 2006. Procedural modeling of buildings. ACM Trans. on Graphics 25, 3, 614–623. Google ScholarDigital Library
14. Müller, P., Zeng, G., Wonka, P., and Gool, L. J. V. 2007. Image-based procedural modeling of facades. ACM Trans. on Graphics 26, 3, 85. Google ScholarDigital Library
15. Parish, Y. I. H., and Müller, P. 2001. Procedural modeling of cities. In Proc. of ACM SIGGRAPH, 301–308. Google ScholarDigital Library
16. Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. J. 2005. Example-based 3D scan completion. In Proc. of Eurographics Symp. on Geometry Processing, 23. Google ScholarDigital Library
17. Pauly, M., Mitra, N. J., Wallner, J., Pottmann, H., and Guibas, L. 2008. Discovering structural regularity in 3D geometry. ACM Trans. on Graphics 27, 3. Google ScholarDigital Library
18. Pollefeys, M., Nistér, D., Frahm, J. M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S. J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., and Towles, H. 2008. Detailed real-time urban 3D reconstruction from video. Int. J. Comput. Vision 78, 2–3, 143–167. Google ScholarDigital Library
19. Schindler, K., and Bauer, J. 2003. A model-based method for building reconstruction. In Proc. of IEEE Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 74. Google ScholarDigital Library
20. Schnabel, R., Wahl, R., and Klein, R. 2007. Efficient RANSAC for point-cloud shape detection. Computer Graphics Forum 26, 2, 214–226.Google ScholarCross Ref
21. Schnabel, R., Degener, P., and Klein, R. 2009. Completion and reconstruction with primitive shapes. Computer Graphics Forum (Proc. of Eurographics) 28, 2, 503–512.Google ScholarCross Ref
22. Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3D architectural modeling from unordered photo collections. ACM Trans. on Graphics 27, 5, 1–10. Google ScholarDigital Library
23. Vanegas, C. A., Aliaga, D. G., Wonka, P., Mueller, P., Waddell, P., and Watson, B. 2009. Modeling the appearance and behavior of urban spaces. In Proc. of Eurographics State-of-the-Art Report.Google Scholar
24. Werner, T., and Zisserman, A. 2002. New techniques for automated architecture reconstruction from photographs. In Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, vol. 2, 541–555. Google ScholarDigital Library
25. Wonka, P., Wimmer, M., Sillion, F., and Ribarsky, W. 2003. Instant architecture. ACM Trans. on Graphics 22, 3, 669–677. Google ScholarDigital Library
26. Xiao, J., Fang, T., Tan, P., Zhao, P., Ofek, E., and Quan, L. 2008. Image-based façade modeling. ACM Trans. on Graphics 27, 5, 1–10. Google ScholarDigital Library
27. Xiao, J., Fang, T., Zhao, P., Maxime, L., and Quan, L. 2009. Image-based street-side city modeling. ACM Trans. on Graphics 28, 5, 1–12. Google ScholarDigital Library
28. Zebedin, L., Bauer, J., Karner, K., and Bischof, H. 2008. Fusion of feature- and area-based information for urban buildings modeling from aerial imagery. In Proc. Euro. Conf. on Comp. Vis., 873–886. Google ScholarDigital Library