“ConstructAide: analyzing and visualizing construction sites through photographs and building models” by Karsch, Golparvar-Fard and Forsyth
Conference:
Type(s):
Title:
- ConstructAide: analyzing and visualizing construction sites through photographs and building models
Session/Category Title: Paintings, Sketches and Buildings
Presenter(s)/Author(s):
Abstract:
We describe a set of tools for analyzing, visualizing, and assessing architectural/construction progress with unordered photo collections and 3D building models. With our interface, a user guides the registration of the model in one of the images, and our system automatically computes the alignment for the rest of the photos using a novel Structure-from-Motion (SfM) technique; images with nearby viewpoints are also brought into alignment with each other. After aligning the photo(s) and model(s), our system allows a user, such as a project manager or facility owner, to explore the construction site seamlessly in time, monitor the progress of construction, assess errors and deviations, and create photorealistic architectural visualizations. These interactions are facilitated by automatic reasoning performed by our system: static and dynamic occlusions are removed automatically, rendering information is collected, and semantic selection tools help guide user input. We also demonstrate that our user-assisted SfM method outperforms existing techniques on both real-world construction data and established multi-view datasets.
References:
1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Ssstrunk, S. 2012. Slic superpixels compared to state-of-the-art superpixel methods. IEEE TPAMI 34, 11, 2274–2282.
2. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., and Szeliski, R. 2011. Building rome in a day. Commun 54, 105–112.
3. Aubry, M., Russell, B. C., and Sivic, J. 2014. Painting-to-3d model alignment via discriminative visual elements. ACM Trans. Graph. 33, 2 (Apr.), 14:1–14:14.
4. Bae, S., Agarwala, A., and Durand, F. 2010. Computational rephotography. ACM Trans. Graph. 29, 3, 24:1–24:15.
5. Behzadan, A. H., and Kamat, V. R. 2005. Visualization of construction graphics in outdoor augmented reality. In Winter Sim, WSC ’05, 1914–1920.
6. Colburn, A., Agarwala, A., Hertzmann, A., Curless, B., and Cohen, M. F. 2013. Image-based remodeling. IEEE TVCG 19, 1, 56–66.
7. Côté, S., Trudel, P., Desbiens, M., Giguère, M., and Snyder, R. 2013. Live mobile panoramic high accuracy augmented reality for engineering and construction. Proc., the Construction Applications of Virtual Reality.
8. Crandall, D., Owens, A., Snavely, N., and Huttenlocher, D. to appear. SfM with MRFs: Discrete-continuous optimization for large-scale reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence.
9. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In SIGGRAPH, ACM, New York, NY, USA, 11–20.
10. Dellepiane, M., Cavarretta, E., Cignoni, P., and Scopigno, R. 2013. Assisted multi-view stereo reconstruction. In 3DV-Conference, 2013 International Conference on, 318–325.
11. Dunston, P., Wang, X., Billinghurst, M., Hampson, B., et al. 2003. Mixed reality benefits for design perception. In Proc., ISARC, 191–196.
12. Franken, T., Dellepiane, M., Ganovelli, F., Cignoni, P., Montani, C., and Scopigno, R. 2005. Minimizing user intervention in registering 2d images to 3d models. The Visual Computer 21, 8-10, 619–628.
13. Furukawa, Y., and Ponce, J. 2010. Accurate, dense, and robust multiview stereopsis. IEEE Transactions on PAMI 32, 8, 1362–1376.
14. Furukawa, Y., and Ponce, J. 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans. Pattern Anal. Mach. Intell. 32, 8 (Aug.), 1362–1376.
15. Golparvar-Fard, M., Sridhara, A., Lee, S., and Peña-Mora, F. 2007. Visual representation of construction progress monitoring metrics on time-lapse photographs. In Int. Conf. on Constr Mgmt & Economics, 1–10.
16. Golparvar-Fard, M., Peña-Mora, F., Arboleda, C. A., and Lee, S. 2009. Visualization of construction progress monitoring with 4d simulation model overlaid on time-lapsed photographs. J. Computing in Civil Eng. 23, 6, 391–404.
17. Golparvar-Fard, M., Pea-Mora, F., and Savarese, S. 2011. Integrated sequential as-built and as-planned representation with tools in support of decision-making tasks in the aec/fm industry. J. Constr. Eng. & Mgmt 137, 12, 1099–1116.
18. Hakkarainen, M., Woodward, C., and Rainio, K. 2009. Software architecture for mobile mixed reality and 4d bim interaction. In Proc. CIB W78, 1–8.
19. Hammad, A., Wang, H., and Mudur, S. P. 2009. Distributed augmented reality for visualizing collaborative construction tasks. J Comp Civil Eng 23, 6, 418–427.Cross Ref
20. Huttenlocher, D. P., and Ullman, S. 1987. Object recognition using alignment. In ICCV.
21. Irizarry, J., Gheisari, M., Williams, G., and Walker, B. N. 2012. Infospot: A mobile augmented reality method for accessing building information through a situation awareness approach. Automation in Construction.
22. Kahkonen, K., Hyvakka, J., Porkka, J., Siltanen, S., and C., W. 2007. Integrating building product models with live video stream. In 7th Int. Conf. on Construction Applications of Virtual Reality, 176–188.
23. Karsch, K., Hedau, V., Forsyth, D., and Hoiem, D. 2011. Rendering synthetic objects into legacy photographs. In Proc., SIGGRAPH Asia, SA ’11, 157:1–12.
24. Kopf, J., Neubert, B., Chen, B., Cohen, M. F., Cohen-Or, D., Deussen, O., Uyttendaele, M., and Lischinski, D. 2008. Deep photo: Model-based photograph enhancement and viewing. SIGGRAPH Asia 27, 5, 116:1–116:10.
25. Lee, S., and Akin, Ö. 2011. Augmented reality-based computational fieldwork support for equipment operations and maintenance. AutoCON 20, 4, 338–352.
26. Lowe, D. 1987. The viewpoint consistency constraint. IJCV.
27. Paris, S., and Durand, F. 2009. A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vision 81, 1 (Jan.), 24–52.
28. Pintus, R., Gobbetti, E., and Combet, R. 2011. Fast and robust semi-automatic registration of photographs to 3d geometry. In The 12th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, 9–16.
29. Russell, B. C., Sivic, J., Ponce, J., and Dessales, H. 2011. Automatic alignment of paintings and photographs depicting a 3d scene. In 3DRR.
30. Schindler, G., and Dellaert, F. 2012. 4d cities: Analyzing, visualizing, and interacting with historical urban photo collections. J Multimedia 7, 2, 124–131.Cross Ref
31. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multiview stereo reconstruction algorithms. In Computer Vision and Pattern Recognition, vol. 1, 519–528.
32. Shin, D. H., and Dunston, P. S. 2010. Technology development needs for advancing augmented reality-based inspection. AutoCON 19, 2, 169–182.
33. Sinha, S. N., Steedly, D., Szeliski, R., Agrawala, M., and Pollefeys, M. 2008. Interactive 3d architectural modeling from unordered photo collections. SIGGRAPH Asia 27, 5, 159:1–159:10.
34. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: Exploring photo collections in 3d. In SIGGRAPH, ACM, New York, NY, USA, 835–846.
35. Snavely, N., Seitz, S. M., and Szeliski, R. 2008. Modeling the world from Internet photo collections. IJCV 80, 2, 189–210.
36. Snavely, N., Garg, R., Seitz, S. M., and Szeliski, R. 2008. Finding paths through the world’s photos. SIGGRAPH 27, 3, 11–21.
37. Sun, M., Schindler, G., Kang, S. B., and Dellaert, F. 2007. 4d view synthesis: Navigating through time and space. In SIGGRAPH, ACM.
38. Turkan, Y., Bosche, F., Haas, C. T., and Haas, R. 2012. Automated progress tracking using 4d schedule and 3d sensing technologies. AutoCON 22, 414–421.
39. van den Hengel, A., Dick, A., Thormählen, T., Ward, B., and Torr, P. H. S. 2007. Videotrace: Rapid interactive scene modelling from video. ACM Trans. Graph. 26, 3 (July).
40. Vedaldi, A., and Fulkerson, B., 2008. VLFeat: An open and portable library of computer vision algorithms.
41. Wan, G., Snavely, N., Cohen-Or, D., Zheng, Q., Chen, B., and Li, S. 2012. Sorting unorganized photo sets for urban reconstruction. Graphical Models 74, 1.
42. Wang, X. 2009. Augmented reality in architecture and design: potentials and challenges for application. Int. Journal of Architectural Computing 7, 2, 309–326.Cross Ref
43. Werner, T., and Zisserman, A. 2002. New techniques for automated architectural reconstruction from photographs. In EECV, 541–555.
44. Woodward, C., Hakkarainen, M., Korkalo, O., Kantonen, T., Aittala, M., Rainio, K., and Kähkönen, K. 2010. Mixed reality for mobile construction site visualization and communication. In CONVR, 35–44.
45. Wu, C., Agarwal, S., Curless, B., and Seitz, S. M. 2011. Multicore bundle adjustment. In CVPR, 3057–3064.
46. Wu, C. 2013. Towards linear-time incremental structure from motion. In 3DV.
47. Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3d object modeling. SIGGRAPH 30, 4, 80:1–10.
48. Yabuki, N., Limsupreeyarat, P., and Tongthong, T. 2010. Collaborative and visualized safety planning for construction performed at high elevation. In Cooperative Design, Visualization, and Engineering. Springer, 282–285.
49. Zollmann, S., Kalkofen, D., Mendez, E., and Reitmayr, G. 2010. Image-based ghostings for single layer occlusions in augmented reality. In ISMAR, 19–26.
50. Zollmann, S., Kalkofen, D., Hoppe, C., Kluckner, S., Bischof, H., and Reitmayr, G. 2012. Interactive 4d overview and detail visualization in augmented reality. ISMAR.
