“Non-local scan consolidation for 3D urban scenes” by Zheng, Sharf, Wan, Li, Mitra, et al. …

Recent advances in scanning technologies, in particular devices that extract depth through active sensing, allow fast scanning of urban scenes. Such rapid acquisition incurs imperfections: large regions remain missing, significant variation in sampling density is common, and the data is often corrupted with noise and outliers. However, buildings often exhibit large scale repetitions and self-similarities. Detecting, extracting, and utilizing such large scale repetitions provide powerful means to consolidate the imperfect data. Our key observation is that the same geometry, when scanned multiple times over reoccurrences of instances, allow application of a simple yet effective non-local filtering. The multiplicity of the geometry is fused together and projected to a base-geometry defined by clustering corresponding surfaces. Denoising is applied by separating the process into off-plane and in-plane phases. We show that the consolidation of the reoccurrences provides robust denoising and allow reliable completion of missing parts. We present evaluation results of the algorithm on several LiDAR scans of buildings of varying complexity and styles.

1. Aiger, D., Mitra, N. J., and Cohen-Or, D. 2008. 4-points congruent sets for robust surface registration. Proc. of ACM SIGGRAPH 27, 3, #85, 1–10. Google ScholarDigital Library
2. Besl, P., and McKay, N. 1992. A method for registration of 3D. In IEEE PAMI. Google ScholarDigital Library
3. Biber, P., and Strasser, W. 2006. nscan-matching: Simultaneous matching of multiple scans and application to slam. In In Robotics and Automation, 2270–2276.Google Scholar
4. Bokeloh, M., Berner, A., Wand, M., Seidel, H.-P., and Schilling, A. 2009. Symmetry detection using line features. Computer Graphics Forum (Proceedings of Eurographics).Google Scholar
5. Buades, A., Coll, B., and Morel, J.-M. 2005. A non-local algorithm for image denoising. In Proc. of IEEE Conf. on Comp. Vis. and Pat. Rec., 60–65. Google ScholarDigital Library
6. Buades, A., Coll, B., and Morel, J.-M. 2008. Nonlocal image and movie denoising. Int. J. Comp. Vis. 76, 2, 123–139. Google ScholarDigital Library
7. Dabov, Foi, Katkovnik, and Egiazarian. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. Image Processing, IEEE Transactions on 16, 8, 2080–2095. Google ScholarDigital Library
8. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering arch. from photographs: A hybrid geometry- and image-based approach. Proc. SIGGRAPH 30, 11–20. Google ScholarDigital Library
9. Fleishman, S., Drori, I., and Cohen-Or, D. 2003. Bilateral mesh denoising. Proc. of ACM SIGGRAPH 22, 3, 950–953. Google ScholarDigital Library
10. Gal, R., Shamir, A., Hassner, T., Pauly, M., and Cohen-Or, D. 2007. Surface reconstruction using local shape priors. In Proc. of Eurographics Symp. on Geometry Processing, 253–262. Google ScholarDigital Library
11. Hays, J. H., Leordeanu, M., Efros, A. A., and Liu, Y. 2006. Discovering texture regularity as a higher-order correspondence problem. In Proc. Euro. Conf. on Comp. Vis. Google ScholarDigital Library
12. Huang, H., Li, D., Zhang, H., Ascher, U., and Cohen-Or, D. 2009. Consolidation of unorganized point clouds for surface reconstruction. ACM Trans. Graph. 28, 5, Article 176. Google ScholarDigital Library
13. Jones, T. R., Durand, F., and Desbrun, M. 2003. Noniterative, feature-preserving mesh smoothing. In Proc. of ACM SIGGRAPH, 943–949. Google ScholarDigital Library
14. Korah, T., and Rasmussen, C. 2007. 2d lattice extraction from structured environments. In ICIP, 61–64.Google Scholar
15. Korah, T., and Rasmussen, C. 2008. Analysis of building textures for reconstructing partially occluded facades. In Proc. Euro. Conf. on Comp. Vis., 359–372. Google ScholarDigital Library
16. Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., and Fulk, D. 2000. The digital michelangelo project: 3d scanning of large statues. In Proc. of ACM SIGGRAPH, 131–144. Google ScholarDigital Library
17. Lindenbaum, M., Fischer, M., and Bruckstein, A. M. 1994. On gabor’s contribution to image enhancement. Pattern Recognition 27, 1, 1–8.Google ScholarCross Ref
18. Lipman, Y., Cohen-Or, D., Levin, D., and Tal-Ezer, H. 2007. Parameterization-free projection for geometry reconstruction. ACM Trans. Graph. 26, 3, 22. Google ScholarDigital Library
19. Liu, Y., Collins, R. T., and Tsin, Y. 2004. A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE PAMI 26, 3, 354–371. Google ScholarDigital Library
20. Liu, Y., Lin, W.-C., and Hays, J. H. 2004. Near regular texture analysis and manipulation. 368–376.Google Scholar
21. Merritt, F., and Ricketts, J. 2001. Building Design and Construction Handbook, 6th ed. McGraw-Hill.Google Scholar
22. Mitra, N. J., Guibas, L., and Pauly, M. 2006. Partial and approximate symmetry detection for 3d geometry. In Proc. of ACM SIGGRAPH, vol. 25, 560–568. Google ScholarDigital Library
23. Müller, P., Zeng, G., Wonka, P., and Gool, L. J. V. 2007. Image-based procedural modeling of facades. ACM Trans. on Graphics 26, 3, 85. Google ScholarDigital Library
24. Musialski, P., Wonka, P., Recheis, M., Maierhofer, S., and Purgathofer, W. 2009. Symmetry-based facade repair. In Vision, Modeling, and Visualization Workshop 2009 in Braunschweig, Germany (VMV09).Google Scholar
25. Oztireli, C., Guennebaud, G., and Gross, M. 2009. Feature preserving point set surfaces based on non-linear kernel regression. Proc. Eurographics) 28, 2, 493–501.Google ScholarCross Ref
26. Park, M., Brocklehurst, K., Collins, R. T., and Liu, Y. 2009. Deformed lattice detection in real-world images using mean-shift belief propagation. IEEE PAMI 31. Google ScholarDigital Library
27. Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. 2005. Example-based 3d scan completion. In Proc. of Symp. of Geometry Processing, 23–32. Google ScholarDigital Library
28. Pauly, M., Mitra, N. J., Wallner, J., Pottmann, H., and Guibas, L. 2008. Discovering structural regularity in 3D geometry. ACM Trans. on Graphics 27, 3. Google ScholarDigital Library
29. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 7, 629–639. Google ScholarDigital Library
30. Rudin, L. I., Osher, S., and Fatemi, E. 1992. Nonlinear total variation based noise removal algorithms. Physica D 60, 1–4, 259–268. Google ScholarDigital Library
31. Schaffalitzky, F., and Zisserman, A. 1999. Geometric grouping of repeated elements within images. In Shape, Contour and Grouping in Computer Vision, 165–181. Google ScholarDigital Library
32. Schnabel, R., Degener, P., and Klein, R. 2009. Completion and reconstruction with primitive shapes. Computer Graphics Forum (Proc. of Eurographics) 28, 2, 503–512.Google ScholarCross Ref
33. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. of Int. Conf. on Comp. Vis., 839. Google ScholarDigital Library
34. Wang, X., Totaro, S., Taill, F., Hanson, A. R., and Teller, S. 2002. Recovering facade texture and microstructure from real-world images. In Texture Analysis and Synth., 381–386.Google Scholar
35. Xiao, J., Fang, T., Zhao, P., Lhuillier, M., and Quan, L. 2009. Image-based street-side city modeling. In ACM SIGGRAPH Asia 2009 papers, 1–12. Google ScholarDigital Library
36. Yoshizawa, S., Belyaev, A., and Seidel, H.-P. 2006. Smoothing by example: Mesh denoising by averaging with similarity-based weights. In SMI, 38–44. Google ScholarDigital Library
37. Yu, Y., Ferencz, A., and Malik, J. 2001. Extracting objects from range and radiance images. IEEE Transactions on Visualization and Computer Graphics 7, 4, 351–364. Google ScholarDigital Library