“Data-driven structural priors for shape completion” by Sung, Kim, Angst and Guibas
Conference:
Type(s):
Title:
- Data-driven structural priors for shape completion
Session/Category Title: 3D Scanning
Presenter(s)/Author(s):
Abstract:
Acquiring 3D geometry of an object is a tedious and time-consuming task, typically requiring scanning the surface from multiple viewpoints. In this work we focus on reconstructing complete geometry from a single scan acquired with a low-quality consumer-level scanning device. Our method uses a collection of example 3D shapes to build structural part-based priors that are necessary to complete the shape. In our representation, we associate a local coordinate system to each part and learn the distribution of positions and orientations of all the other parts from the database, which implicitly also defines positions of symmetry planes and symmetry axes. At the inference stage, this knowledge enables us to analyze incomplete point clouds with substantial occlusions, because observing only a few regions is still sufficient to infer the global structure. Once the parts and the symmetries are estimated, both data sources, symmetry and database, are fused to complete the point cloud. We evaluate our technique on a synthetic dataset containing 481 shapes, and on real scans acquired with a Kinect scanner. Our method demonstrates high accuracy for the estimated part structure and detected symmetries, enabling higher quality shape completions in comparison to alternative techniques.
References:
1. Breiman, L. 2001. Random forests. Machine Learning 45, 1, 5–32.
2. Chaudhuri, S., Kalogerakis, E., Guibas, L., and Koltun, V. 2011. Probabilistic reasoning for assembly-based 3d modeling. ACM TOG 30, 4 (July), 35:1–35:10.
3. Choi, S., Zhou, Q.-Y., and Koltun, V. 2015. Robust reconstruction of indoor scenes. CVPR.
4. Eiter, T., and Mannila, H. 1997. Distance measures for point sets and their computation. Acta Informatica 34, 2, 109–133.
5. Felzenszwalb, P., McAllester, D., and Ramanan, D. 2008. A discriminatively trained, multiscale, deformable part model. In IEEE CVPR, 1–8.
6. Fish, N., Averkiou, M., van Kaick, O., Sorkine-Hornung, O., Cohen-Or, D., and Mitra, N. J. 2014. Meta-representation of shape families. ACM TOG 33, 4, 34:1–34:11.
7. Golovinskiy, A., and Funkhouser, T. 2009. Consistent segmentation of 3D models. Proc. SMI 33, 3, 262–269.
8. Han, F., and Zhu, S.-C. 2009. Bottom-up/top-down image parsing with attribute grammar. IEEE PAMI 31, 1, 59–73.
9. Hu, R., Fan, L., and Liu, L. 2012. Co-segmentation of 3d shapes via subspace clustering. SGP 31, 5, 1703–1713.
10. Huang, Q., Koltun, V., and Guibas, L. 2011. Joint shape segmentation with linear programming. In SIGGRAPH Asia.
11. Huang, Q., Wang, F., and Guibas, L. 2014. Functional map networks for analyzing and exploring large shape collections. ACM TOG 33, 4 (July), 36:1–36:11.
12. Janoch, A., Karayev, S., Jia, Y., Barron, J., Fritz, M., Saenko, K., and Darrell, T. 2011. A category-level 3-d object dataset: Putting the kinect to work. In ICCV Workshop on Consumer Depth Cameras in Computer Vision, 1168–1174.
13. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D mesh segmentation and labeling. In SIGGRAPH, 102:1–102:12.
14. Kalogerakis, E., Chaudhuri, S., Koller, D., and Koltun, V. 2012. A probabilistic model for component-based shape synthesis. ACM TOG 31, 4 (July), 55:1–55:11.
15. Kazhdan, M., and Hoppe, H. 2013. Screened poisson surface reconstruction. Transactions on Graphics 32, 3.
16. Kim, Y. M., Mitra, N. J., Yan, D.-M., and Guibas, L. 2012. Acquiring 3d indoor environments with variability and repetition. ACM TOG 31, 6 (Nov.), 138:1–138:11.
17. Kim, V. G., Li, W., Mitra, N. J., Chaudhuri, S., DiVerdi, S., and Funkhouser, T. 2013. Learning part-based templates from large collections of 3d shapes. ACM TOG 32, 4, 70:1–70:12.
18. Kim, Y. M., Mitra, N. J., Huang, Q., and Guibas, L. 2013. Guided real-time scanning of indoor objects. Computer Graphics Forum 32, 7, 177–186.
19. Kim, V. G., Chaudhuri, S., Guibas, L., and Funkhouser, T. 2014. Shape2pose: Human-centric shape analysis. ACM TOG 33, 4 (July), 120:1–120:12.
20. Kolmogorov, V. 2006. Convergent tree-reweighted message passing for energy minimization. IEEE PAMI 28, 10, 1568–1583.
21. Lai, K., Bo, L., Ren, X., and Fox, D. 2011. A large-scale hierarchical multi-view rgb-d object dataset. In IEEE ICRA, 1817–1824.
22. Li, Y., Dai, A., Guibas, L., and Niessner, M. 2015. Database-assisted object retrieval for real-time 3d reconstruction. Eurographics.
23. Liu, T., Chaudhuri, S., Kim, V. G., Huang, Q., Mitra, N. J., and Funkhouser, T. 2014. Creating consistent scene graphs using a probabilistic grammar. ACM TOG 33, 6, 211:1–211:12.
24. Makadia, A., and Yumer, M. E. 2014. Learning 3d part detection from sparsely labeled data. In 3DV, IEEE.
25. Martinovic, A., and Van Gool, L. 2013. Bayesian grammar learning for inverse procedural modeling. In IEEE CVPR, 201–208.
26. Mitra, N. J., Guibas, L., and Pauly, M. 2006. Partial and approximate symmetry detection for 3d geometry. SIGGRAPH.
27. Mitra, N. J., Pauly, M., Wand, M., and Ceylan, D. 2013. Symmetry in 3d geometry: Extraction and applications. CGF.
28. Nan, L., Xie, K., and Sharf, A. 2012. A search-classify approach for cluttered indoor scene understanding. SIGGRAPH Asia 31, 6.
29. Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. 2005. Example-based 3d scan completion. SGP.
30. Pauly, M., Mitra, N. J., wallner, J., Pottmann, H., and Guibas, L. J. 2008. Discovering structural regularity in 3d geometry. ACM TOG 27, 3 (Aug.), 43:1–43:11.
31. Podolak, J., Shilane, P., Golovinskiy, A., Rusinkiewicz, S., and Funkhouser, T. 2006. A planar-reflective symmetry transform for 3D shapes. ACM TOG 25, 3.
32. Shao, T., Monszpart, A., Zheng, Y., Koo, B., Xu, W., Zhou, K., and Mitra, N. J. 2014. Imagining the unseen: Stability-based cuboid arrangements for scene understanding. ACM TOG 33, 6 (Nov.), 209:1–209:11.
33. Shen, C.-H., Fu, H., Chen, K., and Hu, S.-M. 2012. Structure recovery by part assembly. ACM TOG 31, 6 (Nov.), 180:1–180:11.
34. Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. SIGGRAPH Asia 30, 6, 126:1–126:9.
35. Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. 2012. Indoor segmentation and support inference from rgbd images. In ECCV, 746–760.
36. Sipiran, I., Gregor, R., and Schreck, T. 2014. Approximate symmetry detection in partial 3d meshes. CGF 33, 7, 131–140.
37. Thrun, S., and Wegbreit, B. 2005. Shape from symmetry. In ICCV, 1824–1831.
38. Wächter, A., and Biegler, L. T. 2006. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming 106, 1.
39. Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Active co-analysis of a set of shapes. ACM TOG 31, 6 (Nov.), 165:1–165:10.
40. Wu, C., Lenz, I., and Saxena, A. 2014. Hierarchical semantic labeling for task-relevant rgb-d perception. RSS.
41. Xie, Z., Xu, K., Liu, L., and Xiong, Y. 2014. 3d shape segmentation and labeling via extreme learning machine. SGP.
42. Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., and Cheng, Z. 2010. Style-content separation by anisotropic part scales. SIGGRAPH Asia 29, 5.
43. Xu, K., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Fit and diverse: Set evolution for inspiring 3d shape galleries. 57:1–57:10.
44. Yumer, M. E., and Kara, L. B. 2014. Co-constrained handles for deformation in shape collections. ACM TOG 33, 6.
45. Yumer, M., Chun, W., and Makadia, A. 2014. Co-segmentation of textured 3d shapes with sparse annotations. In IEEE CVPR, 240–247.
46. Zheng, Q., Sharf, A., Wan, G., Li, Y., Mitra, N. J., Cohen-Or, D., and Chen, B. 2010. Non-local scan consolidation for 3d urban scenes. ACM TOG 29, 4 (July), 94:1–94:9.
47. Zheng, Y., Cohen-Or, D., and Mitra, N. J. 2013. Smart variations: Functional substructures for part compatibility. Computer Graphics Forum 32, 195–204.
