“Projective analysis for 3D shape segmentation”
Conference:
Type(s):
Title:
- Projective analysis for 3D shape segmentation
Session/Category Title: Shape and Machine Learning
Presenter(s)/Author(s):
Abstract:
We introduce projective analysis for semantic segmentation and labeling of 3D shapes. The analysis treats an input 3D shape as a collection of 2D projections, labels each projection by transferring knowledge from existing labeled images, and back-projects and fuses the labelings on the 3D shape. The image-space analysis involves matching projected binary images of 3D objects based on a novel bi-class Hausdorff distance. The distance is topology-aware by accounting for internal holes in the 2D figures and it is applied to piecewise-linearly warped object projections to compensate for part scaling and view discrepancies. Projective analysis simplifies the processing task by working in a lower-dimensional space, circumvents the requirement of having complete and well-modeled 3D shapes, and addresses the data challenge for 3D shape analysis by leveraging the massive available image data. A large and dense labeled set ensures that the labeling of a given projected image can be inferred from closely matched labeled images. We demonstrate semantic labeling of imperfect (e.g., incomplete or self-intersecting) 3D models which would be otherwise difficult to analyze without taking the projective analysis approach.
References:
1. Baddeley, A. J. 1992. An error metric for binary images. In IEEE Workshop on Robust Computer Vision, 59–78.
2. Bai, X., Yang, X., Latecki, L. J., Liu, W., and Tu, Z. 2010. Learning context-sensitive shape similarity by graph transduction. PAMI 32, 5, 861–874.
3. Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape context. PAMI 24, 4, 509–522.
4. Berg, A. C., Berg, T. L., and Malik, J. 2005. Shape matching and object recognition using low distortion correspondence. In CVPR, 26–33.
5. Berndt, D., and Clifford, J. 1994. Using dynamic time warping to find patterns in time series. In KDD workshop, 359–370.
6. Blake, A., Rother, C., Brown, M., Perez, P., and Torr, P. 2004. Interactive image segmentation using an adaptive gmmrf model. In ECCV, 428–441.
7. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. PAMI 23, 11, 1222–1239.
8. Chang, J. Y., Raskar, R., and Agrawal, A. 2009. 3D pose estimation and segmentation using specular cues. In CVPR, 1706–1713.
9. Chen, D., Tian, X., Shen, Y., and Ouhyoung, M. 2003. On visual similarity based 3D model retrieval. CGF (EUROGRAPHICS) 22, 3, 223–232.
10. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. ACM Trans. Graph. (SIGGRAPH) 28, 3, 1–12.
11. Chen, X., Li, Q., Song, Y., Jin, X., and Zhao, Q. 2012. Supervised geodesic propagation for semantic label transfer. In ECCV, 553–565.
12. Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, vol. 1, 22.
13. Cyr, C. M., and Kimia, B. B. 2001. 3D object recognition using shape similiarity-based aspect graph. In ICCV, 254–261.
14. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and FeiFei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR, 248–255.
15. Duchenne, O., Joulin, A., and Ponce, J. 2011. A graph-matching kernel for object categorization. In ICCV, 1792–1799.
16. Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. (SIGGRAPH) 31, 4, 31–40.
17. Ferrari, V., Tuytelaars, T., and Gool, L. V. 2004. Integrating multiple model views for object recognition. In CVPR, 105–112.
18. Fleming, R. W., and Singh, M. 2009. Visual perception of 3D shape. In ACM SIGGRAPH 2009 Courses, 24:1–24:94.
19. Golovinskiy, A., and Funkhouser, T. 2009. Consistent segmentation of 3D models. Computers & Graphics (SMI) 33, 3, 262–269.
20. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D mesh segmentation and labeling. ACM Trans. Graph. (SIGGRAPH) 29, 3, 1–11.
21. Klein, P. N., Sebastian, T. B., and Kimia, B. B. 2001. Shape matching using edit-distance: an implementation. In SODA, 781–790.
22. Koenderink, J. J. 1984. What does the occluding contour tell us about solid shape. Perception 13, 321–330.
23. Lai, K., Bo, L., Ren, X., and Fox, D. 2012. Detection-based object labeling in 3D scenes. In ICRA, 1330–1337.
24. Laurentini, A. 1994. The visual hull concept for silhouette-based image understanding. PAMI 16, 2, 150–162.
25. Leordeanu, M., and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In ICCV, 1482–1489.
26. Li, Y., Zheng, Q., Sharf, A., Cohen-Or, D., Chen, B., and Mitra, N. 2011. 2D-3D fusion for layer decomposition of urban facades. In ICCV, 882–889.
27. Lindstrom, P., and Turk, G. 2000. Image-driven simplification. ACM Trans. Graph. 19, 3, 204–241.
28. Ling, H., and Jacobs, D. 2007. Shape classification using the inner-distance. PAMI 29, 2, 286–299.
29. Liu, R., and Zhang, H. 2007. Mesh segmentation via spectral embedding and contour analysis. CGF (EUROGRAPHICS) 26, 3, 385–394.
30. Liu, C., Yuen, J., and Torralba, A. 2011. Nonparametric scene parsing via label transfer. PAMI 33, 12, 2368–2382.
31. Liu, C., Yuen, J., and Torralba, A. 2011. SIFT flow: Dense correspondence across scenes and its applications. PAMI 33, 5, 978–994.
32. Makadia, A., and Daniilidis, K. 2010. Spherical correlation of visual representations for 3D model retrieval. IJCV 89, 2–3, 193–210.
33. Murase, H., and Nayar, S. K. 1995. Visual learning and recognition of 3-d objects from appearance. IJCV 14, 1, 5–24.
34. Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 3, 145–175.
35. Pepik, B., Gehler, P., Stark, M., and Schiele, B. 2012. 3D2pm-3D deformable part models. In ECCV, 356–370.
36. Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (SIGGRAPH) 23, 3, 309–314.
37. Shamir, A. 2008. A survey on mesh segmentation techniques. CGF 27, 6, 1539–1556.
38. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 1–15.
39. Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans. Graph. (SIGGRAPH Asia) 30, 6, 126:1–126:10.
40. Telea, A., and Van Wijk, J. J. 1999. Simplified representation of vector fields. In IEEE Visualization, 35–507.
41. van Kaick, O., Tagliasacchi, A., Sidi, O., Zhang, H., Cohen-Or, D., Wolf, L., and Hamarneh, G. 2011. Prior knowledge for part correspondence. CGF (EUROGRAPHICS) 30, 2, 553–562.
42. Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Active co-analysis of a set of shapes. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 165–174.
43. Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 3485–3492.
44. Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., and Cheng, Z. 2010. Style-content separation by anisotropic part scales. ACM Trans. Graph. (SIGGRAPH Asia) 29, 5.
45. Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3D object modeling. ACM Trans. Graph. (SIGGRAPH) 30, 4, 80:1–80:10.
46. Zhang, H., Xiao, J., and Quan, L. 2010. Supervised label transfer for semantic segmentation of street scenes. In ECCV, 561–574.
