“Learning 3D mesh segmentation and labeling” by Kalogerakis, Hertzmann and Singh

This paper presents a data-driven approach to simultaneous segmentation and labeling of parts in 3D meshes. An objective function is formulated as a Conditional Random Field model, with terms assessing the consistency of faces with labels, and terms between labels of neighboring faces. The objective function is learned from a collection of labeled training meshes. The algorithm uses hundreds of geometric and contextual label features and learns different types of segmentations for different tasks, without requiring manual parameter tuning. Our algorithm achieves a significant improvement in results over the state-of-the-art when evaluated on the Princeton Segmentation Benchmark, often producing segmentations and labelings comparable to those produced by humans.

1. Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., and Ng, A. 2005. Discriminative Learning of Markov Random Fields for Segmentation of 3D Scan Data. In CVPR. Google ScholarDigital Library
2. Attene, M., Katz, S., Mortara, M., Patane, G., Spagnuolo, M., and Tal, A. 2006. Mesh Segmentation – A Comparative Study. In Proc. SMI. Google ScholarDigital Library
3. Attene, M., Falcidieno, B., and Spagnuolo, M. 2006. Hierarchical Mesh Segmentation Based on Fitting Primitives. Vis. Comput. 22, 3. Google ScholarDigital Library
4. Belongie, S., Malik, J., and Puzicha, J. 2002. Shape Matching and Object Recognition Using Shape Contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4. Google ScholarDigital Library
5. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast Approximate Energy Minimization via Graph Cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11. Google ScholarDigital Library
6. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graphics 28, 3. Google ScholarDigital Library
7. Duygulu, P., Barnard, K., de Freitas, N., and Forsyth, D. 2002. Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In Proc. ECCV. Google ScholarDigital Library
8. Friedman, J., Hastie, T., and Tibshirani, R. 2000. Additive Logistic Regression: a Statistical View of Boosting. The Annals of Statistics 38, 2.Google Scholar
9. Fu, H., Cohen-Or, D., Dror, G., and Sheffer, A. 2008. Upright Orientation of Man-made Objects. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
10. Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1. Google ScholarDigital Library
11. Gama, J., and Brazdil, P. 2000. Cascade Generalization. Mach. Learn. 41, 3. Google ScholarDigital Library
12. Geman, S., and Geman, D. 1984. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Trans. PAMI 6, 6, 721–741.Google ScholarDigital Library
13. Golovinskiy, A., and Funkhouser, T. 2008. Randomized Cuts for 3D Mesh Analysis. ACM Trans. on Graph. 27, 5. Google ScholarDigital Library
14. Golovinskiy, A., and Funkhouser, T. 2009. Consistent Segmentation of 3D Models. Proc. SMI 33, 3.Google Scholar
15. Golovinskiy, A., Kim, V. G., and Funkhouser, T. 2009. Shape-based Recognition of 3D Point Clouds in Urban Environments. In Proc. ICCV.Google Scholar
16. He, X., Zemel, R., and Carreira-Perpiñán, M. A. 2004. Multiscale Conditional Random Fields for Image Labeling. In Proc. CVPR, vol. 2. Google ScholarDigital Library
17. Hilaga, M., Shinagawa, Y., Kohmura, T., and Kunii, T. L. 2001. Topology Matching for Fully Automatic Similarity Estimation of 3d Shapes. In SIGGRAPH. Google ScholarDigital Library
18. Huang, Q., Wicke, M., Adams, B., and Guibas, L. 2009. Shape Decomposition Using Modal Analysis. J. Computer Graphics Forum 28.Google Scholar
19. Igarashi, T., Matsuoka, S., and Tanaka, H. 2007. Teddy: A Sketching Interface for 3d Freeform Design. In SIGGRAPH. Google ScholarDigital Library
20. Johnson, A., and Hebert, M. 1999. Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes. IEEE Trans. PAMI 21, 5, 433–449. Google ScholarDigital Library
21. Katz, S., and Tal, A. 2003. Hierarchical Mesh Decomposition Using Fuzzy Clustering and Cuts. ACM Trans. Graphics. Google ScholarDigital Library
22. Katz, S., Leifman, G., and Tal, A. 2005. Mesh segmentation using feature point and core extraction. Visual Computer 21, 8.Google ScholarCross Ref
23. Konishi, S., and Yuille, A. 2000. Statistical Cues for Domain Specific Image Segmentation With Performance Analysis. Proc. CVPR.Google Scholar
24. Kraevoy, V., Julius, D., and Sheffer, A. 2007. Model Composition From Interchangeable Components. In Proc. PG. Google ScholarDigital Library
25. Kumar, S., and Hebert, M. 2003. Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification. In Proc. ICCV. Google ScholarDigital Library
26. Lafferty, J. D., McCallum, A., and Pereira, F. C. N. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML. Google ScholarDigital Library
27. Lai, Y.-K., Hu, S.-M., Martin, R. R., and Rosin, P. L. 2008. Fast Mesh Segmentation Using Random Walks. In ACM symposium on Solid and Physical Modeling. Google ScholarDigital Library
28. Lavoué, G., and Wolf, C. 2008. Markov Random Fields for Improving 3D Mesh Analysis and segmentation. In Eurographics workshop on 3D object retrieval. Google ScholarDigital Library
29. Li, X., Gu, X., and Qin, H. 2008. Surface matching using consistent pants decomposition. In ACM Symposium on Solid and Physical Modeling. Google ScholarDigital Library
30. Lim, E., and Suter, D. 2007. Conditional Random Field for 3D Point Clouds With Adaptive Data Reduction. In Cyberworlds. Google ScholarDigital Library
31. Lin, H.-Y. S., Liao, H.-Y. M., and Lin, J.-C. 2007. Visual Salience-Guided Mesh Decomposition. IEEE Transactions on Multimedia 9, 1. Google ScholarDigital Library
32. Liu, R., and Zhang, H. 2004. Segmentation of 3D Meshes Through Spectral Clustering. In Proc. PG. Google ScholarDigital Library
33. Liu, R. F., Zhang, H., Shamir, A., and Cohen-Or, D. 2009. A Part-Aware Surface Metric for Shape Analysis. Computer Graphics Forum, (Eurographics 2009) 28, 2.Google Scholar
34. Mangan, A. P., and Whitaker, R. T. 1999. Partitioning 3D Surface Meshes Using Watershed Segmentation. IEEE Trans. on Vis. and Comp. Graph. 5, 4. Google ScholarDigital Library
35. Munoz, D., Vandapel, N., and Hebert, M. 2008. Directional Associative Markov Network for 3-D Point Cloud Classification. In Proc. 3DPVT.Google Scholar
36. Pekelny, Y., and Gotsman, C. 2008. Articulated Object Reconstruction and Markerless Motion Capture from Depth Video. J. Computer Graphics Forum 27, 399–408.Google ScholarCross Ref
37. Schnitman, Y., Caspi, Y., Cohen-or, D., and Lischinski, D. 2006. Inducing Semantic Segmentation From an Example. In Proc. ACCV. Google ScholarDigital Library
38. Shamir, A. 2008. A Survey on Mesh Segmentation Techniques. Computer Graphics Forum 26, 6.Google Scholar
39. Shapira, L., Shalom, S., Shamir, A., Zhang, R. H., and Cohen-Or, D. In Press. Contextual Part Analogies in 3D Objects. International Journal of Computer Vision. Google ScholarDigital Library
40. Shlafman, S., Tal, A., and Katz, S. 2002. Metamorphosis of Polyhedral Surfaces Using Decomposition. In Eurographics.Google Scholar
41. Shotton, J., Johnson, M., and Cipolla, R. 2008. Semantic Texton Forests for Image Categorization and Segmentation. In Proc. CVPR.Google Scholar
42. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2009. TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context. Int. J. Comput. Vision 81, 1. Google ScholarDigital Library
43. Simari, P., Kalogerakis, E., and Singh, K. 2006. Folding Meshes: Hierarchical Mesh Segmentation Based on Planar Symmetry. In SGP. Google ScholarDigital Library
44. Simari, P., Nowrouzezahrai, D., Kalogerakis, E., and Singh, K. 2009. Multi-objective shape segmentation and labeling. Computer Graphics Forum 28, 5.Google ScholarDigital Library
45. Torralba, A., Murphy, K. P., and Freeman, W. T. 2007. Sharing Visual Features for Multiclass and Multiview Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5. Google ScholarDigital Library
46. Tu, Z., Chen, X., Yuille, A., and Zhu, S.-C. 2005. Image Parsing: Unifying Segmentation, Detection, and Recognition. International Journal of Computer Vision 63, 2. Google ScholarDigital Library
47. Tu, Z. 2008. Auto-context and its Application to High-level Vision Tasks. In Proc. CVPR.Google Scholar
48. Zhang, E., Mischaikow, K., and Turk, G. 2005. Feature-based Surface Parameterization and Texture Mapping. ACM Trans. Graph. 24, 1. Google ScholarDigital Library