“Learning 3D mesh segmentation and labeling” by Kalogerakis, Hertzmann and Singh

  • ©Evangelos Kalogerakis, Aaron Hertzmann, and Karan Singh




    Learning 3D mesh segmentation and labeling



    This paper presents a data-driven approach to simultaneous segmentation and labeling of parts in 3D meshes. An objective function is formulated as a Conditional Random Field model, with terms assessing the consistency of faces with labels, and terms between labels of neighboring faces. The objective function is learned from a collection of labeled training meshes. The algorithm uses hundreds of geometric and contextual label features and learns different types of segmentations for different tasks, without requiring manual parameter tuning. Our algorithm achieves a significant improvement in results over the state-of-the-art when evaluated on the Princeton Segmentation Benchmark, often producing segmentations and labelings comparable to those produced by humans.


    1. Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., and Ng, A. 2005. Discriminative Learning of Markov Random Fields for Segmentation of 3D Scan Data. In CVPR. Google ScholarDigital Library
    2. Attene, M., Katz, S., Mortara, M., Patane, G., Spagnuolo, M., and Tal, A. 2006. Mesh Segmentation – A Comparative Study. In Proc. SMI. Google ScholarDigital Library
    3. Attene, M., Falcidieno, B., and Spagnuolo, M. 2006. Hierarchical Mesh Segmentation Based on Fitting Primitives. Vis. Comput. 22, 3. Google ScholarDigital Library
    4. Belongie, S., Malik, J., and Puzicha, J. 2002. Shape Matching and Object Recognition Using Shape Contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4. Google ScholarDigital Library
    5. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast Approximate Energy Minimization via Graph Cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11. Google ScholarDigital Library
    6. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graphics 28, 3. Google ScholarDigital Library
    7. Duygulu, P., Barnard, K., de Freitas, N., and Forsyth, D. 2002. Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary. In Proc. ECCV. Google ScholarDigital Library
    8. Friedman, J., Hastie, T., and Tibshirani, R. 2000. Additive Logistic Regression: a Statistical View of Boosting. The Annals of Statistics 38, 2.Google Scholar
    9. Fu, H., Cohen-Or, D., Dror, G., and Sheffer, A. 2008. Upright Orientation of Man-made Objects. ACM Trans. Graph. 27, 3. Google ScholarDigital Library
    10. Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1. Google ScholarDigital Library
    11. Gama, J., and Brazdil, P. 2000. Cascade Generalization. Mach. Learn. 41, 3. Google ScholarDigital Library
    12. Geman, S., and Geman, D. 1984. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images. IEEE Trans. PAMI 6, 6, 721–741.Google ScholarDigital Library
    13. Golovinskiy, A., and Funkhouser, T. 2008. Randomized Cuts for 3D Mesh Analysis. ACM Trans. on Graph. 27, 5. Google ScholarDigital Library
    14. Golovinskiy, A., and Funkhouser, T. 2009. Consistent Segmentation of 3D Models. Proc. SMI 33, 3.Google Scholar
    15. Golovinskiy, A., Kim, V. G., and Funkhouser, T. 2009. Shape-based Recognition of 3D Point Clouds in Urban Environments. In Proc. ICCV.Google Scholar
    16. He, X., Zemel, R., and Carreira-Perpiñán, M. A. 2004. Multiscale Conditional Random Fields for Image Labeling. In Proc. CVPR, vol. 2. Google ScholarDigital Library
    17. Hilaga, M., Shinagawa, Y., Kohmura, T., and Kunii, T. L. 2001. Topology Matching for Fully Automatic Similarity Estimation of 3d Shapes. In SIGGRAPH. Google ScholarDigital Library
    18. Huang, Q., Wicke, M., Adams, B., and Guibas, L. 2009. Shape Decomposition Using Modal Analysis. J. Computer Graphics Forum 28.Google Scholar
    19. Igarashi, T., Matsuoka, S., and Tanaka, H. 2007. Teddy: A Sketching Interface for 3d Freeform Design. In SIGGRAPH. Google ScholarDigital Library
    20. Johnson, A., and Hebert, M. 1999. Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes. IEEE Trans. PAMI 21, 5, 433–449. Google ScholarDigital Library
    21. Katz, S., and Tal, A. 2003. Hierarchical Mesh Decomposition Using Fuzzy Clustering and Cuts. ACM Trans. Graphics. Google ScholarDigital Library
    22. Katz, S., Leifman, G., and Tal, A. 2005. Mesh segmentation using feature point and core extraction. Visual Computer 21, 8.Google ScholarCross Ref
    23. Konishi, S., and Yuille, A. 2000. Statistical Cues for Domain Specific Image Segmentation With Performance Analysis. Proc. CVPR.Google Scholar
    24. Kraevoy, V., Julius, D., and Sheffer, A. 2007. Model Composition From Interchangeable Components. In Proc. PG. Google ScholarDigital Library
    25. Kumar, S., and Hebert, M. 2003. Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification. In Proc. ICCV. Google ScholarDigital Library
    26. Lafferty, J. D., McCallum, A., and Pereira, F. C. N. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML. Google ScholarDigital Library
    27. Lai, Y.-K., Hu, S.-M., Martin, R. R., and Rosin, P. L. 2008. Fast Mesh Segmentation Using Random Walks. In ACM symposium on Solid and Physical Modeling. Google ScholarDigital Library
    28. Lavoué, G., and Wolf, C. 2008. Markov Random Fields for Improving 3D Mesh Analysis and segmentation. In Eurographics workshop on 3D object retrieval. Google ScholarDigital Library
    29. Li, X., Gu, X., and Qin, H. 2008. Surface matching using consistent pants decomposition. In ACM Symposium on Solid and Physical Modeling. Google ScholarDigital Library
    30. Lim, E., and Suter, D. 2007. Conditional Random Field for 3D Point Clouds With Adaptive Data Reduction. In Cyberworlds. Google ScholarDigital Library
    31. Lin, H.-Y. S., Liao, H.-Y. M., and Lin, J.-C. 2007. Visual Salience-Guided Mesh Decomposition. IEEE Transactions on Multimedia 9, 1. Google ScholarDigital Library
    32. Liu, R., and Zhang, H. 2004. Segmentation of 3D Meshes Through Spectral Clustering. In Proc. PG. Google ScholarDigital Library
    33. Liu, R. F., Zhang, H., Shamir, A., and Cohen-Or, D. 2009. A Part-Aware Surface Metric for Shape Analysis. Computer Graphics Forum, (Eurographics 2009) 28, 2.Google Scholar
    34. Mangan, A. P., and Whitaker, R. T. 1999. Partitioning 3D Surface Meshes Using Watershed Segmentation. IEEE Trans. on Vis. and Comp. Graph. 5, 4. Google ScholarDigital Library
    35. Munoz, D., Vandapel, N., and Hebert, M. 2008. Directional Associative Markov Network for 3-D Point Cloud Classification. In Proc. 3DPVT.Google Scholar
    36. Pekelny, Y., and Gotsman, C. 2008. Articulated Object Reconstruction and Markerless Motion Capture from Depth Video. J. Computer Graphics Forum 27, 399–408.Google ScholarCross Ref
    37. Schnitman, Y., Caspi, Y., Cohen-or, D., and Lischinski, D. 2006. Inducing Semantic Segmentation From an Example. In Proc. ACCV. Google ScholarDigital Library
    38. Shamir, A. 2008. A Survey on Mesh Segmentation Techniques. Computer Graphics Forum 26, 6.Google Scholar
    39. Shapira, L., Shalom, S., Shamir, A., Zhang, R. H., and Cohen-Or, D. In Press. Contextual Part Analogies in 3D Objects. International Journal of Computer Vision. Google ScholarDigital Library
    40. Shlafman, S., Tal, A., and Katz, S. 2002. Metamorphosis of Polyhedral Surfaces Using Decomposition. In Eurographics.Google Scholar
    41. Shotton, J., Johnson, M., and Cipolla, R. 2008. Semantic Texton Forests for Image Categorization and Segmentation. In Proc. CVPR.Google Scholar
    42. Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2009. TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context. Int. J. Comput. Vision 81, 1. Google ScholarDigital Library
    43. Simari, P., Kalogerakis, E., and Singh, K. 2006. Folding Meshes: Hierarchical Mesh Segmentation Based on Planar Symmetry. In SGP. Google ScholarDigital Library
    44. Simari, P., Nowrouzezahrai, D., Kalogerakis, E., and Singh, K. 2009. Multi-objective shape segmentation and labeling. Computer Graphics Forum 28, 5.Google ScholarDigital Library
    45. Torralba, A., Murphy, K. P., and Freeman, W. T. 2007. Sharing Visual Features for Multiclass and Multiview Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 29, 5. Google ScholarDigital Library
    46. Tu, Z., Chen, X., Yuille, A., and Zhu, S.-C. 2005. Image Parsing: Unifying Segmentation, Detection, and Recognition. International Journal of Computer Vision 63, 2. Google ScholarDigital Library
    47. Tu, Z. 2008. Auto-context and its Application to High-level Vision Tasks. In Proc. CVPR.Google Scholar
    48. Zhang, E., Mischaikow, K., and Turk, G. 2005. Feature-based Surface Parameterization and Texture Mapping. ACM Trans. Graph. 24, 1. Google ScholarDigital Library

ACM Digital Library Publication: