“Geometry and Context for Semantic Correspondences and Functionality Recognition in Man-Made 3D Shapes” by Laga, Mortara and Spagnuolo

  • ©

Conference:


Type(s):


Title:

    Geometry and Context for Semantic Correspondences and Functionality Recognition in Man-Made 3D Shapes

Session/Category Title:   Shape Collection

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We address the problem of automatic recognition of functional parts of man-made 3D shapes in the presence of significant geometric and topological variations. We observe that under such challenging circumstances, the context of a part within a 3D shape provides important cues for learning the semantics of shapes. We propose to model the context as structural relationships between shape parts and use them, in addition to part geometry, as cues for functionality recognition. We represent a 3D shape as a graph interconnecting parts that share some spatial relationships. We model the context of a shape part as walks in the graph. Similarity between shape parts can then be defined as the similarity between their contexts, which in turn can be efficiently computed using graph kernels. This formulation enables us to: (1) find part-wise semantic correspondences between 3D shapes in a nonsupervised manner and without relying on user-specified textual tags, and (2) design classifiers that learn in a supervised manner the functionality of the shape components. We specifically show that the performance of the proposed context-aware similarity measure in finding part-wise correspondences outperforms geometry-only-based techniques and that contextual analysis is effective in dealing with shapes exhibiting large geometric and topological variations.

References:


    1. Aleotti, J. and Caselli, S. 2012. A 3d shape segmentation approach for robot grasping by parts. Robot. Auton. Syst. 60, 3, 358–366.
    2. Attene, M., Robbiano, F., Spagnuolo, M., and Falcidieno, B. 2009. Characterization of 3d shape parts for semantic annotation. Comput.-Aid. Des. 41, 10, 756–763.
    3. Au, O. K.-C., Tai, C.-L., Cohen-Or, D., Zheng, Y., and Fu, H. 2010. Electors voting for fast automatic shape correspondence. Comput. Graph. Forum 29. 645–654.
    4. Bach, F. R., Thibaux, R., and Jordan, M. I. 2005. Computing regularization paths for learning multiple kernels. In Advances in Neural Information Processing Systems (NIPS’05).
    5. Bohg, J. and Kragic, D. 2009. Grasping familiar objects using shape context. In International Conference on Advanced Robotics.
    6. Catalano, C. E., Mortara, M., Spagnuolo, M., and Falcidieno, B. 2011. Semantics and 3d media: Current issues and perspectives. Comput. Graph. 35, 4, 869–877.
    7. Chang, W. and Zwicker, M. 2009. Range scan registration using reduced deformable models. Comput. Graph. Forum 28, 2, 447–456.
    8. Chaudhuri, S., Kalogerakis, E., Guibas, L., and Koltun, V. 2011. Probabilistic reasoning for assembly-based 3d modeling. ACM Trans. Graph. 30, 35:1–35:10.
    9. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3d mesh segmentation. ACM Trans. Graph. 28, 3.
    10. Cos. 2012. The shape coseg dataset. http://web.siat.ac.cn/yunhai/ssl/ssd.htm.
    11. Fisher, M. and Hanrahan, P. 2010. Context-based search for 3d models. ACM Trans. Graph. 29, 182:1–182:10.
    12. Fisher, M., Savva, M., and Hanrahan, P. 2011. Characterizing structural relationships in scenes using graph kernels. ACM Trans. Graph. 30, 34:1–34:12.
    13. Fu, H., Cohen-Or, D., Dror, G., and Sheffer, A. 2008. Upright orientation of man-made objects. ACM Trans. Graph. 27, 42:1–42:7.
    14. Galleguillos, C. and Belongie, S. J. 2010. Context based object categorization: A critical survey. Comput. Vis. Image Understand. 114, 6, 712–722.
    15. Galleguillos, C., Rabinovich, A., and Belongie, S. 2008. Object categorization using co-occurrence, location and appearance. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
    16. Giorgi, D., Biasotti, S., and Paraboschi, L. 2007. SHREC: Shape retrieval contest: Watertight models track. Tech. rep. http://www.cs.uu.nl/profi/public/SummerSchool/3DRetrieval/USCS07-3dpart6_shrec07.pdf.
    17. Golovinskiy, A. and Funkhouser, T. A. 2009. Consistent segmentation of 3d models. Comput. Graph. 33, 3, 262–269.
    18. Harchaoui, Z. and Bach, F. 2007. Image classification with segmentation graphs. In IEEE Conference on Computer Vision and Pattern Recognition.
    19. Huang, Q., Koltun, V., and Guibas, L. 2011. Joint shape segmentation with linear programming. ACM Trans. Graph. 30, 125:1–125:12.
    20. Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Nonrigid registration under isometric deformations. In Proceedings of the Symposium on Geometry Processing (SGP’08). Eurographics Association, 1449–1457.
    21. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3d mesh segmentation and labeling. ACM Trans. Graph. 29, 4, 102:1–102:12.
    22. Katz, S. and Tal, A. 2003. Hierarchical mesh decomposition using fuzzy clustering and cuts. ACM Trans. Graph. 22, 3, 954–961.
    23. Kim, V. G., Lipman, Y., and Funkhouser, T. 2011. Blended intrinsic maps. ACM Trans. Graph. 30, 79:1–79:12.
    24. Kyota, F., Watabe, T., Saito, S., and Nakajima, M. 2005. Detection and evaluation of grasping positions for autonomous agents. In Proceedings of the 4th International Conference on Cyberworlds. 453–460.
    25. Laga, H. 2010. Semantics-driven approach for automatic selection of best views of 3d shapes. In Proceedings of the 3rd Eurographics Conference on 3D Object Retrieval (EG3DOR’10). Eurographics Association, 15–22.
    26. Laga, H. 2011. Data-driven approach for automatic orientation of 3d shapes. Vis. Comput. 27, 11, 977–989.
    27. Laga, H., Takahashi, H., and Nakajima, M. 2006. Spherical wavelet descriptors for content-based 3D model retrieval. In Proceedings of the IEEE International Conference on Shape Modeling and Applications. 15.
    28. Li, Y., Wu, X., Chrysathou, Y., Sharf, A., Cohen-Or, D., and Mitra, N. J. 2011. Globfit: Consistently fitting primitives by discovering global relations. ACM Trans. Graph. 30, 4, 52:1–52:12.
    29. Lipman, Y. and Funkhouser, T. 2009. Mobius voting for surface correspondence. ACM Trans. Graph. 28, 72:1–72:12.
    30. Malisiewicz, T. and Efros, A. 2009. Beyond categories: The visual memex model for reasoning about object relationships. In Neural Information Processing Systems. http://repository.cmu.edu/cgi/viewcontent.cgi?article=1784&context=robotics.
    31. Mitra, N. J., Guibas, L. J., and Pauly, M. 2006. Partial and approximate symmetry detection for 3d geometry. ACM Trans. Graph. 25, 3, 560–568.
    32. Mortara, M., Patane, G., and Spagnuolo, M. 2006. From geometric to semantic human body models. Comput. Graph. 30, 2, 185–196.
    33. Mortara, M., Patane, G., Spagnuolo, M., Falcidieno, B., and Rossignac, J. 2004. Plumber: A method for a multi-scale decomposition of 3d shapes into tubular primitives and bodies. In Proceedings of the 9th ACM Symposium on Solid Modeling and Applications. 339–344.
    34. Oliva, A. and Torralba, A. 2007. The role of context in object recognition. Trends Cogn. Sci. 11, 520–527.
    35. Osada, R., Funkhouser, T. A., Chazelle, B., and Dobkin, D. P. 2002. Shape distributions. ACM Trans. Graph. 21, 807–832.
    36. Ovsjanikov, M., Li, W., Guibas, L., and Mitra, N. J. 2011. Exploration of continuous variability in collections of 3d shapes. ACM Trans. Graph. 30, 33:1–33:10.
    37. Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. In Proceedings of the 11th International Conference on Computer Vision (ICCV’07). 1–8.
    38. Sahbani, A. and El-Khoury, S. 2009. A hybrid approach for grasping 3d objects. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’09). IEEE Press, 1272–1277.
    39. Shapira, L., Shalom, S., Shamir, A., Cohen-Or, D., and Zhang, H. 2010. Contextual part analogies in 3d objects. Int. J. Comput. Vis. 89, 1–2, 309–326.
    40. Shapira, L., Shamir, A., and Cohen-Or, D. 2008. Consistent mesh partitioning and skeletonisation using the shape diameter function. Vis. Comput. 24, 249–259.
    41. Shilane, P., Min, P., Kazhda, M., and Funkhouser, T. 2004. The princeton shape benchmark. In Proceedings of the International Conference on Shape Modeling and Applications. 167–178.
    42. Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptorspace clustering. ACM Trans. Graph. 30, 6, 126:1–126:9.
    43. Strat, T. M. and Fischler, M. A. 1991. Context-based vision: Recognizing objects using information from both 2d and 3d imagery. IEEE Trans. Pattern Anal. Mach. Intell. 13, 1050–1065.
    44. van Kaick, O., Tagliasacchi, A., Sidi, O., Zhang, H., Cohen-Or, D., Wolf, L., and Hamarneh, G. 2011. Prior knowledge for shape correspondence. Comput. Graph. Forum 30, 2, 553–562.
    45. van Kaick, O., Zhang, H., Hamarneh, G., and Cohen-Or, D. 2010. A survey on shape correspondence. Comput. Graph. Forum 30, 2, 553–562.
    46. Wang, Y., Xu, K., Li, J., Zhang, H., Shamir, A., Liu, L., Cheng, Z.-Q., and Xiong, Y. 2011. Symmetry hierarchy of man-made objects. Comput. Graph. Forum 30, 2, 287–296.
    47. Zeng, Y., Wang, C., Wang, Y., Gu, X., Samaras, D., and Paragios, N. 2010. Dense non-rigid surface registration using high-order graph matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 382–389.
    48. Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., van Kaick, O., and Tagliasacchi, A. 2008. Deformation-driven shape correspondence. In Proceedings of the Symposium on Geometry Processing (SGP’08). Eurographics Association, 1431–1439.
    49. Zheng, Y., Cohen-Or, D., and Mitra, N. J. 2013. Smart variations: Functional substructures for part compatibility. Comput. Graph. Forum 32, 2, 133–263.
    50. Zheng, Y., Fu, H., Cohen-Or, D., Au, O. K.-C., and Tai, C.-L. 2011. Component-wise controllers for structure-preserving shape manipulation. Comput. Graph. Forum 30, 2, 563–572.

ACM Digital Library Publication:


Overview Page: