“Understanding and Exploiting Object Interaction Landscapes” by Pirk, Krs, Hu, Rajasekaran, Benes, et al. …

  • ©Soren Pirk, Vojtech Krs, Kaimo Hu, Suren Deepak Rajasekaran, Bedrich Benes, Yusuke Yoshiyasu, and Leonidas (Leo) J. Guibas

Conference:


Type:


Title:

    Understanding and Exploiting Object Interaction Landscapes

Session/Category Title: Comparing 3D Shapes and Part

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Interactions play a key role in understanding objects and scenes for both virtual and real-world agents. We introduce a new general representation for proximal interactions among physical objects that is agnostic to the type of objects or interaction involved. The representation is based on tracking particles on one of the participating objects and then observing them with sensors appropriately placed in the interaction volume or on the interaction surfaces. We show how to factorize these interaction descriptors and project them into a particular participating object so as to obtain a new functional descriptor for that object, its interaction landscape, capturing its observed use in a spatiotemporal framework. Interaction landscapes are independent of the particular interaction and capture subtle dynamic effects in how objects move and behave when in functional use. Our method relates objects based on their function, establishes correspondences between shapes based on functional key points and regions, and retrieves peer and partner objects with respect to an interaction.

References:


    1. Adobe. 2015. Mixamo. Retrieved from https://www.mixamo.com/.Google Scholar
    2. R. Ali Al-Asqhar, T. Komura, and M. Geol Choi. 2013. Relationship descriptors for interactive motion adaptation. In Proceedings of ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’13). ACM, 45–53. Google ScholarDigital Library
    3. N. Amenta, M. Bern, and M. Kamvysselis. 1998. A new Voronoi-based surface reconstruction algorithm. In Proceedings of SIGGRAPH. ACM, New York, 415–421. Google ScholarDigital Library
    4. D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, and J. Davis. 2005. SCAPE: Shape completion and animation of people. In Proceedings of SIGGRAPH. Google ScholarDigital Library
    5. F. G. Ashby. 1992. Multidimensional Models of Perception and Cognition. L. Erlbaum. https://books.google.com/books?id=eeQh4ESeOfsCGoogle Scholar
    6. E. Bar-Aviv and E. Rivlin. 2006. Functional 3D object classification using simulation of embodied agent. In BMVC. British Machine Vision Association, 307–316.Google Scholar
    7. I. Baran and J. Popović. 2007. Automatic rigging and animation of 3D characters. ACM Trans. Graph. 26, 3 (July 2007), 72:1–72:8. Google ScholarDigital Library
    8. S. Biasotti, A. Cerri, A. Bronstein, and M. Bronstein. 2015. Recent trends, applications, and perspectives in 3D shape similarity assessment. Comp. Graph. Forum (2015). Google ScholarDigital Library
    9. A. W. Black and P. Taylor. 1997. Automatically clustering similar units for unit selection in speech synthesis. In Eurospeech97. 601–604.Google Scholar
    10. M. Caine. 1994. The design of shape interactions using motion constraints. In Proceedings of the 1994 IEEE International Conference on Robotics and Automation, Vol. 1. 366–371.Google ScholarCross Ref
    11. A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. 2015. ShapeNet: An information-rich 3D model repository. ArXiv e-prints (Dec. 2015).Google Scholar
    12. Y.-W. Chao, Z. Wang, Y. He, J. Wang, and J. Deng. 2015. HICO: A benchmark for recognizing human-object interactions in images. ICCV (2015). Google ScholarDigital Library
    13. D.-Y. Chen, X.-P. Tian, Y.-T. Shen, and M. Ouhyoung. 2003. On visual similarity based 3D model retrieval. Comp. Graph. Forum 22, 3 (2003), 223–232.Google ScholarCross Ref
    14. J. Chen, X. Ge, L.-Y. Wei, B. Wang, Y. Wang, H. Wang, Y. Fei, K.-L. Qian, J.-H. Yong, and W. Wang. 2013. Bilateral blue noise sampling. ACM Trans. Graph. 32, 6, Article 216 (Nov. 2013), 11 pages. Google ScholarDigital Library
    15. S. Durrleman. 2010. Statistical Models of Currents for Measuring the Variability of Anatomical Curves, Surfaces and Their Evolution. Ph.D. dissertation. Université Nice – Sophia Antipolis, France.Google Scholar
    16. M. Fisher, M. Savva, Y. Li, P. Hanrahan, and M. Niessner. 2015. Activity-centric scene synthesis for functional 3D scene modeling. ACM Trans. Graph. 34, 6, Article 179 (Oct. 2015), 13 pages. Google ScholarDigital Library
    17. R. Gal and D. Cohen-Or. 2006. Salient geometric features for partial shape matching and similarity. ACM Trans. Graph. 25, 1 (2006), 130–150. Google ScholarDigital Library
    18. H. Grabner, J. Gall, and L. Van Gool. 2011. What makes a chair a chair?. In CVPR. 1529–1536. Google ScholarDigital Library
    19. A. Gupta, A. Kembhavi, and L. S. Davis. 2009. Observing human-object interactions: Using spatial and functional compatibility for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31, 10 (Oct. 2009), 1775–1789. Google ScholarDigital Library
    20. E. S. L. Ho, T. Komura, and C.-L. Tai. 2010. Spatial relationship preserving character motion adaptation. ACM Trans. Graph. 29, 4, Article 33 (2010), 8 pages. Google ScholarDigital Library
    21. R. Hu, O. van Kaick, B. Wu, H. Huang, A. Shamir, and H. Zhang. 2016. Learning how objects function via co-analysis of interactions. ACM Trans. Graph. 35, 4, Article 47 (July 2016), 13 pages. Google ScholarDigital Library
    22. R. Hu, C. Zhu, O. van Kaick, L. Liu, A. Shamir, and H. Zhang. 2015. Interaction context (ICON): Towards a geometric functionality descriptor. ACM Trans. Graph. 34, 4, Article 83 (2015), 12 pages. Google ScholarDigital Library
    23. H. Jiang and D. R. Martin. 2008. Finding actions using shape flows. In Computer Vision ECCV 2008, David Forsyth, Philip Torr, and Andrew Zisserman (Eds.). Lecture Notes in Computer Science, Vol. 5303. Springer, Berlin, 278–292. Google ScholarDigital Library
    24. Germany Karlsruhe Institute of Technology, Karlsruhe. 2015. KIT Whole Human Motion Database. Retrieved from https://motion-database.humanoids.kit.edu/.Google Scholar
    25. V. G. Kim, S. Chaudhuri, L. Guibas, and T. Funkhouser. 2014. Shape2Pose: Human-centric shape analysis. ACM Trans. Graph. 33, 4, Article 120 (2014), 12 pages. Google ScholarDigital Library
    26. H. Laga, M. Mortara, and M. Spagnuolo. 2013. Geometry and context for semantic correspondences and functionality recognition in man-made 3D shapes. ACM Trans. Graph. 32, 5, Article 150 (2013), 16 pages. Google ScholarDigital Library
    27. C. H. Lee, A. Varshney, and D. W. Jacobs. 2005. Mesh saliency. ACM Trans. Graph. 24, 3 (July 2005), 659–666. Google ScholarDigital Library
    28. M. Leordeanu and M. Hebert. 2005. A spectral technique for correspondence problems using pairwise constraints. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). Vol. 2. 1482–1489. Google ScholarDigital Library
    29. P. Li, B. Wang, F. Sun, X. Guo, C. Zhang, and W. Wang. 2015. Q-MAT: Computing medial axis transform by quadratic error minimization. ACM Trans. Graph. 35, 1, Article 8 (Dec. 2015), 16 pages. Google ScholarDigital Library
    30. Y. Li, J. L. Fu, and N. S. Pollard. 2007. Data-driven grasp synthesis using shape matching and task-based pruning. IEEE Trans. Vis. Comput. Graph. 13, 4 (July 2007), 732–747. Google ScholarDigital Library
    31. Z. Liu, C. Xie, S. Bu, X. Wang, J. Han, H. Lin, and H. Zhang. 2015. Indirect shape analysis for 3D shape retrieval. Comput. Graphics 46 (2015), 110–116. Shape Modeling International 2014. Google ScholarDigital Library
    32. N. Mitra, M. Wand, H. (Richard) Zhang, D. Cohen-Or, V. Kim, and Q.-X. Huang. 2013b. Structure-aware shape processing. In SIGGRAPH Asia 2013 Courses (SA’13). ACM, New York, Article 1, 20 pages. Google ScholarDigital Library
    33. N. J. Mitra, M. Pauly, M. Wand, and D. Ceylan. 2013a. Symmetry in 3D geometry: Extraction and applications. Comput. Graph. Forum 32, 6 (2013), 1–23. Google ScholarDigital Library
    34. J. J. Monaghan. 1992. Smoothed particle hydrodynamics. Ann. Rev. Astron. Astrophys. 30 (1992), 543–574.Google Scholar
    35. O. Oreifej and Z. Liu. 2013. HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’13). 716–723. Google ScholarDigital Library
    36. R. Osada, T. Funkhouser, B. Chazelle, and D. Dobkin. 2002. Shape distributions. ACM Trans. Graph. 21, 4 (Oct. 2002), 807–832. Google ScholarDigital Library
    37. M. Ovsjanikov, M. Ben-Chen, J. Solomon, A. Butscher, and L. Guibas. 2012. Functional maps: A flexible representation of maps between shapes. ACM Trans. Graph. 31, 4 (2012). Google ScholarDigital Library
    38. M. Pechuk, O. Soldea, and E. Rivlin. 2008. Learning function-based object classification from 3D imagery. Comput. Vis. Image Underst. 110, 2 (May 2008), 173–191. Google ScholarDigital Library
    39. E. Rivlin, S. J. Dickinson, and A, Rosenfeld. 1994. Recognition by functional parts {function-based object recognition}. In Proceedings of the 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’94). 267–274.Google ScholarCross Ref
    40. Columbia University USA Robotics Lab, Computer Science Laboratory. 2012. GraspIt! (2012). http://www.cs.columbia.edu/ cmatei/graspit/.Google Scholar
    41. Y. Rubner, C. Tomasi, and L. J. Guibas. 1998. A metric for distributions with applications to image databases. In Proceedings of the 6th International Conference on Computer Vision, 1998. 59–66. Google ScholarDigital Library
    42. M. Savva, A. X. Chang, P. Hanrahan, M. Fisher, and M. Nießner. 2014. SceneGrok: Inferring action maps in 3D environments. ACM Trans. Graph. 33, 6, Article 212 (Nov. 2014), 10 pages. Google ScholarDigital Library
    43. M. Savva, A. X. Chang, P. Hanrahan, M. Fisher, and M. Nießner. 2016. PiGraphs: Learning interaction snapshots from observations. ACM Trans. Graph. 35, 4, Article 139 (July 2016), 12 pages. Google ScholarDigital Library
    44. P. Shilane and T. Funkhouser. 2007. Distinctive regions of 3D surfaces. ACM Trans. Graph. 26, 2, Article 7 (June 2007). Google ScholarDigital Library
    45. O. Sidi, O. van Kaick, Y. Kleiman, H. Zhang, and D. Cohen-Or. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans. Graph. 30, 6, Article 126 (2011), 10 pages. Google ScholarDigital Library
    46. H. O. Song, M. Fritz, C. Gu, and T. Darrell. 2011. Visual grasp affordances from appearance-based cues. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). 998–1005.Google Scholar
    47. M. Sutton, L. Stark, and K. Bowyer. 1994. GRUFF-3: Generalizing the domain of a function-based recognition system. Pattern Recog. 27, 12 (1994), 1743–1766.Google ScholarCross Ref
    48. A. Tevs, Q. Huang, M. Wand, H.-P. Seidel, and L. Guibas. 2014. Relating shapes via geometric symmetries and regularities. ACM Trans. Graph. 33, 4, Article 119 (July 2014), 12 pages. Google ScholarDigital Library
    49. A. Tversky. 1977. Features of similarity. Psych. Rev. 84 (1977), 327–352.Google Scholar
    50. D. Tzionas and J. Gall. 2015. 3D object reconstruction from hand-object interactions. In Proceedings of the International Conference on Computer Vision (ICCV). http://files.is.tue.mpg.de/dtzionas/In-Hand-Scanning Google ScholarDigital Library
    51. Z. Wu, S. Song, A. Khosla, X. Tang, and J. Xiao. 2014. 3D shapenets for 2.5d object recognition and next-best-view prediction. CoRR abs/1406.5670 (2014). http://arxiv.org/abs/1406.5670Google Scholar
    52. L. Xu, H. Quynh Dinh, P. Mordohai, and T. Ramsay. 2011. Detecting patterns in vector fields. 49th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition 6 (2011).Google ScholarCross Ref
    53. X. Zhao, H. Wang, and T. Komura. 2014. Indexing 3D scenes using the interaction bisector surface. ACM Trans. Graph. 33, 3, Article 22 (2014), 14 pages. Google ScholarDigital Library
    54. Y. Zheng, D. Cohen-Or, and N. J. Mitra. 2013. Smart variations: Functional substructures for part compatibility. Comp. Graph. Forum 32, 2 pt2 (2013), 195–204.Google ScholarCross Ref
    55. Y. Zhu, A. Fathi, and L. Fei-Fei. 2014. Reasoning about object affordances in a knowledge base representation. In Computer Vision ECCV 2014, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars (Eds.). Lecture Notes in Computer Science, Vol. 8690. Springer International Publishing, 408–424.Google Scholar

ACM Digital Library Publication:


Overview Page: