“Tactile mesh saliency”

  • ©Manfred Lau, Kapil Dev, Weiqi Shi, Julie Dorsey, and Holly E. Rushmeier



Session Title:



    Tactile mesh saliency




    While the concept of visual saliency has been previously explored in the areas of mesh and image processing, saliency detection also applies to other sensory stimuli. In this paper, we explore the problem of tactile mesh saliency, where we define salient points on a virtual mesh as those that a human is more likely to grasp, press, or touch if the mesh were a real-world object. We solve the problem of taking as input a 3D mesh and computing the relative tactile saliency of every mesh vertex. Since it is difficult to manually define a tactile saliency measure, we introduce a crowdsourcing and learning framework. It is typically easy for humans to provide relative rankings of saliency between vertices rather than absolute values. We thereby collect crowdsourced data of such relative rankings and take a learning-to-rank approach. We develop a new formulation to combine deep learning and learning-to-rank methods to compute a tactile saliency measure. We demonstrate our framework with a variety of 3D meshes and various applications including material suggestion for rendering and fabrication.


    1. Bächer, M., Bickel, B., James, D. L., and Pfister, H. 2012. Fabricating Articulated Characters from Skinned Meshes. ACM Trans. Graph. 31, 4 (July), 47:1–47:9. Google ScholarDigital Library
    2. Bohg, J., Morales, A., Asfour, T., and Kragic, D. 2014. Data-Driven Grasp Synthesis — A Survey. IEEE Transactions on Robotics 30, 2, 289–309. Google ScholarDigital Library
    3. Borji, A., Sihite, D. N., and Itti, L. 2012. Salient Object Detection: A Benchmark. ECCV, 414–429.Google Scholar
    4. Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva, A., and Torralba, A., 2015. MIT Saliency Benchmark. http://saliency.mit.edu/.Google Scholar
    5. Chapelle, O., and Keerthi, S. S. 2010. Efficient Algorithms for Ranking with SVMs. Information Retrieval 13, 3, 201–215. Google ScholarDigital Library
    6. Chen, D. Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum 22, 3, 223–232.Google ScholarCross Ref
    7. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graph. 28, 3 (July), 73:1–73:12. Google ScholarDigital Library
    8. Chen, X., Saparov, A., Pang, B., and Funkhouser, T. 2012. Schelling Points on 3D Surface Meshes. ACM Trans. Graph. 31, 4 (July), 29:1–29:12. Google ScholarDigital Library
    9. Cignoni, P., Pietroni, N., Malomo, L., and Scopigno, R. 2014. Field-aligned Mesh Joinery. ACM Trans. Graph. 33, 1 (Feb.), 11:1–11:12. Google ScholarDigital Library
    10. Dorsey, J., and Hanrahan, P. 1996. Modeling and Rendering of Metallic Patinas. In Proceedings of SIGGRAPH 96, Annual Conference Series, 387–396. Google ScholarDigital Library
    11. Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1 (Jan.), 130–150. Google ScholarDigital Library
    12. Garces, E., Agarwala, A., Gutierrez, D., and Hertzmann, A. 2014. A Similarity Measure for Illustration Style. ACM Trans. Graph. 33, 4 (July), 93:1–93:9. Google ScholarDigital Library
    13. Gingold, Y., Shamir, A., and Cohen-Or, D. 2012. Micro Perceptual Human Computation for Visual Tasks. ACM Trans. Graph. 31, 5 (Sept.), 119:1–119:12. Google ScholarDigital Library
    14. Gingold, Y., Vouga, E., Grinspun, E., and Hirsh, H. 2012. Diamonds from the Rough: Improving Drawing, Painting, and Singing via Crowdsourcing. Proceedings of the AAAI Workshop on Human Computation (HCOMP).Google Scholar
    15. Goferman, S., Zelnik-Manor, L., and Tal, A. 2012. Context-Aware Saliency Detection. PAMI 34, 10, 1915–1926. Google ScholarDigital Library
    16. Goldfeder, C., Ciocarlie, M., Dang, H., and Allen, P. K. 2009. The Columbia Grasp Database. ICRA, 3343–3349. Google ScholarDigital Library
    17. Hildebrand, K., Bickel, B., and Alexa, M. 2013. Orthogonal Slicing for Additive Manufacturing. SMI 37, 6, 669–675. Google ScholarDigital Library
    18. Hisada, M., Belyaev, A. G., and Kunii, T. L. 2002. A Skeleton-based Approach for Detection of Perceptually Salient Features on Polygonal Surfaces. CGF 21, 4, 689–700.Google ScholarCross Ref
    19. Horn, B. 1984. Extended Gaussian Images. Proceedings of the IEEE 72, 12, 1671–1686.Google ScholarCross Ref
    20. Howlett, S., Hamill, J., and O’Sullivan, C. 2005. Predicting and Evaluating Saliency for Simplified Polygonal Models. ACM Trans. on Applied Perception 2, 3 (July), 286–308. Google ScholarDigital Library
    21. Hu, J., Lu, J., and Tan, Y. P. 2014. Discriminative Deep Metric Learning for Face Verification in the Wild. CVPR, 1875–1882. Google ScholarDigital Library
    22. Hu, J., Lu, J., and Tan, Y. P. 2015. Deep Transfer Metric Learning. CVPR, 325–333.Google Scholar
    23. Itti, L., Koch, C., and Niebur, E. 1998. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. PAMI 20, 11, 1254–1259. Google ScholarDigital Library
    24. Itti, L., 2000. Models of Bottom-Up and Top-Down Visual Attention. PhD Thesis, California Institute of Technology Pasadena. Google ScholarDigital Library
    25. Jain, A., Thormählen, T., Ritschel, T., and Seidel, H.-P. 2012. Material Memex: Automatic Material Suggestions for 3D Objects. ACM Trans. Graph. 31, 6 (Nov.), 143:1–143:8. Google ScholarDigital Library
    26. Järvelin, K., and Kekälänen, J. 2002. Cumulated Gain-based Evaluation of IR Techniques. ACM Trans. on Information Systems 20, 4, 422–446. Google ScholarDigital Library
    27. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D Mesh Segmentation and Labeling. ACM Trans. Graph. 29, 4 (July), 102:1–102:12. Google ScholarDigital Library
    28. Kim, Y., Varshney, A., Jacobs, D. W., and Guimbretière, F. 2010. Mesh Saliency and Human Eye Fixations. ACM Trans. on Applied Perception 7, 2 (Feb.), 12:1–12:13. Google ScholarDigital Library
    29. Klank, U., Pangercic, D., Rusu, R., and Beetz, M. 2009. Real-time CAD Model Matching for Mobile Manipulation and Grasping. IEEE-RAS Int’l Conf. on Humanoid Robots, 290–296.Google Scholar
    30. Lau, M., Ohgawara, A., Mitani, J., and Igarashi, T. 2011. Converting 3D Furniture Models to Fabricatable Parts and Connectors. ACM Trans. Graph. 30, 4 (July), 85:1–85:6. Google ScholarDigital Library
    31. Lee, C. H., Varshney, A., and Jacobs, D. W. 2005. Mesh Saliency. ACM Trans. Graph. 24, 3 (July), 659–666. Google ScholarDigital Library
    32. Liu, Y., Zhu, H., Liu, X., and Wu, E. 2005. Real-time Simulation of Physically based On-Surface Flow. The Visual Computer 21, 8-10, 727–734.Google ScholarCross Ref
    33. Liu, T., Hertzmann, A., Li, W., and Funkhouser, T. 2015. Style Compatibility for 3D Furniture Models. ACM Trans. Graph. 34, 4 (July), 85:1–85:9. Google ScholarDigital Library
    34. Liu, Z., Wang, X., and Bu, S. 2015. Human-Centered Saliency Detection. IEEE Trans. on Neural Networks and Learning Systems.Google Scholar
    35. Mérillou, S., and Ghazanfarpour, D. 2008. A Survey of Aging and Weathering Phenomena in Computer Graphics. Computers & Graphics 32, 2, 159–174. Google ScholarDigital Library
    36. O’Donovan, P., Libeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory Font Selection Using Crowd-sourced Attributes. ACM Trans. Graph. 33, 4 (July), 92:1–92:9. Google ScholarDigital Library
    37. Osada, R., Funkhouser, T., Chazelle, B., and Dobkin, D. 2001. Matching 3D Models with Shape Distributions. Shape Modeling International, 154–166. Google ScholarDigital Library
    38. Parikh, D., and Grauman, K. 2011. Relative Attributes. International Conference on Computer Vision (ICCV), 503–510. Google ScholarDigital Library
    39. Plaisier, M. A., Tiest, W. M. B., and Kappers, A. M. L. 2009. Salient Features in 3-D Haptic Shape Perception. Attention, Perception, & Psychophysics 71, 2, 421–430.Google ScholarCross Ref
    40. Prévost, R., Whiting, E., Lefebvre, S., and Sorkine-Hornung, O. 2013. Make It Stand: Balancing Shapes for 3D Fabrication. ACM Trans. Graph. 32, 4 (July), 81:1–81:10. Google ScholarDigital Library
    41. Sahbani, A., and El-Khoury, S. 2009. A Hybrid Approach for Grasping 3D Objects. IROS, 1272–1277. Google ScholarDigital Library
    42. Sahbani, A., El-khoury, S., and Bidaud, P. 2012. An Overview of 3D Object Grasp Synthesis Algorithms. Robotics and Autonomous Systems 60, 3, 326–336. Google ScholarDigital Library
    43. Saxena, A., Driemeyer, J., Kearns, J., and Ng, A. Y. 2007. Robotic Grasping of Novel Objects. NIPS, 1209–1216.Google Scholar
    44. Schwartzburg, Y., and Pauly, M. 2013. Fabrication-aware Design with Intersecting Planar Pieces. CGF 32, 2, 317–326.Google ScholarCross Ref
    45. Shilane, P., and Funkhouser, T. 2007. Distinctive Regions of 3D Surfaces. ACM Trans. Graph. 26, 2 (June), 7. Google ScholarDigital Library
    46. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. SMI, 167–178. Google ScholarDigital Library
    47. Shtrom, E., Leifman, G., and Tal, A. 2013. Saliency Detection in Large Point Sets. ICCV, 3591–3598. Google ScholarDigital Library
    48. Song, R., Liu, Y., Martin, R. R., and Rosin, P. L. 2014. Mesh Saliency via Spectral Processing. ACM Trans. Graph. 33, 1 (Feb.), 6:1–6:17. Google ScholarDigital Library
    49. Su, H., Maji, S., Kalogerakis, E., and Learned-Miller, E. 2015. Multi-view Convolutional Neural Networks for 3D Shape Recognition. ICCV. Google ScholarDigital Library
    50. Surazhsky, T., Magid, E., Soldea, O., Elber, G., and Rivlin, E. 2003. A Comparison of Gaussian and Mean Curvatures Estimation Methods on Triangular Meshes. International Conference on Robotics and Automation, 1021–1026.Google Scholar
    51. Tao, P., Cao, J., Li, S., Liu, X., and Liu, L. 2015. Mesh Saliency via Ranking Unsalient Patches in a Descriptor Space. Computers and Graphics 46, C, 264–274. Google ScholarDigital Library
    52. Varadarajan, K., Potapova, E., and Vincze, M. 2012. Attention driven Grasping for Clearing a Heap of Objects. IROS, 2035–2042.Google Scholar
    53. Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., and Wu, Y. 2014. Learning Fine-Grained Image Similarity with Deep Ranking. CVPR, 1386–1393. Google ScholarDigital Library
    54. Watanabe, K., and Belyaev, A. G. 2001. Detection of Salient Curvature Features on Polygonal Surfaces. CGF 20, 3, 385–392.Google ScholarCross Ref
    55. Wei, L., Huang, Q., Ceylan, D., Vouga, E., and Li, H. 2015. Dense Human Body Correspondences Using Convolutional Networks. CVPR.Google Scholar
    56. Xu, M., Ni, B., Dong, J., Huang, Z., Wang, M., and Yan, S. 2012. Touch Saliency. ACM International Conference on Multimedia, 1041–1044. Google ScholarDigital Library
    57. Zagoruyko, S., and Komodakis, N. 2015. Learning to Compare Image Patches via Convolutional Neural Networks. CVPR.Google Scholar
    58. Zhou, Q., Panetta, J., and Zorin, D. 2013. Worst-case Structural Analysis. ACM Trans. Graph. 32, 4 (July), 137:1–137:12. Google ScholarDigital Library
    59. Zimmer, H., Lafarge, F., Alliez, P., and Kobbelt, L. 2014. Zometool Shape Approximation. Graphical Models 76, 5, 390–401. Google ScholarDigital Library

ACM Digital Library Publication: