“Tactile mesh saliency”
Conference:
Type(s):
Title:
- Tactile mesh saliency
Session/Category Title: PERCEPTION OF SHAPES AND PEOPLE
Presenter(s)/Author(s):
Moderator(s):
Abstract:
While the concept of visual saliency has been previously explored in the areas of mesh and image processing, saliency detection also applies to other sensory stimuli. In this paper, we explore the problem of tactile mesh saliency, where we define salient points on a virtual mesh as those that a human is more likely to grasp, press, or touch if the mesh were a real-world object. We solve the problem of taking as input a 3D mesh and computing the relative tactile saliency of every mesh vertex. Since it is difficult to manually define a tactile saliency measure, we introduce a crowdsourcing and learning framework. It is typically easy for humans to provide relative rankings of saliency between vertices rather than absolute values. We thereby collect crowdsourced data of such relative rankings and take a learning-to-rank approach. We develop a new formulation to combine deep learning and learning-to-rank methods to compute a tactile saliency measure. We demonstrate our framework with a variety of 3D meshes and various applications including material suggestion for rendering and fabrication.
References:
1. Bächer, M., Bickel, B., James, D. L., and Pfister, H. 2012. Fabricating Articulated Characters from Skinned Meshes. ACM Trans. Graph. 31, 4 (July), 47:1–47:9. Google ScholarDigital Library
2. Bohg, J., Morales, A., Asfour, T., and Kragic, D. 2014. Data-Driven Grasp Synthesis — A Survey. IEEE Transactions on Robotics 30, 2, 289–309. Google ScholarDigital Library
3. Borji, A., Sihite, D. N., and Itti, L. 2012. Salient Object Detection: A Benchmark. ECCV, 414–429.Google Scholar
4. Bylinskii, Z., Judd, T., Borji, A., Itti, L., Durand, F., Oliva, A., and Torralba, A., 2015. MIT Saliency Benchmark. http://saliency.mit.edu/.Google Scholar
5. Chapelle, O., and Keerthi, S. S. 2010. Efficient Algorithms for Ranking with SVMs. Information Retrieval 13, 3, 201–215. Google ScholarDigital Library
6. Chen, D. Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On Visual Similarity Based 3D Model Retrieval. Computer Graphics Forum 22, 3, 223–232.Google ScholarCross Ref
7. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A Benchmark for 3D Mesh Segmentation. ACM Trans. Graph. 28, 3 (July), 73:1–73:12. Google ScholarDigital Library
8. Chen, X., Saparov, A., Pang, B., and Funkhouser, T. 2012. Schelling Points on 3D Surface Meshes. ACM Trans. Graph. 31, 4 (July), 29:1–29:12. Google ScholarDigital Library
9. Cignoni, P., Pietroni, N., Malomo, L., and Scopigno, R. 2014. Field-aligned Mesh Joinery. ACM Trans. Graph. 33, 1 (Feb.), 11:1–11:12. Google ScholarDigital Library
10. Dorsey, J., and Hanrahan, P. 1996. Modeling and Rendering of Metallic Patinas. In Proceedings of SIGGRAPH 96, Annual Conference Series, 387–396. Google ScholarDigital Library
11. Gal, R., and Cohen-Or, D. 2006. Salient Geometric Features for Partial Shape Matching and Similarity. ACM Trans. Graph. 25, 1 (Jan.), 130–150. Google ScholarDigital Library
12. Garces, E., Agarwala, A., Gutierrez, D., and Hertzmann, A. 2014. A Similarity Measure for Illustration Style. ACM Trans. Graph. 33, 4 (July), 93:1–93:9. Google ScholarDigital Library
13. Gingold, Y., Shamir, A., and Cohen-Or, D. 2012. Micro Perceptual Human Computation for Visual Tasks. ACM Trans. Graph. 31, 5 (Sept.), 119:1–119:12. Google ScholarDigital Library
14. Gingold, Y., Vouga, E., Grinspun, E., and Hirsh, H. 2012. Diamonds from the Rough: Improving Drawing, Painting, and Singing via Crowdsourcing. Proceedings of the AAAI Workshop on Human Computation (HCOMP).Google Scholar
15. Goferman, S., Zelnik-Manor, L., and Tal, A. 2012. Context-Aware Saliency Detection. PAMI 34, 10, 1915–1926. Google ScholarDigital Library
16. Goldfeder, C., Ciocarlie, M., Dang, H., and Allen, P. K. 2009. The Columbia Grasp Database. ICRA, 3343–3349. Google ScholarDigital Library
17. Hildebrand, K., Bickel, B., and Alexa, M. 2013. Orthogonal Slicing for Additive Manufacturing. SMI 37, 6, 669–675. Google ScholarDigital Library
18. Hisada, M., Belyaev, A. G., and Kunii, T. L. 2002. A Skeleton-based Approach for Detection of Perceptually Salient Features on Polygonal Surfaces. CGF 21, 4, 689–700.Google ScholarCross Ref
19. Horn, B. 1984. Extended Gaussian Images. Proceedings of the IEEE 72, 12, 1671–1686.Google ScholarCross Ref
20. Howlett, S., Hamill, J., and O’Sullivan, C. 2005. Predicting and Evaluating Saliency for Simplified Polygonal Models. ACM Trans. on Applied Perception 2, 3 (July), 286–308. Google ScholarDigital Library
21. Hu, J., Lu, J., and Tan, Y. P. 2014. Discriminative Deep Metric Learning for Face Verification in the Wild. CVPR, 1875–1882. Google ScholarDigital Library
22. Hu, J., Lu, J., and Tan, Y. P. 2015. Deep Transfer Metric Learning. CVPR, 325–333.Google Scholar
23. Itti, L., Koch, C., and Niebur, E. 1998. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. PAMI 20, 11, 1254–1259. Google ScholarDigital Library
24. Itti, L., 2000. Models of Bottom-Up and Top-Down Visual Attention. PhD Thesis, California Institute of Technology Pasadena. Google ScholarDigital Library
25. Jain, A., Thormählen, T., Ritschel, T., and Seidel, H.-P. 2012. Material Memex: Automatic Material Suggestions for 3D Objects. ACM Trans. Graph. 31, 6 (Nov.), 143:1–143:8. Google ScholarDigital Library
26. Järvelin, K., and Kekälänen, J. 2002. Cumulated Gain-based Evaluation of IR Techniques. ACM Trans. on Information Systems 20, 4, 422–446. Google ScholarDigital Library
27. Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D Mesh Segmentation and Labeling. ACM Trans. Graph. 29, 4 (July), 102:1–102:12. Google ScholarDigital Library
28. Kim, Y., Varshney, A., Jacobs, D. W., and Guimbretière, F. 2010. Mesh Saliency and Human Eye Fixations. ACM Trans. on Applied Perception 7, 2 (Feb.), 12:1–12:13. Google ScholarDigital Library
29. Klank, U., Pangercic, D., Rusu, R., and Beetz, M. 2009. Real-time CAD Model Matching for Mobile Manipulation and Grasping. IEEE-RAS Int’l Conf. on Humanoid Robots, 290–296.Google Scholar
30. Lau, M., Ohgawara, A., Mitani, J., and Igarashi, T. 2011. Converting 3D Furniture Models to Fabricatable Parts and Connectors. ACM Trans. Graph. 30, 4 (July), 85:1–85:6. Google ScholarDigital Library
31. Lee, C. H., Varshney, A., and Jacobs, D. W. 2005. Mesh Saliency. ACM Trans. Graph. 24, 3 (July), 659–666. Google ScholarDigital Library
32. Liu, Y., Zhu, H., Liu, X., and Wu, E. 2005. Real-time Simulation of Physically based On-Surface Flow. The Visual Computer 21, 8-10, 727–734.Google ScholarCross Ref
33. Liu, T., Hertzmann, A., Li, W., and Funkhouser, T. 2015. Style Compatibility for 3D Furniture Models. ACM Trans. Graph. 34, 4 (July), 85:1–85:9. Google ScholarDigital Library
34. Liu, Z., Wang, X., and Bu, S. 2015. Human-Centered Saliency Detection. IEEE Trans. on Neural Networks and Learning Systems.Google Scholar
35. Mérillou, S., and Ghazanfarpour, D. 2008. A Survey of Aging and Weathering Phenomena in Computer Graphics. Computers & Graphics 32, 2, 159–174. Google ScholarDigital Library
36. O’Donovan, P., Libeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory Font Selection Using Crowd-sourced Attributes. ACM Trans. Graph. 33, 4 (July), 92:1–92:9. Google ScholarDigital Library
37. Osada, R., Funkhouser, T., Chazelle, B., and Dobkin, D. 2001. Matching 3D Models with Shape Distributions. Shape Modeling International, 154–166. Google ScholarDigital Library
38. Parikh, D., and Grauman, K. 2011. Relative Attributes. International Conference on Computer Vision (ICCV), 503–510. Google ScholarDigital Library
39. Plaisier, M. A., Tiest, W. M. B., and Kappers, A. M. L. 2009. Salient Features in 3-D Haptic Shape Perception. Attention, Perception, & Psychophysics 71, 2, 421–430.Google ScholarCross Ref
40. Prévost, R., Whiting, E., Lefebvre, S., and Sorkine-Hornung, O. 2013. Make It Stand: Balancing Shapes for 3D Fabrication. ACM Trans. Graph. 32, 4 (July), 81:1–81:10. Google ScholarDigital Library
41. Sahbani, A., and El-Khoury, S. 2009. A Hybrid Approach for Grasping 3D Objects. IROS, 1272–1277. Google ScholarDigital Library
42. Sahbani, A., El-khoury, S., and Bidaud, P. 2012. An Overview of 3D Object Grasp Synthesis Algorithms. Robotics and Autonomous Systems 60, 3, 326–336. Google ScholarDigital Library
43. Saxena, A., Driemeyer, J., Kearns, J., and Ng, A. Y. 2007. Robotic Grasping of Novel Objects. NIPS, 1209–1216.Google Scholar
44. Schwartzburg, Y., and Pauly, M. 2013. Fabrication-aware Design with Intersecting Planar Pieces. CGF 32, 2, 317–326.Google ScholarCross Ref
45. Shilane, P., and Funkhouser, T. 2007. Distinctive Regions of 3D Surfaces. ACM Trans. Graph. 26, 2 (June), 7. Google ScholarDigital Library
46. Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. SMI, 167–178. Google ScholarDigital Library
47. Shtrom, E., Leifman, G., and Tal, A. 2013. Saliency Detection in Large Point Sets. ICCV, 3591–3598. Google ScholarDigital Library
48. Song, R., Liu, Y., Martin, R. R., and Rosin, P. L. 2014. Mesh Saliency via Spectral Processing. ACM Trans. Graph. 33, 1 (Feb.), 6:1–6:17. Google ScholarDigital Library
49. Su, H., Maji, S., Kalogerakis, E., and Learned-Miller, E. 2015. Multi-view Convolutional Neural Networks for 3D Shape Recognition. ICCV. Google ScholarDigital Library
50. Surazhsky, T., Magid, E., Soldea, O., Elber, G., and Rivlin, E. 2003. A Comparison of Gaussian and Mean Curvatures Estimation Methods on Triangular Meshes. International Conference on Robotics and Automation, 1021–1026.Google Scholar
51. Tao, P., Cao, J., Li, S., Liu, X., and Liu, L. 2015. Mesh Saliency via Ranking Unsalient Patches in a Descriptor Space. Computers and Graphics 46, C, 264–274. Google ScholarDigital Library
52. Varadarajan, K., Potapova, E., and Vincze, M. 2012. Attention driven Grasping for Clearing a Heap of Objects. IROS, 2035–2042.Google Scholar
53. Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., and Wu, Y. 2014. Learning Fine-Grained Image Similarity with Deep Ranking. CVPR, 1386–1393. Google ScholarDigital Library
54. Watanabe, K., and Belyaev, A. G. 2001. Detection of Salient Curvature Features on Polygonal Surfaces. CGF 20, 3, 385–392.Google ScholarCross Ref
55. Wei, L., Huang, Q., Ceylan, D., Vouga, E., and Li, H. 2015. Dense Human Body Correspondences Using Convolutional Networks. CVPR.Google Scholar
56. Xu, M., Ni, B., Dong, J., Huang, Z., Wang, M., and Yan, S. 2012. Touch Saliency. ACM International Conference on Multimedia, 1041–1044. Google ScholarDigital Library
57. Zagoruyko, S., and Komodakis, N. 2015. Learning to Compare Image Patches via Convolutional Neural Networks. CVPR.Google Scholar
58. Zhou, Q., Panetta, J., and Zorin, D. 2013. Worst-case Structural Analysis. ACM Trans. Graph. 32, 4 (July), 137:1–137:12. Google ScholarDigital Library
59. Zimmer, H., Lafarge, F., Alliez, P., and Kobbelt, L. 2014. Zometool Shape Approximation. Graphical Models 76, 5, 390–401. Google ScholarDigital Library
