“Synthesis of detailed hand manipulations using contact sampling” by Ye and Liu

Capturing human activities that involve both gross full-body motion and detailed hand manipulation of objects is challenging for standard motion capture systems. We introduce a new method for creating natural scenes with such human activities. The input to our method includes motions of the full-body and the objects acquired simultaneously by a standard motion capture system. Our method then automatically synthesizes detailed and physically plausible hand manipulation that can seamlessly integrate with the input motions. Instead of producing one “optimal” solution, our method presents a set of motions that exploit a wide variety of manipulation strategies. We propose a randomized sampling algorithm to search for as many as possible visually diverse solutions within the computational time budget. Our results highlight complex strategies human hands employ effortlessly and unconsciously, such as static, sliding, rolling, as well as finger gaits with discrete relocation of contact points.

1. Albrecht, I., Haber, J., and Seidel, H.-P. 2003. Construction and animation of anatomically based human hand models. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 98–109. Google ScholarDigital Library
2. Andersen, E. D., Jensen, B., Jensen, J., and Sandvik, R. 2009. Mosek version 6. Tech. Rep. TR-2009-3, Ulf Worsøe, October.Google Scholar
3. Aydin, Y., and Nakajima, M. 1999. Database guided computer animation of human grasping using forward and inverse kinematics. Computers and Graphics 23, 1, 145–154.Google ScholarCross Ref
4. Cai, C., and Roth, B. 1987. On the spatial motion of a rigid body with point contact. In Proc. IEEE Int. Con. Robotics and Automation, 686–695.Google Scholar
5. Chenney, S., and Forsyth, D. A. 2000. Sampling plausible solutions to multi-body constraint problems. In SISGGRAPH, 219–228. Google ScholarDigital Library
6. Cherif, M., and Gupta, K. K. 1999. Planning quasi-static fingertip manipulations for reconfiguring objects. IEEE Trans. on Robotics and Automation 15, 5, 837–848.Google ScholarCross Ref
7. Choi, M. G., Lee, J., and Shin, S. Y. 2003. Planning biped locomotion using motion capture data and probabilistic roadmaps. ACM Transactions on Graphics 22, 182–203. Google ScholarDigital Library
8. Ciocarlie, M. T., and Allen, P. K. 2008. On-line interactive dexterous grasping. Eurohaptics (June). Google ScholarDigital Library
9. Ciocarlie, M. T., Lackner, C., and Allen, P. K. 2007. Soft finger model with adaptive contact geometry for grasping and manipulation tasks. IEEE Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (March). Google ScholarDigital Library
10. Cole, A., Hsu, P., and Sastry, S. 1992. Dynamic control of sliding by robot hands for regrasping. IEEE Trans. on Robotics and Automation 8, 1, 42–52.Google ScholarCross Ref
11. Coumans, E., 2005. Bullet physics engine. http://bulletphysics.org.Google Scholar
12. Cyberglove Systems, http://www.cyberglovesystems.com/.Google Scholar
13. Deshpande, A., Ko, J., Fox, D., and Matsuoka, Y. 2009. Anatomically correct testbed hand control: Muscle and joint control strategies. In Robotics and Automation, 2009. ICRA ’09. IEEE International Conference on, 4416–4422. Google ScholarDigital Library
14. ElKoura, G., and Singh, K. 2003. Handrix: Animating the human hand. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 110–119. Google ScholarDigital Library
15. Gill, P., Saunders, M., and Murray, W. 1996. Snopt: An sqp algorithm for large-scale constrained optimization. Tech. Rep. NA 96-2, University of California, San Diego.Google Scholar
16. Gleicher, M. 1998. Retargeting motion to new characters. In SIGGRAPH, 33–42. Google ScholarDigital Library
17. Hamer, H., Gall, J., Urtasun, R., and Gool, L. V. 2011. Data-driven animation of hand-object interactions. In IEEE Conference on Automatic Face and Gesture Recognition, 360–367.Google Scholar
18. Han, L., and Trinkle, J. 1998. Dextrous manipulation by rolling and finger gaiting. In ICRA, IEEE, 730–735.Google Scholar
19. Ho, E. S., Komura, T., and Tai, C.-L. 2010. Spatial relationship preserving character motion adaptation. ACM Trans. Graph 29, 3. Google ScholarDigital Library
20. Hong, J., Lafferriere, G., Mishra, B., and Tang, X. 1990. Fine manipulation with multifinger hand. In ICRA, IEEE, 1568–1573.Google Scholar
21. Huang, Z., Boulic, R., and Thalmann, D. 1995. A multi-sensor approach for grasping and 3-D interaction. In Computer Graphics International ’95. Google ScholarDigital Library
22. Jain, S., and Liu, C. K. 2009. Interactive synthesis of human-object interaction. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 173–182. Google ScholarDigital Library
23. Jain, S., and Liu, C. K. 2011. Controlling physics-based characters using soft contacts. ACM Trans. Graph. (SIGGRAPH Asia) 30 (Dec.), 163:1–163:10. Google ScholarDigital Library
24. Joerg, S., Hodgins, J., and Sullivan, C. 2010. The perception of finger motions. Applied Perception in Graphics and Visualization (APGV). Google ScholarDigital Library
25. Kavraki, L., Svestka, P., Latombe, J.-C., and Overmars, M. H. 1996. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. on Robotics and Automation 12, 4, 566–580.Google ScholarCross Ref
26. Kim, J., Cordier, F., and Magnenat-Thalmann, N. 2000. Neural network-based violinistâs hand animation. In Conference on Computer Graphics International, 37–44. Google ScholarDigital Library
27. Koga, Y., Kondo, K., Kuffner, J., and Latombe, J.-C. 1994. Planning motions with intentions. In SIGGRAPH, 395–408. Google ScholarDigital Library
28. Kry, P. G., and Pai, D. K. 2006. Interaction capture and synthesis. ACM Trans. on Graphics 25, 3 (Aug.), 872–880. Google ScholarDigital Library
29. Lavalle, S. M., and Kuffner, J. J. 2000. Rapidly-exploring random trees: Progress and prospects.Google Scholar
30. Liu, L., Yin, K., van de Panne, M., Shao, T., and Xu, W. 2010. Sampling-based contact-rich motion control. ACM Trans. Graph.(SIGGRAPH) 29, 4, 1–10. Google ScholarDigital Library
31. Liu, C. K. 2009. Dextrous manipulation from a single grasping pose. ACM Transactions on Graphics (SIGGRAPH) 28, 3 (Aug.). Google ScholarDigital Library
32. Majkowska, A., Zordan, V. B., and Faloutsos, P. 2006. Automatic splicing for hand and body animations. In Eurographics/SIGGRAPH Symposium on Computer Animation. Google ScholarDigital Library
33. Miller, A., and Allen, P. 1999. Examples of 3d grasp quality computations. In Robotics and Automation, 1999. Proceedings. 1999 IEEE International Conference on, vol. 2, 1240–1246 vol.2.Google Scholar
34. Ngo, J. T., and Marks, J. 1993. Spacetime constraints revisited. In SIGGRAPH, vol. 27, 343–350. Google ScholarDigital Library
35. Pollard, N. S., and Hodgins, J. K. 2002. Generalizing demonstrated manipualtion tasks. In Workshop on the Algorithmic Foundations of Robotics (WAFR ’02).Google Scholar
36. Pollard, N. S., and Wolf, A. 2004. Grasp synthesis from example: tuning the example to a task or object. In Workshop on Multi-point Interaction in Robotics and Virtual Reality, IEEE International Conference on Robotics and Automation, 77–90.Google Scholar
37. Pollard, N. S., and Zordan, V. B. 2005. Physically based grasping control from example. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 311–318. Google ScholarDigital Library
38. Pollard, N. S. 2004. Closure and quality equivalence for efficient synthesis of grasps from examples. International Journal of Robotics Research 23, 6 (June), 595–614.Google ScholarCross Ref
39. Sims, K. 1994. Evolving virtual creatures. In SIGGRAPH. Google ScholarDigital Library
40. Sok, K. W., Kim, M., and Lee, J. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. (SIGGRAPH) 26, 3, 107. Google ScholarDigital Library
41. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. on Graphics 27, 3 (Aug.). Google ScholarDigital Library
42. Tournassoud, P., Lozano-Perez, T., and Mazer, E. 1987. Regrasping. In Proc. IEEE Int. Con. Robotics and Automation, 1924–1928.Google Scholar
43. Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: An anatomically accurate inverse dynamics solution for unconstrained hand motion. In Eurographics/SIGGRAPH Symposium on Computer Animation, 1–10. Google ScholarDigital Library
44. Twigg, C. D., and James, D. L. 2007. Many-worlds browsing for control of multibody dynamics. ACM Trans. Graph 26, 3. Google ScholarDigital Library
45. van de Panne, M., and Fiume, E. 1993. Sensor-actuator networks. In SIGGRAPH, vol. 27, 335–342. Google ScholarDigital Library
46. Xu, J., Koo, T.-K. J., and Li, Z. 2007. Finger gaits planning for multifingered manipulation. In IROS, IEEE, 2932–2937.Google Scholar
47. Yamane, K., Kuffner, J. J., and Hodgins, J. K. 2004. Synthesizing animations of human manipulation tasks. ACM Trans. Graph 23, 3, 532–539. Google ScholarDigital Library