“Eyecatch: simulating visuomotor coordination for object interception” by Yeo, Lesmana, Neog and Pai

We present a novel framework for animating human characters performing fast visually guided tasks, such as catching a ball. The main idea is to consider the coordinated dynamics of sensing and movement. Based on experimental evidence about such behaviors, we propose a generative model that constructs interception behavior online, using discrete submovements directed by uncertain visual estimates of target movement. An important aspect of this framework is that eye movements are included as well, and play a central role in coordinating movements of the head, hand, and body. We show that this framework efficiently generates plausible movements and generalizes well to novel scenarios.

1. Abe, Y., and Popović, J. 2006. Interactive animation of dynamic manipulation. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, 195–204. Google ScholarDigital Library
2. Bauml, B., Schmidt, F., Wimbock, T., Birbach, O., Dietrich, A., Fuchs, M., Friedl, W., Frese, U., Borst, C., Grebenstein, M., et al. 2011. Catching flying balls and preparing coffee: Humanoid rollin’justin performs dynamic and sensitive tasks. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 3443–3444.Google ScholarCross Ref
3. Beek, P., and Lewbel, A. 1995. The science of juggling. Scientific American 273, 5, 92–97.Google ScholarCross Ref
4. Berthoz, A. 2000. The brain’s sense of movement. Harvard Univ Pr.Google Scholar
5. Birbach, O., Frese, U., and Bauml, B. 2011. Realtime perception for catching a flying ball with a mobile humanoid. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, IEEE, 5955–5962.Google Scholar
6. Carpenter, R., and Williams, M. 1995. Neural computation of log likelihood in control of saccadic eye movements. Nature 377, 6544, 59–62.Google Scholar
7. Cooper, S., Hertzmann, A., and Popović, Z. 2007. Active learning for real-time motion controllers. In ACM Transactions on Graphics (TOG), vol. 26, ACM, 5. Google ScholarDigital Library
8. Crawford, J., Martinez-Trujillo, J., and Klier, E. 2003. Neural control of three-dimensional eye and head movements. Current opinion in neurobiology 13, 6, 655–662.Google Scholar
9. Dessing, J., Peper, C., Bullock, D., and Beek, P. 2005. How position, velocity, and temporal information combine in the prospective control of catching: Data and model. Journal of cognitive neuroscience 17, 4, 668–686. Google ScholarDigital Library
10. Dipietro, L., Krebs, H., Fasoli, S., Volpe, B., and Hogan, N. 2009. Submovement changes characterize generalization of motor recovery after stroke. Cortex 45, 3, 318–324.Google ScholarCross Ref
11. Flash, T., and Hogan, N. 1985. The coordination of arm movements: an experimentally confirmed mathematical model. The journal of Neuroscience 5, 7, 1688–1703.Google ScholarCross Ref
12. Francik, J., and Szarowicz, A. 2005. Character animation with decoupled behaviour and smart objects. In 6th International Conference on Computer Games CGAIMS, Louisville, Kentucky, USA.Google Scholar
13. Freedman, E., Stanford, T., and Sparks, D. 1996. Combined eye-head gaze shifts produced by electrical stimulation of the superior colliculus in rhesus monkeys. Journal of neurophysiology 76, 2, 927–952.Google ScholarCross Ref
14. Freedman, E. 2001. Interactions between eye and head control signals can account for movement kinematics. Biological cybernetics 84, 6, 453–462.Google Scholar
15. Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., and Sasse, M. 2003. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. in Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, 529–536. Google ScholarDigital Library
16. Gillies, M., and Dodgson, N. 1999. Ball catching: An example of psychologically-based behavioural animation. Eurographics UK.Google Scholar
17. Grillon, H., and Thalmann, D. 2009. Simulating gaze attention behaviors for crowds. Computer Animation and Virtual Worlds 20, 2-3, 111–119. Google ScholarDigital Library
18. Grochow, K., Martin, S. L., Hertzmann, A., and Popović, Z. 2004. Style-based inverse kinematics. In ACM SIGGRAPH 2004 Papers, ACM, New York, NY, USA, SIGGRAPH ’04, 522–531. Google ScholarDigital Library
19. Gu, E., and Badler, N. 2006. Visual attention and eye gaze during multiparty conversations with distractions. In Intelligent Virtual Agents, Springer, 193–204. Google ScholarDigital Library
20. Hayhoe, M., and Ballard, D. 2005. Eye movements in natural behavior. Trends in cognitive sciences 9, 4, 188–194.Google Scholar
21. Hebb, D. 1949. The organization of behavior: A neuropsychological theory. Lawrence Erlbaum.Google Scholar
22. Hove, B., and Slotine, J. 1991. Experiments in robotic catching. In American Control Conference, 1991, IEEE, 380–386.Google Scholar
23. Itti, L. 2003. Realistic avatar eye and head animation using a neurobiological model of visual attention. Tech. rep., DTIC Document.Google Scholar
24. Itti, L. 2006. Quantitative modelling of perceptual salience at human eye position. Visual cognition 14, 4–8, 959–984.Google Scholar
25. Johansson, R., Westling, G., Bäckström, A., and Flanagan, J. 2001. Eye-hand coordination in object manipulation. the Journal of Neuroscience 21, 17, 6917–6932.Google Scholar
26. Lanman, J., Bizzi, E., and Allum, J. 1978. The coordination of eye and head movement during smooth pursuit. Brain Research 153, 1, 39–53.Google ScholarCross Ref
27. Lee, S., and Terzopoulos, D. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM Transactions on Graphics (TOG), vol. 25, ACM, 1188–1198. Google ScholarDigital Library
28. Lee, S., Badler, J., and Badler, N. 2002. Eyes alive. In ACM Transactions on Graphics (TOG), vol. 21, ACM, 637–644. Google ScholarDigital Library
29. Lee, S., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics (TOG) 28, 4, 99. Google ScholarDigital Library
30. Leigh, R., and Zee, D. 1999. The neurology of eye movements. No. 55. Oxford Univ Pr.Google Scholar
31. Liu, C. 2009. Dextrous manipulation from a grasping pose. In ACM Transactions on Graphics (TOG), vol. 28, ACM, 59. Google ScholarDigital Library
32. McIntyre, J., Zago, M., Berthoz, A., Lacquaniti, F., et al. 2001. Does the brain model newton’s laws? Nature Neuroscience 4, 7, 693–694.Google ScholarCross Ref
33. McKee, S. 1981. A local mechanism for differential velocity detection. Vision Research 21, 4, 491–500.Google ScholarCross Ref
34. Meyer, C., Lasker, A., and Robinson, D. 1985. The upper limit of human smooth pursuit velocity. Vision Research 25, 4, 561–563.Google ScholarCross Ref
35. Novak, K., Miller, L., and Houk, J. 2002. The use of overlapping submovements in the control of rapid hand movements. Experimental Brain Research 144, 3, 351–364.Google ScholarCross Ref
36. Orban de Xivry, J., and Lefèvre, P. 2007. Saccades and pursuit: two outcomes of a single sensorimotor process. The Journal of Physiology 584, 1, 11–23.Google ScholarCross Ref
37. Pelachaud, C., and Bilvi, M. 2003. Modelling gaze behavior for conversational agents. In Intelligent Virtual Agents, Springer, 93–100.Google ScholarCross Ref
38. Peters, C., and Qureshi, A. 2010. A head movement propensity model for animating gaze shifts and blinks of virtual characters. Computers & Graphics 34, 6, 677–687. Google ScholarDigital Library
39. Plamondon, R. 1995. A kinematic theory of rapid human movements. Biological Cybernetics 72, 4, 295–307.Google ScholarDigital Library
40. Pollard, N., and Zordan, V. 2005. Physically based grasping control from example. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 311–318. Google ScholarDigital Library
41. Riley, M., and Atkeson, C. 2002. Robot catching: Towards engaging human-humanoid interaction. Autonomous Robots 12, 1, 119–128. Google ScholarDigital Library
42. Robinson, D., Gordon, J., and Gordon, S. 1986. A model of the smooth pursuit eye movement system. Biological Cybernetics 55, 1, 43–57. Google ScholarDigital Library
43. Robinson, D. 1965. The mechanics of human smooth pursuit eye movement. The Journal of Physiology 180, 3, 569.Google ScholarCross Ref
44. Rohrer, B., and Hogan, N. 2003. Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological cybernetics 89, 3, 190–199.Google Scholar
45. Shao, W., and Terzopoulos, D. 2005. Autonomous pedestrians. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 19–28. Google ScholarDigital Library
46. Soechting, J., and Terzuolo, C. 1987. Organization of arm movements. motion is segmented. Neuroscience 23, 1, 39–51.Google ScholarCross Ref
47. Starkes, J., Helsen, W., and Elliott, D. 2002. A menage a trois: the eye, the hand and on-line processing. Journal of sports sciences 20, 3, 217–224.Google ScholarCross Ref
48. Sternad, D., and Schaal, S. 1999. Segmentation of end-point trajectories does not imply segmented control. Experimental Brain Research 124, 1, 118–136.Google ScholarCross Ref
49. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3, 83:1–83:8. Google ScholarDigital Library
50. Terzopoulos, D., and Rabie, T. 1997. Animat vision: Active vision in artificial animals. Videre. Journal of Computer Vision Research 1, 1, 2–19.Google Scholar
51. Tsang, W., Singh, K., and Fiume, E. 2005. Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, ACM, 319–328. Google ScholarDigital Library
52. Tu, X., and Terzopoulos, D. 1994. Artificial fishes: Physics, locomotion, perception, behavior. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, ACM, 43–50. Google ScholarDigital Library
53. Urtasun, R., Fleet, D. J., Geiger, A., Popović, J., Darrell, T. J., and Lawrence, N. D. 2008. Topologically-constrained latent variable models. In Proceedings of the 25th international conference on Machine learning, ACM, New York, NY, USA, ICML ’08, 1080–1087. Google ScholarDigital Library
54. Vallbo, A., and Wessberg, J. 1993. Organization of motor output in slow finger movements in man. The Journal of physiology 469, 1, 673.Google ScholarCross Ref
55. Wang, J. M., Fleet, D. J., and Hertzmann, A. 2007. Multifactor gaussian process models for style-content separation. In Proceedings of the 24th international conference on Machine learning, ACM, New York, NY, USA, ICML ’07, 975–982. Google ScholarDigital Library
56. Wolpert, D., Ghahramani, Z., and Jordan, M. 1995. An internal model for sensorimotor integration. Science 269, 5232, 1880.Google ScholarCross Ref
57. Woodworth, R. 1899. Accuracy of voluntary movement. The Psychological Review: Monograph Supplements 3, 3, i.Google ScholarCross Ref
58. Yamane, K., Kuffner, J., and Hodgins, J. 2004. Synthesizing animations of human manipulation tasks. In ACM Transactions on Graphics (TOG), vol. 23, ACM, 532–539. Google ScholarDigital Library
59. Young, L., and Stark, L. 1963. Variable feedback experiments testing a sampled data model for eye tracking movements. Human Factors in Electronics, IEEE Transactions on, 1, 38–51.Google Scholar
60. Zago, M., McIntyre, J., Senot, P., and Lacquaniti, F. 2009. Visuo-motor coordination and internal models for object interception. Experimental Brain Research 192, 4, 571–604.Google ScholarCross Ref