“Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture” by Zhang, Siu, Zhang, Liu and Chai – ACM SIGGRAPH HISTORY ARCHIVES

“Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture” by Zhang, Siu, Zhang, Liu and Chai

  • 2014 SA Technical Papers Zhang_Leveraging Depth Cameras and Wearable Pressure Sensors

Conference:


Type(s):


Title:

    Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture

Session/Category Title:   Character Animation

Presenter(s)/Author(s):


Abstract:


    We present a new method for full-body motion capture that uses input data captured by three depth cameras and a pair of pressure-sensing shoes. Our system is appealing because it is low-cost, non-intrusive and fully automatic, and can accurately reconstruct both full-body kinematics and dynamics data. We first introduce a novel tracking process that automatically reconstructs 3D skeletal poses using input data captured by three Kinect cameras and wearable pressure sensors. We formulate the problem in an optimization framework and incrementally update 3D skeletal poses with observed depth data and pressure data via iterative linear solvers. The system is highly accurate because we integrate depth data from multiple depth cameras, foot pressure data, detailed full-body geometry, and environmental contact constraints into a unified framework. In addition, we develop an efficient physics-based motion reconstruction algorithm for solving internal joint torques and contact forces in the quadratic programming framework. During reconstruction, we leverage Newtonian physics, friction cone constraints, contact pressure information, and 3D kinematic poses obtained from the kinematic tracking process to reconstruct full-body dynamics data. We demonstrate the power of our approach by capturing a wide range of human movements and achieve state-of-the-art accuracy in our comparison against alternative systems.

References:


    1. Adelsberger, R., and Tröster, G. 2013. Pimu: A wireless pressure-sensing imu. In ISSNIP, IEEE Proceedings of the 8th International Conference on Intelligent Sensors, Sensor Networks and Information Processing.
    2. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3 (July), 587–594.
    3. Ascension, 2014. http://www.ascension-tech.com/.
    4. Baak, A., Müller, M., Bharaj, G., Seidel, H.-P., and Theobalt, C. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. In IEEE 13th International Conference on Computer Vision (ICCV), 1092–1099.
    5. Bregler, C., Malik, J., and k Pullen. 2004. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision. 56(3):179–194.
    6. Bridson, R., Marino, S., and Fedkiw, R. 2003. Simulation of clothing with folds and wrinkles. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’03, 28–36.
    7. Brubaker, M. A., and Fleet, D. J. 2008. The Kneed Walker for human pose tracking. In Proceedings of IEEE CVPR.
    8. Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH ’96, 303–312.
    9. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1–98:10.
    10. Elgammal, A., and Lee, C. 2004. Inferring 3D body pose from silhouettes using activity manifold learning. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2: 681–688.
    11. Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24, 6, 381–395.
    12. Fisher, S., and Lin, M. C. 2001. Deformed distance fields for simulation of non-penetrating flexible bodies. In Proceedings of the Eurographic Workshop on Computer Animation and Simulation, Springer-Verlag New York, Inc., New York, NY, USA, 99–111.
    13. Grest, D., Kruger, V., and Koch, R. 2007. Single view motion tracking by depth and silhouette information. In Proceedings of the 15th Scandinavian Conference on Image Analysis (SCIA), 719–729.
    14. Guendelman, E., Bridson, R., and Fedkiw, R. 2003. Non-convex rigid bodies with stacking. ACM Trans. Graph. 22, 3 (July), 871–878.
    15. Ha, S., Bai, Y., and Liu, C. K. 2011. Human motion reconstruction from force sensors. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA ’11, 129–138.
    16. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In Computer Graphics Forum (Proc. Eurographics 2008), P. Dutr’e and M. Stamminger, Eds., vol. 2.
    17. Knoop, S., Vacek, S., and Dillmann, R. 2006. Sensor fusion for 3d human body tracking with an articulated 3d body model. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 1686–1691.
    18. Kovar, L., Schreiner, J., and Gleicher, M. 2002. Footskate cleanup for motion capture editing. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA ’02, 97–104.
    19. Microsoft Kinect API for Windows, 2014. http://www.microsoft.com/en-us/kinectforwindows/.
    20. Moeslund, T. B., Hilton, A., and Krüger, V. 2006. A survey of advances in vision-based human motion capture and analysis. Journal of Computer Vision and Image Understanding. 104:90–126.
    21. Pavlović, V., Rehg, J. M., and MacCormick, J. 2000. Learning Switching Linear Models of Human Motion. In Advances in Neural Information Processing Systems 13, 981–987.
    22. Plagemann, C., Ganapathi, V., Koller, D., and Thrun, S. 2010. Realtime identification and localization of body parts from depth images. In Proceedings of International Conferences on Robotics and Automation (ICRA 2010), 3108–3113.
    23. Rosales, R., and Sclaroff, S. 2000. Specialized mappings and the estimation of human body pose from a single image. In Proceedings of the Workshop on Human Motion. 19–24.
    24. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from a single depth image. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1297–1304.
    25. Tekscan, 2014. http://www.tekscan.com/.
    26. Urtasun, R., Fleet, D. J., Hertzmann, A., and Fua., P. 2005. Priors for people tracking from small training sets. In IEEE International Conference on Computer Vision, 403–C410.
    27. Vicon Systems, 2014. http://www.vicon.com.
    28. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3 (Aug.), 97:1–97:9.
    29. Vondrak, M., Sigal, L., and Jenkins, O. C. 2008. Physical simulation for probabilistic motion tracking. In IEEE Conference on Computer Vision and Pattern Recognition, 1–8.
    30. Vondrak, M., Sigal, L., Hodgins, J., and Jenkins, O. 2012. Video-based 3d motion capture through biped control. ACM Trans. Graph. 31, 4 (July), 27:1–27:12.
    31. Wei, X. K., and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Transactions on Graphics. 29(4): Article No. 42.
    32. Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (Nov.), 188:1–188:12.
    33. Xsens, 2014. http://www.xsens.com.
    34. Ye, M., Wang, X., Yang, R., Ren, L., and Pollefeys, M. 2011. Accurate 3d pose estimation from a single depth image. In Proceedings of IEEE 13th International Conference on Computer Vision, 731–738.
    35. Yin, K., and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’03, 329–338.

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org