“Deformation capture and modeling of soft objects”
Conference:
Type:
Title:
- Deformation capture and modeling of soft objects
Session/Category Title: Transfer & Capture
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a data-driven method for deformation capture and modeling of general soft objects. We adopt an iterative framework that consists of one component for physics-based deformation tracking and another for spacetime optimization of deformation parameters. Low cost depth sensors are used for the deformation capture, and we do not require any force-displacement measurements, thus making the data capture a cheap and convenient process. We augment a state-of-the-art probabilistic tracking method to robustly handle noise, occlusions, fast movements and large deformations. The spacetime optimization aims to match the simulated trajectories with the tracked ones. The optimized deformation model is then used to boost the accuracy of the tracking results, which can in turn improve the deformation parameter estimation itself in later iterations. Numerical experiments demonstrate that the tracking and parameter optimization components complement each other nicely.Our spacetime optimization of the deformation model includes not only the material elasticity parameters and dynamic damping coefficients, but also the reference shape which can differ significantly from the static shape for soft objects. The resulting optimization problem is highly nonlinear in high dimensions, and challenging to solve with previous methods. We propose a novel splitting algorithm that alternates between reference shape optimization and deformation parameter estimation, and thus enables tailoring the optimization of each subproblem more efficiently and robustly.Our system enables realistic motion reconstruction as well as synthesis of virtual soft objects in response to user stimulation. Validation experiments show that our method not only is accurate, but also compares favorably to existing techniques. We also showcase the ability of our system with high quality animations generated from optimized deformation parameters for a variety of soft objects, such as live plants and fabricated models.
References:
1. Barbič, J., and Zhao, Y. 2011. Real-time large-deformation substructuring. Proc. of SIGGRAPH 30, 3, 91:1–91:8. Google ScholarDigital Library
2. Barbič, J., Sin, F., and Grinspun, E. 2012. Interactive editing of deformable simulations. ACM Trans. on Graphics 31, 4, 70:1–70:8. Google ScholarDigital Library
3. Barbič, J. 2012. Exact corotational linear FEM stiffness matrix. Tech. rep., Technical report, University of Southern California.Google Scholar
4. Becker, M., and Teschner, M. 2007. Robust and efficient estimation of elasticity parameters using the linear finite element method. In Proc. Simulation und Visualization, 15–28.Google Scholar
5. Bhat, K. S., Twigg, C. D., Hodgins, J. K., Khosla, P. K., Popović, Z., and Seitz, S. M. 2003. Estimating cloth simulation parameters from video. In Proc. ACM SIGGRAPH/Eurographics Symp. on Computer Animation, 37–51. Google ScholarDigital Library
6. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. on Graphics 28, 3, 89:1–89:9. Google ScholarDigital Library
7. Bickel, B., Bächer, M., Otaduy, M. A., and Lee, H. R. 2010. Design and fabrication of materials with desired deformation behavior. ACM Trans. on Graphics 29, 4, 63:1–63:10. Google ScholarDigital Library
8. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM Trans. on Graphics 27, 3, 99:1–99:9. Google ScholarDigital Library
9. Capell, S., Green, S., Curless, B., Duchamp, T., and Popović, Z. 2002. Interactive skeleton-driven dynamic deformations. ACM Trans. on Graphics 21, 3, 586–593. Google ScholarDigital Library
10. Chen, X., Zheng, C., Xu, W., and Zhou, K. 2014. An asymptotic numerical method for inverse elastic shape design. ACM Trans. on Graphics 33, 4, 95:1–95:11. Google ScholarDigital Library
11. Choi, J., and Szymczak, A. 2009. Fitting solid meshes to animated surfaces using linear elasticity. ACM Trans. on Graphics 28, 1, 6:1–6:10. Google ScholarDigital Library
12. Coros, S., Martin, S., Thomaszewski, B., Schumacher, C., Sumner, R., and Gross, M. 2012. Deformable objects alive! ACM Trans. on Graphics 31, 4, 69:1–69:9. Google ScholarDigital Library
13. de Aguiar, E., Stoll, C., Theobalt, C., Amd Hans-Peter Seidel, N. A., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. on Graphics 27, 3, 98:1–98:10. Google ScholarDigital Library
14. Derouet-Jourdan, A., Bertails-Descoubes, F., Daviet, G., and Thollot, J. 2013. Inverse dynamic hair modeling with frictional contact. ACM Trans. on Graphics 32, 6, 159:1–159:10. Google ScholarDigital Library
15. Giles, M. B., and Pierce, N. A. 2000. An introduction to the adjoint approach to design. Flow, Turbulence and Combustion 65, 393–415.Google ScholarCross Ref
16. Helten, T., Mü, M., Seidel, H.-P., and Theobalt, C. 2013. Real-time body tracking with one depth camera and inertial sensors. In Proc. Int. Conf. on Computer Vision, 1105–1112. Google ScholarDigital Library
17. Hildebrandt, K., Schulz, C., Von Tycowicz, C., and Polthier, K. 2012. Interactive spacetime control of deformable objects. ACM Trans. on Graphics 31, 4, 71:1–71:8. Google ScholarDigital Library
18. Huang, J., Tong, Y., Zhou, K., Bao, H., and Desbrun, M. 2011. Interactive shape interpolation through controllable dynamic deformation. IEEE Trans. Visualization & Computer Graphics 17, 7, 983–992. Google ScholarDigital Library
19. James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Trans. on Graphics 24, 3, 399–407. Google ScholarDigital Library
20. Kass, M., Witkin, A., and Terzopoulos, D. 1988. Snakes: Active contour models. Int. J. Computer Vision 1, 4, 321–331.Google ScholarCross Ref
21. Kim, J., and Pollard, N. S. 2011. Fast simulation of skeleton-driven deformable body characters. ACM Trans. on Graphics 30, 5, 121:1–121:19. Google ScholarDigital Library
22. Lee, H.-P., Foskey, M., Niethammer, M., Krajcevski, P., and Lin, M. C. 2012. Simulation-based joint estimation of body deformation and elasticity parameters for medical image analysis. IEEE Trans. on Medical Imaging 31, 11, 2156–2168.Google ScholarCross Ref
23. Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. on Graphics 28, 5, 175:1–175:10. Google ScholarDigital Library
24. Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M., and Rusinkiewicz, S. 2012. Temporally coherent completion of dynamic shapes. ACM Trans. on Graphics 31, 1, 2:1–2:11. Google ScholarDigital Library
25. Li, S., Huang, J., De Goes, F., Jin, X., Bao, H., and Desbrun, M. 2014. Space-time editing of elastic motion through material optimization and reduction. ACM Trans. on Graphics 33, 4, 108:1–108:10. Google ScholarDigital Library
26. McAdams, A., Zhu, Y., Selle, A., Empey, M., Tamstorf, R., Teran, J., and Sifakis, E. 2011. Efficient elasticity for character skinning with contact and collisions. ACM Trans. on Graphics 30, 4, 37:1–37:12. Google ScholarDigital Library
27. McNamara, A., Treuille, A., Popovic, Z., and Stam, J. 2004. Fluid control using the adjoint method. ACM Trans. on Graphics 23, 3, 449–456. Google ScholarDigital Library
28. Miguel, E., Bradley, D., Thomaszewski, B., Bickel, B., Matusik, W., Otaduy, M. A., and Marschner, S. 2012. Data-driven estimation of cloth simulation models. Computer Graphics Forum 31, 2, 519–528. Google ScholarDigital Library
29. Müller, M., and Gross, M. 2004. Interactive virtual materials. In Proc. Int. Conf. on Graphics Interface, 239–246. Google ScholarDigital Library
30. Müller, M., Dorsey, J., McMillan, L., Jagnow, R., and Cutler, B. 2002. Stable real-time deformations. In Proc. ACM SIGGRAPH/Eurographics Symp. on Computer Animation, 49–54. Google ScholarDigital Library
31. Nealen, A., Mller, M., Keiser, R., Boxerman, E., Carlson, M., and Ageia, N. 2006. Physically based deformable models in computer graphics. Computer Graphics Forum 25, 4, 809–836.Google ScholarCross Ref
32. Nocedal, J., and Wright, S. J. 2006. Numerical Optimization. Springer. 2nd Ed.Google Scholar
33. Otaduy, M. A., Bickel, B., Bradley, D., and Wang, H. 2012. Data-driven simulation methods in computer graphics: cloth, tissue and faces. In ACM SIGGRAPH 2012 Courses, SIGGRAPH ’12, 12:1–12:96. Google ScholarDigital Library
34. Pai, D. K., Doel, K. v. d., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau, S. H. 2001. Scanning physical interaction behavior of 3d objects. 87–96. Google ScholarDigital Library
35. Petersen, K. B., and Pedersen, M. S. 2006. The matrix cookbook. Tech. rep., MIT.Google Scholar
36. Schoner, J., Lang, J., and Seidel, H.-P. 2004. Measurement-based interactive simulation of viscoelastic solids. Computer Graphics Forum 23, 3, 547–556.Google ScholarCross Ref
37. Schulman, J., Lee, A., Ho, J., and Abbeel, P. 2013. Tracking deformable objects with point clouds. In Proc. IEEE Int. Conf. on Robotics & Automation.Google Scholar
38. Si, H. 2011. Tetgen, a quality tetrahedral mesh generator and a 3d delaunay triangulator. Tech. rep., WIAS.Google Scholar
39. Sifakis, E., and Barbic, J. 2012. FEM simulation of 3D deformable solids: A practitioner’s guide to theory, discretization and model reduction. In ACM SIGGRAPH Courses, 20:1–20:50. Google ScholarDigital Library
40. Skouras, M., Thomaszewski, B., Bickel, B., and Gross, M. 2012. Computational design of rubber balloons. Computer Graphics Forum 31, 2, 835–844. Google ScholarDigital Library
41. Skouras, M., Thomaszewski, B., Coros, S., Bickel, B., and Gross, M. 2013. Computational design of actuated deformable characters. ACM Trans. on Graphics 32, 4, 82:1–82:10. Google ScholarDigital Library
42. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. Proc. of SIGGRAPH 21, 4, 205–214. Google ScholarDigital Library
43. Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., and Seidel, H.-P. 2012. Animation cartography – intrinsic reconstruction of shape and motion. ACM Trans. on Graphics 31, 2, 12:1–12:15. Google ScholarDigital Library
44. Vlasic, D., Baran, I., Matusik, W., and Popovic, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. on Graphics 27, 3, 97:1–97:9. Google ScholarDigital Library
45. Wang, H., O’Brien, J. F., and Ramamoorthi, R. 2011. Data-driven elastic models for cloth: Modeling and measurement. ACM Trans. on Graphics 30, 4, 71:1–71:12. Google ScholarDigital Library
46. Wei, X., and Chai, J. 2010. Videomocap: Modeling physically realistic human motion from monocular video sequences. ACM Trans. on Graphics 29, 4, 42:1–42:10. Google ScholarDigital Library
47. Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. on Graphics 31, 6, 188:1–188:12. Google ScholarDigital Library
48. Wuhrer, S., Lang, J., Tekieh, M., and Shu, C. 2013. Finite element based tracking of deforming surfaces. Graphical Models 77, 1, 1–17. Google ScholarDigital Library
49. Xu, H., Li, Y., Chen, Y., and Barbic, J. 2015. Interactive material design using model reduction. ACM Trans. on Graphics 34, 2, 18:1–18:14. Google ScholarDigital Library
