“Physical face cloning” by Bickel, Kaufmann, Skouras, Thomaszewski, Bradley, et al. …

  • ©Bernd Bickel, Peter Kaufmann, Melina Skouras, Bernhard Thomaszewski, Derek Bradley, Thabo Beeler, Philip (Phil) Jackson, Steve Marschner, Wojciech Matusik, and Markus Gross


    We propose a complete process for designing, simulating, and fabricating synthetic skin for an animatronics character that mimics the face of a given subject and its expressions. The process starts with measuring the elastic properties of a material used to manufacture synthetic soft tissue. Given these measurements we use physics-based simulation to predict the behavior of a face when it is driven by the underlying robotic actuation. Next, we capture 3D facial expressions for a given target subject. As the key component of our process, we present a novel optimization scheme that determines the shape of the synthetic skin as well as the actuation parameters that provide the best match to the target expressions. We demonstrate this computational skin design by physically cloning a real human face onto an animatronics figure.


    1. Becker, M., and Teschner, M. 2007. Robust and efficient estimation of elasticity parameters using the linear finite element method. In SimVis, 15–28.Google Scholar
    2. Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29, 4 (July), 40:1–40:9. Google ScholarDigital Library
    3. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30, 4 (August), 75:1–75:10. Google ScholarDigital Library
    4. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July), 33:1–33:10. Google ScholarDigital Library
    5. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1–89:9. Google ScholarDigital Library
    6. Bickel, B., Bächer, M., Otaduy, M. A., Lee, H. R., Pfister, H., Gross, M., and Matusik, W. 2010. Design and fabrication of materials with desired deformation behavior. ACM Trans. Graph. 29, 4 (July), 63:1–63:10. Google ScholarDigital Library
    7. Bonet, J., and Wood, R. D. 1997. Nonlinear Continuum Mechanics for Finite Element Analysis. Cambridge Uni. Press.Google Scholar
    8. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.Google Scholar
    9. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 4 (July), 41:1–41:10. Google ScholarDigital Library
    10. Bucur, D., and Buttazzo, G. 2005. Variational Methods in Shape Optimization Problems. Birkhuser Mathematics.Google Scholar
    11. Chentanez, N., Alterovitz, R., Ritchie, D., Cho, L., Hauser, K. K., Goldberg, K., Shewchuk, J. R., and O’Brien, J. F. 2009. Interactive simulation of surgical needle insertion and steering. ACM Trans. Graph. 28, 3 (July), 88:1–88:10. Google ScholarDigital Library
    12. Clarberg, P., Jarosz, W., Akenine-Möller, T., and Jensen, H. W. 2005. Wavelet importance sampling: efficiently evaluating products of complex functions. ACM Trans. Graph. 24, 3 (Aug.), 1166–1175. Google ScholarDigital Library
    13. Dong, Y., Wang, J., Pellacini, F., Tong, X., and Guo, B. 2010. Fabricating spatially-varying subsurface scattering. ACM Trans. Graph. 29 (July), 62:1–62:10. Google ScholarDigital Library
    14. Gourret, J.-P., Thalmann, N. M., and Thalmann, D. 1989. Simulation of object and human skin deformations in a grasping task. In Comp. Graph. (Proc. SIGGRAPH), 21–30. Google ScholarDigital Library
    15. Hara, F., Akazawa, H., and Kobayashi, H. 2001. Realistic facial expressions by sma driven face robot. In Proc. of IEEE International Workshop on Robot and Human Interactive Communication, 504–511.Google Scholar
    16. Hašan, M., Fuchs, M., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2010. Physical reproduction of materials with specified subsurface scattering. ACM Trans. Graph. 29 (July), 61:1–61:10. Google ScholarDigital Library
    17. Irving, G., Teran, J., and Fedkiw, R. 2004. Invertible finite elements for robust simulation of large deformation. In 2004 ACM SIGGRAPH/Eurographics SCA, 131–140. Google ScholarDigital Library
    18. Kauer, M., Vuskovic, V., Dual, J., Szekely, G., and Bajka, M. 2002. Inverse finite element characterization of soft tissues. Medical Image Analysis 6, 3, 257–287.Google ScholarCross Ref
    19. Kharevych, L., Mullen, P., Owhadi, H., and Desbrun, M. 2009. Numerical coarsening of inhomogeneous elastic materials. ACM Trans. Graph. 28 (July), 51:1–51:8. Google ScholarDigital Library
    20. Koch, R. M., Gross, M. H., Carls, F. R., von Büren, D. F., Fankhauser, G., and Parish, Y. I. H. 1996. Simulating facial surgery using finite element models. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH ’96, 421–428. Google ScholarDigital Library
    21. Lee, S.-H., and Terzopoulos, D. 2006. Heads up!: biomechanical modeling and neuromuscular control of the neck. In ACM SIGGRAPH 2006 Papers, ACM, 1188–1198. Google ScholarDigital Library
    22. Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28 (September), 99:1–99:17. Google ScholarDigital Library
    23. Levin, D. 1998. The approximation power of moving least-squares. Math. Comput. 67 (October), 1517–1531. Google ScholarDigital Library
    24. Li, X.-Y., Shen, C.-H., Huang, S.-S., Ju, T., and Hu, S.-M. 2010. Popup: automatic paper architectures from 3d models. ACM Trans. Graph. 29 (July), 111:1–111:9. Google ScholarDigital Library
    25. Mori, Y., and Igarashi, T. 2007. Plushie: an interactive design system for plush toys. In ACM SIGGRAPH 2007 papers, ACM, SIGGRAPH ’07. Google ScholarDigital Library
    26. Nesme, M., Kry, P. G., Jeřábková, L., and Faure, F. 2009. Preserving topology and elasticity for embedded deformable models. In ACM SIGGRAPH 2009 papers, ACM, 52:1–52:9. Google ScholarDigital Library
    27. Nishio, S., Ishiguro, H., and Hagita, N. 2007. Humanoid Robots: New Developments. I-Tech, ch. Geminoid: Teleoperated Android of an Existing Person.Google Scholar
    28. Nocedal, J., and Wright, S. J. 2000. Numerical Optimization. Springer, August.Google Scholar
    29. Oh, J.-H., Hanson, D., Kim, W.-S., Han, I. Y., Kim, J.-Y., and Park, I.-W. 2006. Design of android type humanoid robot albert hubo. In Proc. of IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 1428–1433.Google Scholar
    30. Pai, D. K., Doel, K. v. d., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., and Yau, S. H. 2001. Scanning physical interaction behavior of 3d objects. In Proc. of Comp. graph. and int. tech., ACM, SIGGRAPH ’01, 87–96. Google ScholarDigital Library
    31. Schenk, O., and Gärtner, K. 2006. On fast factorization pivoting methods for symmetric indefinite systems. In Elec. Trans. Numer., no. 23, 158–179.Google Scholar
    32. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. In ACM SIGGRAPH 2005, 417–425. Google ScholarDigital Library
    33. Sueda, S., Kaufman, A., and Pai, D. K. 2008. Musculotendon simulation for hand animation. ACM Trans. Graph. (Proc. SIGGRAPH) 27, 3. Google ScholarDigital Library
    34. Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. on Vis. and Comp. Graph. 11, 3, 317–328. Google ScholarDigital Library
    35. Terzopoulos, D., and Waters, K. 1993. Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Mach. Intell. 15 (June), 569–579. Google ScholarDigital Library
    36. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Proc. Comp. graph. and int. tech., ACM, SIGGRAPH ’87, 205–214. Google ScholarDigital Library
    37. Thoutireddy, P., and Ortiz, M. 2004. A variational r-adaption and shape-optimization method for finite-deformation elasticity. Int. J. Numer. Meth. Engng. 61, 1–21.Google ScholarCross Ref
    38. Van Gelder, A. 1998. Approximate simulation of elastic membranes by triangulated spring meshes. J. Graph. Tools 3, 2, 21–42. Google ScholarDigital Library
    39. Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., and Huang, P. 2004. High resolution acquisition, learning and transfer of dyanmic 3-d facial expressions. In Comp. Graph. Forum, 677–686.Google Scholar
    40. Weyrich, T., Peers, P., Matusik, W., and Rusinkiewicz, S. 2009. Fabricating microgeometry for custom surface reflectance. ACM Trans. Graph. 28 (July), 32:1–32:6. Google ScholarDigital Library
    41. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3 (Aug.), 548–558. Google ScholarDigital Library

ACM Digital Library Publication: