“Face Poser: Interactive Modeling of 3D Facial Expressions Using Facial Priors” by Lau, Chai, Xu and Shum

This article presents an intuitive and easy-to-use system for interactively posing 3D facial expressions. The user can model and edit facial expressions by drawing freeform strokes, by specifying distances between facial points, by incrementally editing curves on the face, or by directly dragging facial points in 2D screen space. Designing such an interface for 3D facial modeling and editing is challenging because many unnatural facial expressions might be consistent with the user’s input. We formulate the problem in a maximum a posteriori framework by combining the user’s input with priors embedded in a large set of facial expression data. Maximizing the posteriori allows us to generate an optimal and natural facial expression that achieves the goal specified by the user. We evaluate the performance of our system by conducting a thorough comparison of our method with alternative facial modeling techniques. To demonstrate the usability of our system, we also perform a user study of our system and compare with state-of-the-art facial expression modeling software (Poser 7).

1. Bishop, C. M. and Winn, J. M. 2000. Non-Linear bayesian image modeling. In Proceedings of the European Conference on Computer Vision (ECCV). 3–17. 
2. Bishop, C. 1996. Neural Network for Pattern Recognition. Cambridge University Press. 
3. Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 187–194. 
4. Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. Comput. Graph. Forum. 22, 3, 641–650.
5. Brand, M. 2002. Incremental singular value decomposition of uncertain data with missing values. In Proceedings of the European Conference on Computer Vision (ECCV). 707–720. 
6. Chai, J. and Hodgins, J. 2005. Performance animation from low-dimensional control signals. ACM Trans. Graph. 24, 3, 686–696. 
7. Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-Based control of 3D facial animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 193–206. 
8. Chang, E. and Jenkins, O. C. 2006. Sketching articulation and pose for facial animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 271–280. 
9. Der, K. G., Sumner, R. W., and Popovic, J. 2006. Inverse kinematics for reduced deformable models. ACM Trans. Graph. 25, 3, 1174–1179. 
10. Duda, R. and Hart, P. 1973. Pattern Classification and Scene Analysis. John Wiley and Sons.
11. Feng, W.-W., Kim, B.-U., and Yu, Y. 2008. Real-Time data driven deformation using kernel canonical correlation analysis. ACM Trans. Graph. 27, 3, Article no. 91. 
12. Galassi, M., Davies, J., Theiler, J., Gough, B., Jungman, G., Booth, M., and Rossi, F. 2003. GNU Scientific Library Reference Manual, Revised 2nd Ed. Network Theory Ltd. 
13. Ghahramani, Z. and Hinton, G. E. 1997. The EM algorithm for mixtures of factor analyzers. Tech. rep. CRG-TR-96-1.
14. Grochow, K., Martin, S. L., Hertzmann, A., and Popovic, Z. 2004. Style-Based inverse kinematics. ACM Trans. Graph. 23, 3, 522–531. 
15. Hinton, G. E., Dayan, P., and Revow, M. 1997. Modeling the manifolds of images of handwritten digits. IEEE Trans. Neural Netw. 8, 1, 65–74. 
16. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: A sketching interface for 3D freeform design. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 409–416. 
17. Joshi, P., Tien, W. C., Desbrun, M., and Pighin, F. 2003. Learning controls for blendshape based realistic facial animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Aimation. 187–192. 
18. Kho, Y. and Garland, M. 2005. Sketching mesh deformations. In Proceedings of the ACM Symposium on Interactive 3D Graphics and Games. 147–154. 
19. Lau, M., Chai, J.-X., Xu, Y.-Q., and Shum, H.-Y. 2007. Face poser: Interactive modeling of 3D facial expressions using model priors. In Proceedings of the ACM SIGGRAPH /Eurographics Symposium on Computer Animation. 161–170. 
20. Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 165–172. 
21. Meyer, M. and Anderson, J. 2007. Key point subspace acceleration and soft caching. ACM Trans. Graph. 26, 3, Article no. 74. 
22. Nealen, A., Sorkine, O., Alexa, M., and Cohen-Or, D. 2005. A sketch-based interface for detail-preserving mesh editing. ACM Trans. Graph. 24, 3, 1142–1147. 
23. Parke, F. I. 1972. Computer generated animation of faces. In Proceedings of the ACM National Conference. 1, 451–457. 
24. Pighin, F., Szeliski, R., and Salesin, D. 1999. Resynthesizing facial animation through 3D model-based tracking. In Proceedings of the International Conference on Computer Vision. 143–150.
25. Sloan, P.-P., Rose, C., and Cohen, M. F. 2001. Shape by example. In Proceedings of the ACM Symposium on Interactive 3D Graphics. 135–143. 
26. Sumner, R. W., Zwicker, M., Gotsman, C., and Popovic, J. 2005. Mesh-Based inverse kinematics. ACM Trans. Graph. 24, 3, 488–495. 
27. Vicon Systems. 2007. http://www.vicon.com.
28. Yang, C., Sharon, D., and van de Panne, M. 2005. Sketch- Based modeling of parameterized objects. In Proceedings of the 2nd Eurographics Workshop on Sketch-Based Interfaces and Modeling.
29. Zeleznik, R. C., Herndon, K. P., and Hughes, J. F. 1996. Sketch: An interface for sketching 3D scenes. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 163–170. 
30. Zhang, Q., Liu, Z., Guo, B., and Shum, H. 2003. Geometry-Driven photorealistic facial expression synthesis. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 177–186. 
31. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. 23, 3, 548–558.