“Pinscreen: 3D Avatar From a Single Image”
Conference:
Experience Type(s):
Title:
- Pinscreen: 3D Avatar From a Single Image
Description:
The age of social media and immersive technologies has created a growing need for processing detailed visual representations of ourselves as virtual and augmented reality is growing into the next generation platform for online communication, connecting hundreds of millions of users. A realistic simulation of our presence in a mixed reality environment is unthinkable without a compelling and directable 3D digitization of ourselves. With the wide availability of mobile cameras and internet images, we introduce a technology that can build a realistic 3D avatar from a single photograph. This textured 3D face model includes hair and can be instantly animated by anyone in real-time through natural facial performances captured from a regular RGB camera. Immediate applications include personalized gaming and VR-enabled social networks using automatically digitized 3D avatars, as well as mobile apps such as video messengers (e.g., Snapchat) with face-swapping capabilities. As opposed to existing solutions, our technology enables the automatic generation of a complete head model from a fully unconstrained image.
References:
[1]
Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Transactions on Graphics (Proceedings SIGGRAPH 2015) 34, 4 (July).
[2]
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, ACM, 675–678.
[3]
Saito, S., Li, T., and Li, H. 2016. Real-time facial segmentation and performance capture from rgb input. In Proceedings of the European Conference on Computer Vision (ECCV).
