“High-fidelity Facial Reconstruction From a Single Photo Using Photo-realistic Rendering” by Dias, Roche, Fernandes and Orvalho
Conference:
Type:
Entry Number: 17
Title:
- High-fidelity Facial Reconstruction From a Single Photo Using Photo-realistic Rendering
Presenter(s)/Author(s):
Abstract:
We propose a fully automated method for realistic 3D face reconstruction from a single frontal photo that produces a high-resolution head mesh and a diffuse map. The photo is input to a convolutional neural network that estimates the weights of a morphable model to produce an initial head shape that is further adjusted through landmark-guided deformation. Two key features of the method are: 1) the network is exclusively trained on synthetic photos that are photo-realistic enough to learn real shape predictive features, making it unnecessary to train with real facial photos and corresponding 3D scans; 2) the landmarking statistical errors are incorporated in the reconstruction for optimal accuracy. While the method is based on a limited amount of real data, we show that it robustly and quickly performs plausible face reconstructions from real photos.
References:
V. Blanz and T. Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH’99. ACM Press/Addison-Wesley Publishing Co., USA, 187–194.Google Scholar
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proc. CVPR’09. 248–255.Google Scholar
G. Huang, Z. Liu, L. Van Der Maaten, and K.Q. Weinberger. 2017. Densely connected convolutional networks. In Proc. CVPR’20. 4700–4708.Google Scholar
V. Kazemi and J. Sullivan. 2014. One millisecond face alignment with an ensemble of regression trees. In Proc. CVPR’14. 1867–1874.Google Scholar
R. Li, K. Bladin, Y. Zhao, C. Chinara, O. Ingraham, P. Xiang, X. Ren, P. Prasad, B. Kishore, J. Xing, and H. Li. 2020. Learning formation of physically-based face attributes. In Proc. CVPR’20. 3410–3419.Google Scholar
R. Wang, C.-F. Chen, H. Peng, X. Liu, O. Liu, and X. Li. 2019. Digital Twin: Acquiring High-Fidelity 3D Avatar from a Single Image. Technical Report. arxiv:1912.03455Google Scholar
E. Wood, T. Baltrušaitis, C. Hewitt, S. Dziadzio, T.J. Cashman, and J. Shotton. 2021. Fake It Till You Make It: Face analysis in the wild using synthetic data alone. In Proc. IEEE International Conference on Computer Vision. 3681–3691.Google Scholar
