“Geo-Metric: A Perceptual Dataset of Distortions on Faces” by Wolski, Trutoiu, Dong, Shen, Mackenzie, et al. …
Conference:
Type(s):
Title:
- Geo-Metric: A Perceptual Dataset of Distortions on Faces
Session/Category Title: Faces and Avatars
Presenter(s)/Author(s):
Abstract:
In this work we take a novel perception-centered approach to quantify distortions on 3D geometry of faces, to which humans are particularly sensitive. We generated a dataset, composed of 100 high-quality and demographically-balanced face scans. We then subjected these meshes to distortions that cover relevant use cases in computer graphics, and conducted a large-scale perceptual study to subjectively evaluate them. Our dataset consists of over 84,000 quality comparisons, making it the largest ever psychophysical dataset for geometric distortions. Finally, we demonstrated how our data can be used for applications like metrics, compression, and level-of-detail rendering.
References:
1. Andrew D. Bagdanov, Alberto Del Bimbo, and Iacopo Masi. 2011. The Florence 2D/3D Hybrid Face Dataset. In Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding. 79–80.
2. Volker Blanz and Thomas Vetter. 1999. A Morphable Model for the Synthesis of 3D Faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’99). ACM Press/Addison-Wesley Publishing Co., USA, 187–194.
3. James Booth, Anastasios Roussos, Evangelos Ververas, Epameinondas Antonakos, Stylianos Ploumpis, Yannis Panagakis, and Stefanos Zafeiriou. 2018. 3D Reconstruction of “In-the-Wild” Faces in Images and Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 11 (2018), 2638–2652.
4. Mario Botsch, Mark Pauly, Leif Kobbelt, Pierre Alliez, Bruno Lévy, Stephan Bischoff, and Christian Rossl. 2007. Geometric Modeling Based on Polygonal Meshes. In ACM SIGGRAPH Course Notes.
5. Fergus W. Campbell and John G. Robson. 1968. Application of Fourier Analysis to the Visibility of Gratings. The Journal of physiology 197, 3 (1968), 551–566.
6. Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. 2014. FaceWarehouse: A 3D Facial Expression Database for Visual Computing. IEEE Transactions on Visualization and Computer Graphics 20, 3 (2014), 413–425.
7. P. Cignoni, C. Rocchini, and R. Scopigno. 1998. Metro: Measuring Error on Simplified Surfaces. Computer Graphics Forum 17, 2 (1998), 167–174.
8. Darren Cosker, Eva Krumhuber, and Adrian Hilton. 2011. A FACS Valid 3D Dynamic Action Unit Database with Applications to 3D Dynamic Morphable Facial Modeling. Proceedings of the IEEE International Conference on Computer Vision, 2296–2303.
9. D. Cudeiro, T. Bolkart, C. Laidlaw, A. Ranjan, and M. J. Black. 2019. Capture, Learning, and Synthesis of 3D Speaking Styles. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 10093–10103.
10. Hang Dai, Nick Pears, William Smith, and Christian Duncan. 2019. Statistical Modeling of Craniofacial Shape and Texture. International Journal of Computer Vision 128, 2 (09 Nov 2019), 547–571.
11. Scott J. Daly. 1992. The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity. In Human Vision, Visual Processing, and Digital Display III, Vol. 1666. International Society for Optics and Photonics, 2–15.
12. Herbert Aron David. 1963. The Method of Paired Comparisons. Vol. 12. London.
13. Michael Deering. 1995. Geometry Compression. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’95). Association for Computing Machinery, New York, NY, USA, 13–20.
14. Xiang Feng, Wanggen Wan, Richard Yi Da Xu, Stuart Perry, Pengfei Li, and Song Zhu. 2018. A Novel Spatial Pooling Method for 3D Mesh Quality Assessment Based on Percentile Weighting Strategy. Computers & Graphics 74 (2018), 12–22.
15. James A. Ferwerda, Peter Shirley, Sumanta N. Pattanaik, and Donald P. Greenberg. 1997. A Model of Visual Masking for Computer Graphics. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97). ACM Press/Addison-Wesley Publishing Co., USA, 143–152.
16. Yudong Guo, Juyong Zhang, Jianfei Cai, Boyi Jiang, and Jianmin Zheng. 2018. CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images. IEEE Transactions on Pattern Analysis and Machine Intelligence PP (05 2018), 1–1.
17. Alain Horé and Djemel Ziou. 2010. Image Quality Metrics: PSNR vs. SSIM. In 2010 20th International Conference on Pattern Recognition. 2366–2369.
18. Gordon Kindlmann, Erik Reinhard, and Sarah Creem. 2002. Face-Based Luminance Matching for Perceptual Colormap Generation. In IEEE Visualization, 2002. 299–306.
19. Venkat Krishnamurthy and Marc Levoy. 1996. Fitting Smooth Surfaces to Dense Polygon Meshes. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96). Association for Computing Machinery, New York, NY, USA, 313–324.
20. Guillaume Lavoué. 2011. A Multiscale Metric for 3D Mesh Visual Quality Assessment. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 1427–1437.
21. Guillaume Lavoué, Elisa Drelie Gelasca, Florent Dupont, Atilla Baskurt, and Touradj Ebrahimi. 2006. Perceptually Driven 3D Distance Metrics with Application to Watermarking. In Applications of Digital Image Processing XXIX, Vol. 6312. International Society for Optics and Photonics, 63120L.
22. Guillaume Lavoué, Mohamed Chaker Larabi, and Libor Váša. 2016. On the Efficiency of Image Metrics for Evaluating the Visual Quality of 3D Models. IEEE Transactions on Visualization and Computer Graphics 22, 8 (2016), 1987–1999.
23. Michael B. Lewis. 2001. The Lady’s Not for Turning: Rotation of the Thatcher Illusion. Perception 30, 6 (2001), 769–774.
24. Charles Loop. 1987. Smooth Subdivision Surfaces Based on Triangles. Master’s thesis, University of Utah, Department of Mathematics (1987). https://www.microsoft.com/en-us/research/publication/smooth-subdivision-surfaces-based-on-triangles/
25. David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. 2003. Level of Detail for 3D Graphics. Morgan Kaufmann.
26. Rafał Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Trans. Graph. 30, 4, Article 40 (jul 2011), 14 pages.
27. Rafał K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. 2021. FovVideoVDP: A Visible Difference Predictor for Wide Field-of-View Video. ACM Trans. Graph. 40, 4, Article 49 (jul 2021), 19 pages.
28. Georges Nader, Kai Wang, Franck Hétroy-Wheeler, and Florent Dupont. 2015. Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces. IEEE transactions on visualization and computer graphics 22, 11 (2015), 2423–2436.
29. Anass Nouri, Christophe Charrier, and Olivier Lézoray. 2016. Full-Reference Saliency-Based 3D Mesh Quality Assessment Index. In 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 1007–1011.
30. Aude Oliva, Antonio Torralba, and Philippe G. Schyns. 2006. Hybrid Images. ACM Trans. Graph. 25, 3 (jul 2006), 527–532.
31. James P. O’Shea, Martin S. Banks, and Maneesh Agrawala. 2008. The Assumed Light Direction for Perceiving Shape from Shading. In Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization (Los Angeles, California) (APGV ’08). Association for Computing Machinery, 135–142.
32. Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. 2009. A 3D Face Model for Pose and Illumination Invariant Face Recognition. In IEEE international conference on advanced video and signal based surveillance. 296–301.
33. Maria Perez-Ortiz, Aliaksei Mikhailiuk, Emin Zerman, Vedad Hulusic, Giuseppe Valenzise, and Rafał K Mantiuk. 2019. From Pairwise Comparisons and Rating to a Unified Quality Scale. IEEE Transactions on Image Processing 29 (2019), 1139–1151.
34. Ken Perlin. 2002. Improving Noise. ACM Trans. Graph. 21, 3, 681–682.
35. Nikolay Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al. 2015. Image Database TID2013: Peculiarities, Results and Perspectives. Signal processing: Image communication 30 (2015), 57–77.
36. Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, and Michael J Black. 2018. Generating 3D Faces Using Convolutional Mesh Autoencoders. In Proceedings of the European Conference on Computer Vision. 704–720. http://arxiv.org/abs/1807.10267
37. Bernice E. Rogowitz and Holly E. Rushmeier. 2001. Are Image Quality Metrics Adequate to Evaluate the Quality of Geometric Objects?. In Human Vision and Electronic Imaging VI (Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 4299). 340–348.
38. Jeffrey A. Saunders and Benjamin T. Backus. 2006. Perception of Surface Slant From Oriented Textures. Journal of Vision 6, 9 (08 2006), 3–3.
39. Peter H Schönemann. 1966. A Generalized Solution of the Orthogonal Procrustes Problem. Psychometrika 31, 1 (1966), 1–10.
40. Ana Serrano, Bin Chen, Chao Wang, Michal Piovarči, Hans-Peter Seidel, Piotr Didyk, and Karol Myszkowski. 2021. The Effect of Shape and Illumination on Material Perception: Model and Applications. ACM Trans. Graph. 40, 4, Article 125 (jul 2021), 16 pages.
41. Hamid R. Sheikh, Zhou Wang, Lawrence Cormack, and Alan C Bovik. 2005. LIVE Image Quality Assessment Database Release 2. (2005). http://live.ece.utexas.edu/research/quality
42. Daniel J Simons and Ronald A Rensink. 2005. Change Blindness: Past, Present, and Future. Trends in Cognitive Sciences 9, 1 (2005), 16–20.
43. Olga Sorkine, Daniel Cohen-Or, and Sivan Toledo. 2003. High-Pass Quantization for Mesh Encoding. In Proceedings of the 2003 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (Aachen, Germany) (SGP ’03). Eurographics Association, Goslar, DEU, 42–51.
44. Jennifer Sun and Pietro Perona. 1998. Where Is the Sun? Nature neuroscience 1, 3 (1998), 183–184.
45. Peter Thompson. 1980. Margaret Thatcher: A New Illusion. Perception 9, 4 (1980), 483–484.
46. United States Census Bureau. 2021. US census. https://www.census.gov/quickfacts/fact/table/US/PST045221 [Online; accessed 14-December-2021].
47. Libor Váša and Jan Rus. 2012. Dihedral Angle Mesh Error: A Fast Perception Correlated Distortion Measure for Fixed Connectivity Triangle Meshes. Comput. Graph. Forum 31, 5, 1715–1724.
48. Kai Wang, Fakhri Torkhani, and Annick Montanvert. 2012. A Fast Roughness-Based Approach to the Assessment of 3D Mesh Visual Quality. Computers & Graphics 36, 7 (2012), 808–818.
49. Andrew P. Witkin. 1987. Scale-Space Filtering. In Readings in Computer Vision. Elsevier, 329–332.
50. Nanyang Ye, Krzysztof Wolski, and Rafal K. Mantiuk. 2019. Predicting Visible Image Differences Under Varying Display Brightness and Viewing Distance. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5429–5437.
51. Lijun Yin, Xiaochen Chen, Yi Sun, Tony Worm, and Michael Reale. 2008. A High-Resolution 3D Dynamic Facial Expression Database. In 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition. 1–6.
52. Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, and Matthew J. Rosato. 2006. A 3D Facial Expression Database For Facial Behavior Research. In International Conference on Automatic Face and Gesture Recognition. IEEE, 211–216.
53. Xing Zhang, Lijun Yin, Jeffrey F. Cohn, Shaun Canavan, Michael Reale, Andy Horowitz, and Peng Liu. 2013. A High-Resolution Spontaneous 3D Dynamic Facial Expression Database. In International Conference on Automatic Face and Gesture Recognition. IEEE, 1–6.
54. Wenyi Zhao, Rama Chellappa, P. Jonathon Phillips, and Azriel Rosenfeld. 2003. Face Recognition: A Literature Survey. ACM computing surveys 35, 4 (2003), 399–458.
55. Hao Zhu, Haotian Yang, Longwei Guo, Yidi Zhang, Yanru Wang, Mingkai Huang, Qiu Shen, Ruigang Yang, and Xun Cao. 2021. FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction. arXiv preprint (2021).
