“Body talk: crowdshaping realistic 3D avatars with words”

  • ©Stephan Streuber, M. Alejandra Quiros-Ramirez, Matthew Q Hill, Carina A. Hahn, Silvia Zuffi, Alice O'Toole, and Michael J. Black

Conference:


Type:


Title:

    Body talk: crowdshaping realistic 3D avatars with words

Session/Category Title: PERCEPTION OF SHAPES AND PEOPLE

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Realistic, metrically accurate, 3D human avatars are useful for games, shopping, virtual reality, and health applications. Such avatars are not in wide use because solutions for creating them from high-end scanners, low-cost range cameras, and tailoring measurements all have limitations. Here we propose a simple solution and show that it is surprisingly accurate. We use crowdsourcing to generate attribute ratings of 3D body shapes corresponding to standard linguistic descriptions of 3D shape. We then learn a linear function relating these ratings to 3D human shape parameters. Given an image of a new body, we again turn to the crowd for ratings of the body shape. The collection of linguistic ratings of a photograph provides remarkably strong constraints on the metric 3D shape. We call the process crowdshaping and show that our Body Talk system produces shapes that are perceptually indistinguishable from bodies created from high-resolution scans and that the metric accuracy is sufficient for many tasks. This makes body “scanning” practical without a scanner, opening up new applications including database search, visualization, and extracting avatars from books.

References:


    1. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM transactions on graphics (TOG) 22, 3, 587–594. Google ScholarDigital Library
    2. Allen, B., Curless, B., and Popović, Z. 2004. Exploring the space of human body shapes: Data-driven synthesis under anthropometric control. In SAE International Proc. Digital Human Modeling for Design and Engineering Conference.Google Scholar
    3. Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’06, 147–156. Google ScholarDigital Library
    4. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. (Proc. SIGGRAPH 24, 3, 408–416. Google ScholarDigital Library
    5. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH ’99, 187–194. Google ScholarDigital Library
    6. Bogo, F., Black, M. J., Loper, M., and Romero, J. 2015. Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In Int. Conf. Comp. Vis. (ICCV), 2300–2308. Google ScholarDigital Library
    7. Carter, J., and Heath, B. 1990. Somatotyping: Development and applications. Cambridge University Press, Cambridge.Google Scholar
    8. Chaudhuri, S., Kalogerakis, E., Giguere, S., and Funkhouser, T. 2013. Attribit: content creation with semantic attributes. In Proceedings of the 26th annual ACM symposium on User interface software and technology, ACM, 193–202. Google ScholarDigital Library
    9. Gorber, S. C., Tremblay, M., Moher, D., and Gorber, B. 2007. A comparison of direct vs. self-report measures for assessing height, weight and body mass index: A systematic review. Obesity Reviews 8, 4, 307–326.Google ScholarCross Ref
    10. Gordon, C., Churchill, T., Clauser, C., Bradtmiller, B., and McConville, J. 1989. Anthropometric survey of US Army personnel: Methods and summary statistics 1988. Tech. rep., DTIC Document.Google Scholar
    11. Greenacre, M. 2007. Correspondence analysis in practice (2nd ed.). Chapman and Hall/CRC.Google Scholar
    12. Guy, S. J., Kim, S., Lin, M. C., and Manocha, D. 2011. Simulating heterogeneous crowd behaviors using personality trait theory. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA ’11, 43–52. Google ScholarDigital Library
    13. Hammett, D. 1929. The Maltese Falcon. Alfred A. Knopf.Google Scholar
    14. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337–346.Google ScholarCross Ref
    15. Hill, M. Q., Streuber, S., Hahn, C. A., Black, M. J., and O’Toole, A. J. 2015. Exploring the relationship between body shapes and descriptions by linking similarity spaces. J Vis. 15, 12, 931.Google ScholarCross Ref
    16. Hirshberg, D., Loper, M., Rachlin, E., and Black, M. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In European Conf. on Computer Vision (ECCV), Springer-Verlag, A. F. et al. (Eds.), Ed., LNCS 7577, Part IV, 242–255. Google ScholarDigital Library
    17. Jain, A., Thormählen, T., Seidel, H.-P., and Theobalt, C. 2010. Moviereshape: Tracking and reshaping of humans in videos. ACM Trans. Graph. 29, 6 (Dec.), 148:1–148:10. Google ScholarDigital Library
    18. Klare, B., Klum, S., Klontz, J., Taborsky, E., Akgul, T., and Jain, A. 2014. Suspect identification based on descriptive facial attributes. In Biometrics (IJCB), 2014 IEEE International Joint Conference on, 1–8.Google Scholar
    19. Kleindessner, M., and von Luxburg, U. 2015. Dimensionality estimation without distances. In AISTATS.Google Scholar
    20. Li, H., Vouga, E., Gudym, A., Luo, L., Barron, J. T., and Gusev, G. 2013. 3D self-portraits. ACM Trans. Graph. (Proceedings SIGGRAPH Asia 2013) 32, 6 (November). Google ScholarDigital Library
    21. Little, A. C., Roberts, S. C., Jones, B. C., and Debruine, L. M. 2012. The perception of attractiveness and trustworthiness in male faces affects hypothetical voting decisions differently in wartime and peacetime scenarios. Q J Exp Psychol 65, 10 (May), 2018–2032.Google ScholarCross Ref
    22. Liu, T., Hertzmann, A., Li, W., and Funkhouser, T. 2015. Style compatibility for 3D furniture models. ACM Trans. Graph. (TOG) 34, 4, 85. Google ScholarDigital Library
    23. Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. 2015. SMPL: A skinned multi-person linear model. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 34, 6 (Oct.), 248:1–248:16. Google ScholarDigital Library
    24. Lun, Z., Kalogerakis, E., and Sheffer, A. 2015. Elements of style: Learning perceptual shape style similarity. ACM Trans. Graph. 34, 4. Google ScholarDigital Library
    25. Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Trans. Graph. 22, 3 (July), 759–769. Google ScholarDigital Library
    26. O’Donovan, P., Lībeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory font selection using crowdsourced attributes. ACM Trans. Graph. 33, 4 (July), 92:1–92:9. Google ScholarDigital Library
    27. O’Toole, A., Price, T., Vetter, T., Bartlett, J., and Blanz, V. 1999. 3D shape and 2D surface textures of human faces: The role of averages in attractiveness and age. Image and Vision Computing 18, 1, 9–19.Google ScholarCross Ref
    28. Parikh, D., and Grauman, K. 2011. Relative attributes. In Int. Conf. Comp. Vis. (ICCV), 503–510. Google ScholarDigital Library
    29. Pons-Moll, G., Fleet, D., and Rosenhahn, B. 2014. Posebits for monocular human pose estimation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, 2345–2352. Google ScholarDigital Library
    30. Reid, D., and Nixon, M. 2013. Human identification using facial comparative descriptions. In Biometrics (ICB), 2013 International Conference on, 1–7.Google Scholar
    31. Reid, D. A., Nixon, M. S., and Stevenage, S. V. 2014. Soft biometrics; human identification using comparative descriptions. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) 36 (June), 1216–1228. Google ScholarDigital Library
    32. Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.Google Scholar
    33. Sekunova, A., Black, M., Parkinson, L., and Barton, J. J. S. 2013. Viewpoint and pose in body-form adaptation. Perception 42, 2, 176–186.Google ScholarCross Ref
    34. Seo, H.-W., and Magnenat-Thalmann, N. 2003. An automatic modeling of human bodies from sizing parameters. In In Proceedings of the 2003 Symposium on Interactive 3D Graphics, ACM Press, 1926. Google ScholarDigital Library
    35. Seo, H.-W., Cordier, F., and Mangnenat-Thalmann, N. 2003. Synthesizing body models with parameterized shape modifications. In In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, 120125. Google ScholarDigital Library
    36. Shapiro, A., Feng, A., Wang, R., Li, H., Bolas, M., Medioni, G., and Suma, E. 2014. Rapid avatar capture and simulation using commodity depth sensors. Computer Animation and Virtual Worlds. Google ScholarDigital Library
    37. Sheldon, W. H. 1940. The Varieties of Human Physique (An Introduction to Constitutional Psychology). Harper and Brothers.Google Scholar
    38. Sigal, L., Mahler, M., Diaz, S., McIntosh, K., Carter, E., Richards, T., and Hodgins, J. 2015. A perceptual control space for garment simulation. ACM Trans. Graph. 34, 4 (July), 117:1–117:10. Google ScholarDigital Library
    39. 2015. SMPL model. http://smpl.is.tue.mpg.de/. Accessed: 2015-12-10.Google Scholar
    40. 2016. Somatotypes. http://users.rider.edu/~suler/somato.html. Accessed: 2016-01-14.Google Scholar
    41. Spencer, E. A., Appleby, P. N., Davey, G. K., and Key, T. J. 2002. Validity of self-reported height and weight in 4808 epicoxford participants. Public Health Nutrition 5 (8), 561–565.Google ScholarCross Ref
    42. Sumner, R. W. 2005. Mesh modification using deformation gradients. PhD thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science. Google ScholarDigital Library
    43. Talton, J. O., Gibson, D., Yang, L., Hanrahan, P., and Koltun, V. 2009. Exploratory modeling with collaborative design spaces. ACM Trans. Graph. 28, 5, 167. Google ScholarDigital Library
    44. Treynor, J. L. 1987. Market efficiency and the bean jar experiment. Financial Analysts Journal 43, 3 (May/June), 50–53.Google ScholarCross Ref
    45. Tsoli, A., Loper, M., and Black, M. J. 2014. Model-based anthropometry: Predicting measurements from 3D human scans in multiple poses. In Proceedings Winter Conference on Applications of Computer Vision, IEEE, 83–90.Google Scholar
    46. Weiss, A., Hirshberg, D., and Black, M. 2011. Home 3D body scans from noisy image and range data. In Int. Conf. Comp. Vis. (ICCV), IEEE, Barcelona, 1951–1958. Google ScholarDigital Library
    47. Xia, S., Wang, C., Chai, J., and Hodgins, J. 2015. Realtime style transfer for unlabeled heterogeneous human motion. ACM Trans. Graph. (TOG) 34, 4, 119. Google ScholarDigital Library
    48. Yumer, M. E., Chaudhuri, S., Hodgins, J. K., and Kara, L. B. 2015. Semantic shape editing using deformation handles. ACM Trans. Graph. (Proceedings of SIGGRAPH 2015) 34. Google ScholarDigital Library
    49. Zhou, S., Fu, H., Liu, L., Cohen-Or, D., and Han, X. 2010. Parametric reshaping of human bodies in images. ACM Trans. Graph. 29, 4 (July), 126:1–126:10. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page: