“Gesture3D: posing 3D characters via gesture drawings” by Bessmeltsev, Vining and Sheffer
Conference:
Type(s):
Title:
- Gesture3D: posing 3D characters via gesture drawings
Session/Category Title: Human Motion
Presenter(s)/Author(s):
Abstract:
Artists routinely use gesture drawings to communicate ideated character poses for storyboarding and other digital media. During subsequent posing of the 3D character models, they use these drawing as a reference, and perform the posing itself using 3D interfaces which require time and expert 3D knowledge to operate. We propose the first method for automatically posing 3D characters directly using gesture drawings as an input, sidestepping the manual 3D posing step. We observe that artists are skilled at quickly and effectively conveying poses using such drawings, and design them to facilitate a single perceptually consistent pose interpretation by viewers. Our algorithm leverages perceptual cues to parse the drawings and recover the artist-intended poses. It takes as input a vector-format rough gesture drawing and a rigged 3D character model, and plausibly poses the character to conform to the depicted pose. No other input is required. Our contribution is two-fold: we first analyze and formulate the pose cues encoded in gesture drawings; we then employ these cues to compute a plausible image space projection of the conveyed pose and to imbue it with depth. Our framework is designed to robustly overcome errors and inaccuracies frequent in typical gesture drawings. We exhibit a wide variety of character models posed by our method created from gesture drawings of complex poses, including poses with occlusions and foreshortening. We validate our approach via result comparisons to artist-posed models generated from the same reference drawings, via studies that confirm that our results agree with viewer perception, and via comparison to algorithmic alternatives.
References:
1. Attneave, F., and Frost, R. 1969. The determination of perceived tridimensional orientation by minimum criteria. Perception & Psychophysics 6, 6, 391–396. Cross Ref
2. Bai, Y., Kaufman, M. D., Liu, C. K., and Popovic, J. 2016. Artistic-dynamics for 2d animation. ACM Trans. Graph. 35, 4.
3. Bessmeltsev, M., Chang, W., Vining, N., Sheffer, A., and Singh, K. 2015. Modeling character canvases from cartoon drawings. Transactions on Graphics 34, 5.
4. Blair, P. 1994. Cartoon Animation. Walter Foster Publishing.
5. Bregler, C., Malik, J., and Pullen, K. 2004. Twist based acquisition and tracking of animal and human kinematics. Int. J. Comput. Vision 56, 3, 179–194.
6. Buchanan, P., Mukundan, R., and Doggett, M. 2013. Automatic single-view character model reconstruction. In Proc. Symp. Sketch-Based Interfaces and Modeling, 5–14.
7. Chen, Y., Kim, T., and Cipolla, R. 2011. Silhouette-based object phenotype recognition using 3d shape priors. In IEEE International Conference on Computer Vision, ICCV, 25–32.
8. Cherlin, J. J., Samavati, F., Sousa, M. C., and Jorge, J. a. 2005. Sketch-based modeling with few strokes. Proc. 21st spring conference on Computer graphics 1, 212, 137.
9. Choi, M. G., Yang, K., Igarashi, T., Mitani, J., and Lee, J. 2012. Retrieval and visualization of human motion data via stick figures. Computer Graphics Forum 31, 2057–2065.
10. Cordier, F., Seo, H., Park, J., and Noh, J. Y. 2011. Sketching of mirror-symmetric shapes. IEEE Trans. Visualization and Computer Graphics 17, 11.
11. Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A Sketching Interface for Articulated Figure Animation. Proc. Symposium on Computer Animation, 320–328.
12. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Transactions on Graphics 27, 1.
13. Eissen, K., and Steur, R. 2011. Sketching: The Basics. Bis Publishers.
14. Favreau, J.-D., Lafarge, F., and Bousseau, A. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Trans. Graph (Proc. SIGGRAPH 2016).
15. Gall, J., Stoll, C., De Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H. P. 2009. Motion capture using joint skeleton tracking and surface estimation. IEEE Computer Vision and Pattern Recognition Workshops, 1746–1753.
16. Gall, J., Rosenhahn, B., Brox, T., and Seidel, H. P. 2010. Optimization and filtering for human motion capture : AAA multi-layer framework. International Journal of Computer Vision 87, 1-2, 75–92.
17. Gingold, Y., Igarashi, T., and Zorin, D. 2009. Structured annotations for 2D-to-3D modeling. ACM Trans. Graph. 28, 5.
18. Grochow, K., Martin, S. L., Hertzmann, A., and Popović, Z. 2004. Style-based inverse kinematics. ACM Trans. Graph. 23, 3 (Aug.), 522–531.
19. Guay, M., Cani, M.-p., and Ronfard, R. 2013. The Line of Action : an Intuitive Interface for Expressive Character Posing. ACM Trans. on Graphics, 6, 8.
20. Hahn, F., Mutzel, F., Coros, S., Thomaszewski, B., Nitti, M., Gross, M., and Sumner, R. W. 2015. Sketch abstractions for character posing. In Proc. Symp. Computer Animation, 185–191.
21. Hale, R., and Coyle, T. 1991. Master Class in Figure Drawing. Watson-Guptill.
22. Hampton, M. 2009. Figure Drawing: Design and Invention. Figuredrawing.info.
23. Hecker, R., and Perlin, K. 1992. Controlling 3d objects by sketching 2d views. Proc. SPIE 1828, 46–48.
24. Hess, R., and Field, D. 1999. Integration of contours: new insights. Trends in Cognitive Sciences 3, 12 (Dec.), 480–486. Cross Ref
25. Hogarth, B. 1996. Dynamic Figure Drawing. Watson-Guptill.
26. Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. 2014. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans. Pattern Analysis & Machine Intelligence 36, 7, 1325–1339.
27. Jain, E., Sheikh, Y., Mahler, M., and Hodgins, J. 2012. Three-dimensional proxies for hand-drawn characters. ACM Trans. on Graphics 31, 1, 1–16.
28. Karpenko, O. A., and Hughes, J. F. 2006. SmoothSketch: 3D free-form shapes from complex sketches. ACM Trans. on Graphics 1, 212, 589–598.
29. Koffka, K. 1955. Principles of Gestalt Psychology. Routledge & K. Paul.
30. Koller, D., and Friedman, N. 2009. Probabilistic Graphical Models: Principles and Techniques. MIT Press.
31. Kraevoy, V., Sheffer, A., and van de Panne, M. 2009. Modeling from contour drawings. In Proc. Symposium on Sketch-Based Interfaces and Modeling, 37–44.
32. Leland, N. 2006. The New Creative Artist. F+W Media.
33. Lin, J., Igarashi, T., Mitani, J., and Saul, G. 2010. A sketching interface for sitting-pose design. In Proc. Sketch-Based Interfaces and Modeling Symposium, 111–118.
34. Mao, C., Qin, S. F., and Wright, D. K. 2005. A sketch-based gesture interface for rough 3D stick figure animation. Proc. Sketch Based Interfaces and Modeling.
35. Nicolades, K. 1975. The Natural Way to Draw. Houghton Mifflin.
36. Sapp, B., Toshev, A., and Taskar, B. 2010. Cascaded models for articulated pose estimation. Lecture Notes in Computer Science 6312, 406–420.
37. Schmidt, R., Khan, A., Kurtenbach, G., and Singh, K. 2009. On expert performance in 3D curve-drawing tasks. Proc. Symposium on Sketch-Based Interfaces and Modeling 1, 133.
38. Shimony, S. E. 1994. Finding maps for belief networks is np-hard. Artificial Intelligence 68, 2, 399 — 410.
39. Shtof, A., Agathos, A., Gingold, Y., Shamir, A., and Cohen-Or, D. 2013. Geosemantic snapping for sketch-based modeling. Computer Graphics Forum 32, 2, 245–253. Cross Ref
40. Tekin, B., Sun, X., Wang, Lepetit, V., and Fua, P. 2015. Predicting people’s 3d poses from short sequences. In arXiv preprint arXiv:1504.08200.
41. Thiery, J.-M., Guy, E., and Boubekeur, T. 2013. Spheremeshes: Shape approximation using spherical quadric error metrics. ACM Transaction on Graphics 32, 6.
42. Wei, X., and Chai, J. 2011. Intuitive interactive human-character posing with millions of example poses. IEEE Comput. Graph. Appl. 31, 4, 78–88.
43. Xu, B., Chang, W., Sheffer, A., Bousseau, A., McCrae, J., and Singh, K. 2014. True2form: 3d curve networks from 2d sketches via selective regularization. ACM Transactions on Graphics 33, 4.
44. Zhao, J., and Badler, N. I. 1994. Inverse kinematics positioning using nonlinear programming for highly articulated figures. ACM Transactions on Graphics 13, 4, 313–336.
