“HelpingHand: example-based stroke stylization” by Lu, Yu, Finkelstein and DiVerdi

Digital painters commonly use a tablet and stylus to drive software like Adobe Photoshop. A high quality stylus with 6 degrees of freedom (DOFs: 2D position, pressure, 2D tilt, and 1D rotation) coupled to a virtual brush simulation engine allows skilled users to produce expressive strokes in their own style. However, such devices are difficult for novices to control, and many people draw with less expensive (lower DOF) input devices. This paper presents a data-driven approach for synthesizing the 6D hand gesture data for users of low-quality input devices. Offline, we collect a library of strokes with 6D data created by trained artists. Online, given a query stroke as a series of 2D positions, we synthesize the 4D hand pose data at each sample based on samples from the library that locally match the query. This framework optionally can also modify the stroke trajectory to match characteristic shapes in the style of the library. Our algorithm outputs a 6D trajectory that can be fed into any virtual brush stroke engine to make expressive strokes for novices or users of limited hardware.

1. Ashikhmin, M. 2001. Synthesizing natural textures. In Symposium on Interactive 3D Graphics, 217–226. Google ScholarDigital Library
2. Barnes, C., Goldman, D., Shechtman, E., and Finkelstein, A. 2011. The PatchMatch randomized matching algorithm for image manipulation. Commun. ACM 54 (Nov.), 103–110. Google ScholarDigital Library
3. Baxter, W., and Lin, M. 2004. A versatile interactive 3D brush model. In Pacific Graphics, 319–328. Google ScholarDigital Library
4. Belongie, S., Malik, J., and Puzicha, J. 2001. Shape matching and object recognition using shape contexts. Trans. on Patt. Anal. and Mach. Int. 24, 509–522. Google ScholarDigital Library
5. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? In SIGGRAPH, vol. 28. Google ScholarDigital Library
6. DiVerdi, S., Krishnaswamy, A., and Hadap, S. 2010. Industrial-strength painting with a virtual bristle brush. In Virtual Reality Software and Technology, 119–126. Google ScholarDigital Library
7. Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision, 1033–1038. Google ScholarDigital Library
8. Franke, K., Schomaker, L., and Köppen, M. 2005. Pen force emulating robotic writing device and its application. In Workshop on Advanced Robotics and its Social Impacts, 36–46.Google Scholar
9. Goodwin, T., Vollick, I., and Hertzmann, A. 2007. Isophote distance: a shading approach to artistic stroke thickness. In Non-Photorealistic Animation and Rendering, 53–62. Google ScholarDigital Library
10. Hertzmann, A., Oliver, N., Curless, B., and Seitz, S. 2002. Curve analogies. In Eurographics Workshop on Rendering, 233–246. Google ScholarDigital Library
11. Hinton, G., and Nair, V. 2005. Inferring motor programs from images of handwritten digits. In Advances in Neural Information Processing, 515–522.Google Scholar
12. House, D., and Singh, M. 2007. Line drawing as a dynamic process. In Pacific Graphics, 351–360. Google ScholarDigital Library
13. Hsu, E., Gentry, S., and Popović, J. 2004. Example-based control of human motion. In Symposium on Computer Animation, 69–77. Google ScholarDigital Library
14. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. In SIGGRAPH, 473–482. Google ScholarDigital Library
15. McCrae, J., and Singh, K. 2009. Sketching piecewise clothoid curves. In Sketch-Based Interfaces and Modeling. Google ScholarDigital Library
16. McCrae, J., and Singh, K. 2011. Neatening sketched strokes using piecewise french curves. In Sketch-Based Interfaces and Modeling, 141–148. Google ScholarDigital Library
17. Muja, M., and Lowe, D. G. 2009. Fast approximate nearest neighbors with automatic algorithm configuration. In Internat. Conf. on Comp. Vision Theory and Application, 331–340.Google Scholar
18. Okabe, Y., Saito, S., and Nakajima, M. 2005. Paintbrush rendering of lines using HMMs. In GRAPHITE, 91–98. Google ScholarDigital Library
19. Plamondon, R. 1995. A kinematic theory of rapid human movements. Biological Cybernetics 72, 295–320.Google ScholarDigital Library
20. Saito, S., Kani, A., Chang, Y., and Nakajima, M. 2007. Curvature-based stroke rendering. Visual Computing 24 (November), 1–11. Google ScholarDigital Library
21. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH, 489–498. Google ScholarDigital Library
22. Silver, N., and Dunlap, W. P. 1987. Averaging correlation coefficients: Should fisher’s z transformation be used? Journal of App. Psych. 72(1), 146–148.Google ScholarCross Ref
23. Thiel, Y., Singh, K., and Balakrishnan, R. 2011. Elasti-curves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves. In User Interface Software and Technology, 383–392. Google ScholarDigital Library
24. Vandoren, P., Van Laerhoven, T., Claesen, L., Taelman, J., Raymaekers, C., and Van Reeth, F. 2008. IntuPaint: Bridging the gap between physical and digital painting. In Horizontal Interactive Human Computer Systems, 65–72.Google Scholar
25. Varga, T., Kilchhofer, D., and Bunke, H. 2005. Template-based synthetic handwriting generation for the training of recognition systems. In Conference of the International Graphonomics Society, 206–211.Google Scholar
26. Wei, L.-Y., Lefebvre, S., Kwatra, V., and Turk, G. 2009. State of the art in example-based texture synthesis. In Eurographics 2009, State of the Art Report.Google Scholar
27. Yu, K., Wang, Y., and Tan, T. 2004. Writer identification using dynamic features. In Biometric Authentication, D. Zhang and A. Jain, Eds., vol. 3072 of Lec. Notes in Comp. Sci. 1–8.Google Scholar