“Handwriting beautification using token means” by Zitnick

  • ©Lawrence C. Zitnick




    Handwriting beautification using token means

Session/Category Title: Line Drawing




    In this paper, we propose a general purpose approach to handwriting beautification using online input from a stylus. Given a sample of writings, drawings, or sketches from the same user, our method improves a user’s strokes in real-time as they are drawn. Our approach relies on one main insight. The appearance of the average of multiple instances of the same written word or shape is better than most of the individual instances. We utilize this observation using a two-stage approach. First, we propose an efficient real-time method for finding matching sets of stroke samples called tokens in a potentially large database of writings from a user. Second, we refine the user’s most recently written strokes by averaging them with the matching tokens. Our approach works without handwriting recognition, and does not require a database of predefined letters, words, or shapes. Our results show improved results for a wide range of writing styles and drawings.


    1. Arvo, J., and Novins, K. 2000. Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. In In Proc. of UIST, ACM. Google ScholarDigital Library
    2. Bahlmann, C., and Burkhardt, H. 2004. The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. PAMI 26, 3. Google ScholarDigital Library
    3. Bahlmann, C., Haasdonk, B., and Burkhardt, H. 2002. Online handwriting recognition with support vector machines-a kernel approach. In Proc. of Work. on Frontiers in Handwriting Recognition, IEEE. Google ScholarDigital Library
    4. Baran, I., Lehtinen, J., and Popović, J. 2010. Sketching clothoid splines using shortest paths. In Computer Graphics Forum, vol. 29.Google Scholar
    5. Belhumeur, P. 1996. A bayesian approach to binocular steropsis. IJCV 19, 3. Google ScholarDigital Library
    6. Buades, A., Coll, B., and Morel, J. 2005. A non-local algorithm for image denoising. In In Proc. of CVPR, vol. 2. Google ScholarDigital Library
    7. Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2012. Where do people draw lines? ACM Trans. Graph. 55, 1. Google ScholarDigital Library
    8. Dixon, D., Prasad, M., and Hammond, T. 2010. icandraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In Proc. of the Int. Conf. on Human Factors in Computing Systems, ACM. Google ScholarDigital Library
    9. Dudek, G., and Tsotsos, J. 1997. Shape representation and recognition from multiscale curvature. Computer Vision and Image Understanding 68, 2. Google ScholarDigital Library
    10. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. 31, 4. Google ScholarDigital Library
    11. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., and Schmidhuber, J. 2009. A novel connectionist system for unconstrained handwriting recognition. PAMI 31, 5. Google ScholarDigital Library
    12. Guyon, I., Schomaker, L., Plamondon, R., Liberman, M., and Janet, S. 1994. Unipen project of on-line data exchange and recognizer benchmarks. In In Proc. of ICPR, vol. 2, IEEE.Google Scholar
    13. Igarashi, T., Matsuoka, S., Kawachiya, S., and Tanaka, H. 1997. Interactive beautification: a technique for rapid geometric design. In In Proc. of UIST. Google ScholarDigital Library
    14. Igarashi, T., Matsuoka, S., and Tanaka, H. 1999. Teddy: a sketching interface for 3d freeform design. In Proc. of the Conf. on Computer Graphics and Interactive Techniques. Google ScholarDigital Library
    15. Jain, A., and Namboodiri, A. 2003. Indexing and retrieval of on-line handwritten documents. In Conf. on Document Analysis and Recognition. Google ScholarDigital Library
    16. Jawahar, C., Balasubramanian, A., Meshesha, M., and Namboodiri, A. 2009. Retrieval of online handwriting by synthesis and matching. Pattern Recognition 42, 7. Google ScholarDigital Library
    17. Karpenko, O., and Hughes, J. 2006. Smoothsketch: 3d free-form shapes from complex sketches. In ACM Trans. Graph., vol. 25. Google ScholarDigital Library
    18. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proc. of the IEEE 86, 11.Google ScholarCross Ref
    19. Lee, Y., Zitnick, C., and Cohen, M. 2011. Shadowdraw: real-time user guidance for freehand drawing. In ACM Trans. Graph., vol. 30. Google ScholarDigital Library
    20. Limpaecher, A., Feltman, N., Treuille, A., and Cohen, M. 2013. Real-time drawing assistance through crowdsourcing. ACM Trans. Graph. 32, 4. Google ScholarDigital Library
    21. Lu, J., Yu, F., Finkelstein, A., and DiVerdi, S. 2012. Helpinghand: example-based stroke stylization. ACM Trans. Graph. 31, 4. Google ScholarDigital Library
    22. Mokhtarian, F., and Mackworth, A. 1992. A theory of multiscale, curvature-based shape representation for planar curves. PAMI 14, 8. Google ScholarDigital Library
    23. Olsen, L., Samavati, F., Sousa, M., and Jorge, J. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1. Google ScholarDigital Library
    24. Orbay, G., and Kara, L. 2011. Beautification of design sketches using trainable stroke clustering and curve fitting. TVCG 17, 5. Google ScholarDigital Library
    25. Pavlidis, T., and Van Wyk, C. 1985. An automatic beautifier for drawings and illustrations. Computer Graphics 85 19, 3. Google ScholarDigital Library
    26. Plamondon, R., and Srihari, S. 2000. Online and off-line handwriting recognition: a comprehensive survey. PAMI 22, 1. Google ScholarDigital Library
    27. Plötz, T., and Fink, G. 2009. Markov models for offline handwriting recognition: a survey. Int. J. on Document Analysis and Recognition 12, 4. Google ScholarDigital Library
    28. Rivers, A., Adams, A., and Durand, F. 2012. Sculpting by numbers. ACM Trans. Graph. 31, 6. Google ScholarDigital Library
    29. Schmid, J., Senn, M., Gross, M., and Sumner, R. 2011. Overcoat: an implicit canvas for 3d painting. ACM Trans. Graph. 30, 4. Google ScholarDigital Library
    30. Senior, A., and Robinson, A. 1998. An off-line cursive handwriting recognition system. PAMI 20, 3. Google ScholarDigital Library
    31. Simard, P., Steinkraus, D., and Agrawala, M. 2005. Ink normalization and beautification. In Conf. on Document Analysis and Recognition, IEEE. Google ScholarDigital Library
    32. Thiel, Y., Singh, K., and Balakrishnan, R. 2011. Elasti-curves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2d curves. In In Proc. of UIST. Google ScholarDigital Library
    33. Vinciarelli, A., and Perrone, M. 2003. Combining online and offline handwriting recognition. In Conf. on Document Analysis and Recognition. Google ScholarDigital Library
    34. Whitney, H. 1937. On regular closed curves in the plane. Compositio Mathematica 4.Google Scholar
    35. Zanibbi, R., Novins, K., Arvo, J., and Zanibbi, K. 2001. Aiding manipulation of handwritten mathematical expressions through style-preserving morphs. Graphics Interface 2001. Google ScholarDigital Library

ACM Digital Library Publication:

Overview Page: