“From Image Parsing to Painterly Rendering” by Zeng, Zhao, Xiong and Zhu

We present a semantics-driven approach for stroke-based painterly rendering, based on recent image parsing techniques [Tu et al. 2005; Tu and Zhu 2006] in computer vision. Image parsing integrates segmentation for regions, sketching for curves, and recognition for object categories. In an interactive manner, we decompose an input image into a hierarchy of its constituent components in a parse tree representation with occlusion relations among the nodes in the tree. To paint the image, we build a brush dictionary containing a large set (760) of brush examples of four shape/appearance categories, which are collected from professional artists, then we select appropriate brushes from the dictionary and place them on the canvas guided by the image semantics included in the parse tree, with each image component and layer painted in various styles. During this process, the scene and object categories also determine the color blending and shading strategies for inhomogeneous synthesis of image details. Compared with previous methods, this approach benefits from richer meaningful image semantic information, which leads to better simulation of painting techniques of artists using the high-quality brush dictionary. We have tested our approach on a large number (hundreds) of images and it produced satisfactory painterly effects.

1. Boykov, Y. and Jolly, M.-P. 2001. Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV’01). Vol. 1. 105–112.
2. Chen, H. and Zhu, S.-C. 2006. A generative sketch model for human hair analysis and synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 28, 7, 1025–1040. 
3. Chu, N. S.-H. and Tai, C.-L. 2005. Moxi: Real-Time ink dispersion in absorbent paper. ACM Trans. Graph. 24, 3, 504–511. 
4. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graph. 25, 3, 624–630. 
5. Comaniciu, D. and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603–619. 
6. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-Generated watercolor. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’97). 421–430. 
7. Funch, B. S. 1997. The Psychology of Art Appreciation. Museum Tusculanum Press.
8. Gooch, A., Gooch, B., Shirley, P., and Cohen, E. 1998. A non-photorealistic lighting model for automatic technical illustration. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’98). 447–452. 
9. Gooch, B., Coombe, G., and Shirley, P. 2002. Artistic vision: Painterly rendering using computer vision techniques. In Proceedings of the 2nd International Symposium on Non-Photorealistic Animation and Rendering (NPAR’02). 83–90. 
10. Gooch, B. and Gooch, A. 2001. Non-Photorealistic Rendering. A K Peters, Ltd. 
11. Gooch, B., Sloan, P.-P. J., Gooch, A., Shirley, P., and Riesenfeld, R. 1999. Interactive technical illustration. In Proceedings of the 1999 Symposium on Interactive 3D Graphics (I3D’99). 31–38. 
12. Guo, C.-E., Zhu, S.-C., and Wu, Y. N. 2007. Primal sketch: Integrating structure and texture. Comput. Vis. Image Understand. 106, 1, 5–19. 
13. Haeberli, P. 1990. Paint by numbers: Abstract image representations. In Proceedings of the 17th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’90). 207–214. 
14. Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’98). 453–460. 
15. Hertzmann, A. 2003. Tutorial: A survey of stroke-based rendering. IEEE Comput. Graph. Appl. 23, 4, 70–81. 
16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01). 327–340. 
17. Li, F.-F., Fergus, R., and Torralba, A. 2005. Recognizing and learning object categories. A short course at ICCV 2005.
18. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. Graph. 23, 3, 303–308. 
19. Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’97). 407–414. 
20. Lowe, D. G. 1999. Object recognition from local scale-invariant features. In Proceedings of the International Conference on Computer Vision (ICCV’99), Volume 2. 1150–1157. 
21. Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman. 
22. Perona, P. 1998. Orientation diffusions. IEEE Trans. Image Proces. 7, 3, 457–467. 
23. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5, 34–41. 
24. Sousa, M. C. and Buchanan, J. W. 1999. Computer-Generated graphite pencil rendering of 3d polygonal models. In Proceedings of EuroGraphics’99 Conference. 195–207.
25. Strassmann, S. 1986. Hairy brushes. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’86). 225–232. 
26. Strothotte, T. and Schlechtweg, S. 2002. Non-Photorealistic Computer Graphics: Modeling, Rendering and Animation. Morgan Kaufmann. 
27. Teece, D. 1998. 3d painting for non-photorealistic rendering. In ACM Conference on Abstracts and Applications (SIGGRAPH’98). 248. 
28. Tu, Z., Chen, X., Yuille, A. L., and Zhu, S.-C. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vis. 63, 2, 113–140. 
29. Tu, Z. and Zhu, S.-C. 2006. Parsing images into regions, curves, and curve groups. Int. J. Comput. Vis. 69, 2, 223–249. 
30. Turk, G. and Banks, D. 1996. Image-Guided streamline placement. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96). 453–460. 
31. Winkenbach, G. and Salesin, D. H. 1994. Computer-Generated pen-and-ink illustration. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’94). 91–100. 
32. Xu, S., Xu, Y., Kang, S. B., Salesin, D. H., Pan, Y., and Shum, H.-Y. 2006. Animating chinese paintings through stroke-based decomposition. ACM Trans. Graph. 25, 2, 239–267. 
33. Yao, B., Yang, X., and Zhu, S.-C. 2007. Introduction to a large-scale general purpose ground truth database: Methodology, annotation tool and benchmarks. In Proceedings of the International Conferences on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR ’07). 169–183.