“Photo2clipart: image abstraction and vectorization using layered linear gradients”
Conference:
Type(s):
Title:
- Photo2clipart: image abstraction and vectorization using layered linear gradients
Session/Category Title: HDR and Image Manipulation
Presenter(s)/Author(s):
Abstract:
We present a method to create vector cliparts from photographs. Our approach aims at reproducing two key properties of cliparts: they should be easily editable, and they should represent image content in a clean, simplified way. We observe that vector artists satisfy both of these properties by modeling cliparts with linear color gradients, which have a small number of parameters and approximate well smooth color variations. In addition, skilled artists produce intricate yet editable artworks by stacking multiple gradients using opaque and semi-transparent layers. Motivated by these observations, our goal is to decompose a bitmap photograph into a stack of layers, each layer containing a vector path filled with a linear color gradient. We cast this problem as an optimization that jointly assigns each pixel to one or more layer and finds the gradient parameters of each layer that best reproduce the input. Since a trivial solution would consist in assigning each pixel to a different, opaque layer, we complement our objective with a simplicity term that favors decompositions made of few, semi-transparent layers. However, this formulation results in a complex combinatorial problem combining discrete unknowns (the pixel assignments) and continuous unknowns (the layer parameters). We propose a Monte Carlo Tree Search algorithm that efficiently explores this solution space by leveraging layering cues at image junctions. We demonstrate the effectiveness of our method by reverse-engineering existing cliparts and by creating original cliparts from studio photographs.
References:
1. Adobe. 2013. Adobe Illustrator Image Trace, http://blogs.adobe.com/adobeillustrator/2013/07/image-trace-in-illustrator-a-tutorial-and-guide.html. (2013).
2. Yaǧiz Aksoy, Tunç Ozan Aydin, Aljoša Smolić, and Marc Pollefeys. 2017. Unmixing-Based Soft Color Segmentation for Image Manipulation. ACM Transactions on Graphics 36, 2 (2017).
3. Sean Bell, Kavita Bala, and Noah Snavely. 2014. Intrinsic Images in the Wild. ACM Transactions on Graphics (Proc. SIGGRAPH) 33, 4 (2014).
4. C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (2012). Cross Ref
5. Dorin Comaniciu and Peter Meer. 2002. Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 5 (2002).
6. Doug DeCarlo and Anthony Santella. 2002. Stylization and Abstraction of Photographs. ACM Transactions on Graphics (Proc. SIGGRAPH) 21, 3 (2002).
7. Michael D’Zmura, Philippe Colantoni, Kenneth Knoblauch, and Bernard Laget. 1997. Color transparency. Perception 26, 4 (1997).
8. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. Simplicity: a Global Approach to Line Drawing Vectorization. ACM Transactions on Graphics (SIGGRAPH Conference Proceedings) (2016).
9. Timothy Gerstner, Doug DeCarlo, Marc Alexa, Adam Finkelstein, Yotam Gingold, and Andrew Nealen. 2012. Pixelated Image Abstraction. In Proc. International Symposium on Non-photorealistic Animation and Rendering.
10. Stefan Jeschke, David Cline, and Peter Wonka. 2011. Estimating Color and Texture Parameters for Vector Graphics. Computer Graphics Forum (Proc. Eurographics) 30, 2 (April 2011), 523–532. Cross Ref
11. Zhaoyin Jia, A. Gallagher, Yao-Jen Chang, and Tsuhan Chen. 2012. A learning-based framework for depth ordering. In IEEE Computer Vision and Pattern Recognition.
12. Gregory Lecot and Bruno Lévy. 2006. Ardeco: automatic region detection and conversion. In Proceedings of the Eurographics Symposium on Rendering Techniques.
13. Anat Levin, Assaf Zomet, and Yair Weiss. 2004. Separating reflections from a single image using local features. In Proc. of the IEEE conference on Computer Vision and Pattern Recognition. Cross Ref
14. Zicheng Liao, Hugues Hoppe, David Forsyth, and Yizhou Yu. 2012. A Subdivision-Based Representation for Vector Image Editing. IEEE Transactions on Visualization and Computer Graphics 18, 11 (2012).
15. Xueting Liu, Xiangyu Mao, Xuan Yang, Linling Zhang, and Tien-Tsin Wong. 2013. Stereoscopizing Cel Animations. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 32, 6 (2013).
16. Fabio Metelli. 1974. The perception of transparency. Scientific American (1974).
17. Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. 2008. Diffusion curves: a vector representation for smooth-shaded images. ACM Transactions on Graphics (Proc. SIGGRAPH) 27, 3 (2008).
18. Christian Richardt, Jorge Lopez-Moreno, Adrien Bousseau, Maneesh Agrawala, and George Drettakis. 2014. Vectorising Bitmaps into Semi-Transparent Gradient Layers. Computer Graphics Forum (Proc. Eurographics Symposium on Rendering) 33, 4 (2014).
19. Bilge Sayim and Patrick Cavanagh. 2011. The art of transparency. i-Perception 2 (2011).
20. Peter Selinger. 2017. Potrace. (2017).
21. Manish Singh and Xiaolei Huang. 2003. Computing layered surface representations: an algorithm for detecting and separating transparent overlays. In IEEE Computer Vision and Pattern Recognition.
22. Alvy Ray Smith and James F. Blinn. 1996. Blue screen matting. SIGGRAPH (1996).
23. Jian Sun, Lin Liang, Fang Wen, and Heung-Yeung Shum. 2007. Image vectorization using optimized gradient meshes. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3 (2007).
24. Jianchao Tan, Marek Dvorožňák, Daniel Sýkora, and Yotam Gingold. 2015. Decomposing Time-Lapse Paintings into Layers. ACM Transactions on Graphics (Proc. SIGGRAPH) 34, 4 (2015).
25. Jianchao Tan, Jyh-Ming Lien, and Yotam Gingold. 2016. Decomposing Images into Layers via RGB-space Geometry. ACM Transactions on Graphics 36, 1 (2016).
26. Holger Winnemöller, Sven C. Olsen, and Bruce Gooch. 2006. Real-time Video Abstraction. ACM Transactions on Graphics (Proc. SIGGRAPH) 25, 3 (2006).
27. S. Zhao, F. Durand, and C. Zheng. 2017. Inverse Diffusion Curves using Shape Optimization. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2017).
