“Image vectorization and editing via linear gradient layer decomposition” by Du, Kang, Tan, Gingold and Xu
Conference:
Type:
Title:
- Image vectorization and editing via linear gradient layer decomposition
Session/Category Title: Colorful Topics in Imaging
Presenter(s)/Author(s):
Moderator(s):
Abstract:
A key advantage of vector graphics over raster graphics is their editability. For example, linear gradients define a spatially varying color fill with a few intuitive parameters, which are ubiquitously supported in standard vector graphics formats and libraries. By layering regions filled with linear gradients, complex appearances can be created. We propose an automatic method to convert a raster image into layered regions of linear gradients. Given an input raster image segmented into regions, our approach decomposes the resulting regions into opaque and semi-transparent linear gradient fills. Our approach is fully automatic (e.g., users do not identify a background as in previous approaches) and exhaustively considers all possible decompositions that satisfy perceptual cues. Experiments on a variety of images demonstrate that our method is robust and effective.
References:
1. Yağiz Aksoy, Tunç Ozan Aydin, Aljoša Smolić, and Marc Pollefeys. 2017. Unmixing-based soft color segmentation for image manipulation. ACM Transactions on Graphics (TOG) 36, 2 (2017), 19.
2. Yağiz Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, and Wojciech Matusik. 2018. Semantic soft segmentation. ACM Transactions on Graphics (TOG) 37, 4 (2018), 72.
3. Edoardo Alberto Dominici, Nico Schertler, Jonathan Griffin, Shayan Hoshyari, Leonid Sigal, and Alla Sheffer. 2020. PolyFit: Perception-aligned Vectorization of Raster Clip-art via Intermediate Polygonal Fitting. ACM Transaction on Graphics 39, 4 (2020).
4. Mikhail Bessmeltsev and Justin Solomon. 2019. Vectorization of line drawings via polyvector fields. ACM Transactions on Graphics (TOG) 38, 1 (2019), 1–12.
5. Dorin Comaniciu and Peter Meer. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on pattern analysis and machine intelligence 24, 5 (2002), 603–619.
6. Damir Demirović. 2019. An implementation of the mean shift algorithm. Image Processing On Line 9 (2019), 251–268.
7. Zheng-Jun Du, Kai-Xiang Lei, Kun Xu, Jianchao Tan, and Yotam Gingold. 2021. Video recoloring via spatial-temporal geometric palettes. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–16. Publisher: ACM New York, NY, USA.
8. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: a global approach to line drawing vectorization. ACM Transactions on Graphics (TOG) 35, 4 (2016), 120.
9. Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2017. Photo2ClipArt: Image Abstraction and Vectorization Using Layered Linear Gradients. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 36, 6 (November 2017). http://www-sop.inria.fr/reves/Basilic/2017/FLB17
10. Harold N Gabow and Eugene W Myers. 1978. Finding all spanning trees of directed and undirected graphs. SIAM J. Comput. 7, 3 (1978), 280–287.
11. Shayan Hoshyari, Edoardo Alberto Dominici, Alla Sheffer, Nathan Carr, Zhaowen Wang, Duygu Ceylan, I Shen, et al. 2018. Perception-driven semi-structured boundary vectorization. ACM Transactions on Graphics (TOG) 37, 4 (2018), 118.
12. Jiahui Huang, Jun Gao, Vignesh Ganapathi-Subramanian, Hao Su, Yin Liu, Chengcheng Tang, and Leonidas J. Guibas. 2018. DeepPrimitive: Image decomposition by layered primitive detection. Computational Visual Media 4, 4 (01 Dec 2018), 385–397.
13. Steven G. Johnson. 2011. The NLopt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt
14. Byungsoo Kim, Oliver Wang, A. Cengiz Öztireli, and Markus Gross. 2018. Semantic Segmentation for Line Drawing Vectorization Using Neural Networks. Computer Graphics Forum (Proc. Eurographics) 37, 2 (2018), 329–338.
15. Yuki Koyama and Masataka Goto. 2018. Decomposing Images into Layers with Advanced Color Blending. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 397–407.
16. Allan M. M. Leal et al. 2018. autodiff, a modern, fast and expressive C++ library for automatic differentiation. https://autodiff.github.io. https://autodiff.github.io
17. Gregory Lecot and Bruno Levy. 2006. Ardeco: automatic region detection and conversion. In 17th Eurographics Symposium on Rendering-EGSR’06. 349–360.
18. Zicheng Liao, Hugues Hoppe, David Forsyth, and Yizhou Yu. 2012. A subdivision-based representation for vector image editing. IEEE transactions on visualization and computer graphics 18, 11 (2012), 1858–1867.
19. Dong C Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical programming 45, 1 (1989), 503–528.
20. Jorge Lopez-Moreno, Stefan Popov, Adrien Bousseau, Maneesh Agrawala, and George Drettakis. 2013. Depicting Stylized Materials with Vector Shade Trees. ACM Trans. Graph. 32, 4, Article 118 (July 2013), 10 pages.
21. Abhimitra Meka, Mohammad Shafiei, Michael Zollhöfer, Christian Richardt, and Christian Theobalt. 2021. Real-Time Global Illumination Decomposition of Videos. ACM Trans. Graph. 40, 3, Article 22 (aug 2021), 16 pages.
22. Fabio Metelli. 1974. The perception of transparency. Scientific American 230, 4 (1974), 90–99.
23. Jorge Nocedal. 1980. Updating quasi-Newton matrices with limited storage. Mathematics of computation 35, 151 (1980), 773–782.
24. Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. 2008. Diffusion curves: a vector representation for smooth-shaded images. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 92.
25. Ivan Puhachov, William Neveu, Edward Chien, and Mikhail Bessmeltsev. 2021. Keypoint-driven line drawing vectorization via PolyVector flow. ACM Transactions on graphics 40, 6 (2021).
26. Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J Mitra. 2021. Im2vec: Synthesizing vector graphics without vector supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7342–7351.
27. Christian Richardt, Jorge Lopez-Moreno, Adrien Bousseau, Maneesh Agrawala, and George Drettakis. 2014. Vectorising Bitmaps into Semi-Transparent Gradient Layers. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 11–19.
28. Bilge Sayim and Patrick Cavanagh. 2011. The art of transparency. i-Perception 2, 7 (2011), 679–696.
29. Othman Sbai, Camille Couprie, and Mathieu Aubry. 2020. Unsupervised image decomposition in vector layers. In 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 1576–1580.
30. Peter Selinger. 2003. Potrace: a polygon-based tracing algorithm. Potrace (online), http://potrace.sourceforge.net/potrace.pdf (2009-07-01) 2 (2003).
31. I-Chao Shen and Bing-Yu Chen. 2021. ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis. IEEE Transactions on Visualization and Computer Graphics (2021), 1–1.
32. Li Shen, Tan Ping, and Stephen Lin. 2008. Intrinsic Image Decomposition with NonLocal Texture Cues. In Computer Vision and Pattern Recognition (CVPR).
33. Manish Singh and Xiaolei Huang. 2003. Computing layered surface representations: an algorithm for detecting and separating transparent overlays. In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, Vol. 2. IEEE, II–11.
34. Ying Song, Jiaping Wang, Li-Yi Wei, and Wencheng Wang. 2015. Vector regression functions for texture compression. ACM Transactions on Graphics (TOG) 35, 1 (2015), 5.
35. Tibor Stanko, Mikhail Bessmeltsev, David Bommes, and Adrien Bousseau. 2020. Integer-Grid Sketch Simplification and Vectorization. In Computer graphics forum, Vol. 39. Wiley Online Library, 149–161.
36. Jian Sun, Lin Liang, Fang Wen, and Heung-Yeung Shum. 2007. Image vectorization using optimized gradient meshes. In ACM Transactions on Graphics (TOG), Vol. 26. ACM, 11.
37. J. Tan, S. DiVerdi, J. Lu, and Y. Gingold. 2018a. Pigmento: Pigment-Based Image Analysis and Editing. IEEE Transactions on Visualization and Computer Graphics (2018), 1–1.
38. Jianchao Tan, Marek Dvorožňák, Daniel Sýkora, and Yotam Gingold. 2015. Decomposing Time-Lapse Paintings into Layers. ACM Transactions on Graphics (TOG) 34, 4 (July 2015), 61:1–61:10.
39. Jianchao Tan, Jose Echevarria, and Yotam Gingold. 2018b. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry. In SIGGRAPH Asia 2018 Technical Papers. ACM, 262.
40. Jianchao Tan, Jose Echevarria, and Yotam Gingold. 2018c. Palette-based image decomposition, harmonization, and color transfer. arXiv preprint arXiv:1804.01225 (2018).
41. Jianchao Tan, Jyh Ming Lien, and Yotam Gingold. 2016. Decomposing Images into Layers via RGB-Space Geometry. Acm Transactions on Graphics 36, 1 (2016), 1–14.
42. Johan Wagemans, James H. Elder, Michael Kubovy, Stephen E. Palmer, Mary A. Peterson, Manish Singh, and Rüdiger von der Heydt. 2012. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychological Bulletin 138, 6 (2012), 1172–1217.
43. David L Waltz. 1972. Generating semantic descriptions from drawings of scenes with shadows. (1972).
44. Yili Wang, Yifan Liu, and Kun Xu. 2019. An Improved Geometric Approach for Palette-based Image Decomposition and Recoloring. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 11–22. Issue: 7.
45. Guofu Xie, Xin Sun, Xin Tong, and Derek Nowrouzezahrai. 2014. Hierarchical Diffusion Curves for Accurate Automatic Image Vectorization. ACM Trans. Graph. 33, 6, Article 230 (Nov. 2014), 11 pages.
46. Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A Benchmark for Rough Sketch Cleanup. ACM Transactions on Graphics (TOG) 39, 6, Article 163 (Nov. 2020), 14 pages.
47. Q. Zhang, Y. Nie, L. Zhu, C. Xiao, and W.-S. Zheng. 2021. A Blind Color Separation Model for Faithful Palette-based Image Recoloring. IEEE Transactions on Multimedia (2021), 1–1. Conference Name: IEEE Transactions on Multimedia.
48. Qing Zhang, Chunxia Xiao, Hanqiu Sun, and Feng Tang. 2017. Palette-Based Image Recoloring Using Color Decomposition Optimization. IEEE Transactions on Image Processing 26, 4 (2017), 1952–1964.
49. Shuang Zhao, Frédo Durand, and Changxi Zheng. 2018. Inverse diffusion curves using shape optimization. IEEE transactions on visualization and computer graphics 24, 7 (2018), 2153–2166.
