“Unmixing-Based Soft Color Segmentation for Image Manipulation” by Aydin, Aksoy, Smolic and Pollefeys
Conference:
Type:
Title:
- Unmixing-Based Soft Color Segmentation for Image Manipulation
Session/Category Title: Color & Compositing
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a new method for decomposing an image into a set of soft color segments that are analogous to color layers with alpha channels that have been commonly utilized in modern image manipulation software. We show that the resulting decomposition serves as an effective intermediate image representation, which can be utilized for performing various, seemingly unrelated, image manipulation tasks. We identify a set of requirements that soft color segmentation methods have to fulfill, and present an in-depth theoretical analysis of prior work. We propose an energy formulation for producing compact layers of homogeneous colors and a color refinement procedure, as well as a method for automatically estimating a statistical color model from an image. This results in a novel framework for automatic and high-quality soft color segmentation that is efficient, parallelizable, and scalable. We show that our technique is superior in quality compared to previous methods through quantitative analysis as well as visually through an extensive set of examples. We demonstrate that our soft color segments can easily be exported to familiar image manipulation software packages and used to produce compelling results for numerous image manipulation applications without forcing the user to learn new tools and workflows.
References:
1. Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys, and Aljoša Smolić. 2016. Interactive high-quality green-screen keying via color unmixing. ACM Trans. Graph. 35, 5 (2016), 152:1–152:12. Google ScholarDigital Library
2. Dimitri P. Bertsekas. 1982. The method of multipliers for equality constrained problems. In Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York, NY, 96–157.Google Scholar
3. Robert Carroll, Ravi Ramamoorthi, and Maneesh Agrawala. 2011. Illumination decomposition for material recoloring with consistent interreflections. ACM Trans. Graph. 30, 4 (2011), 43:1–43:10. Google ScholarDigital Library
4. Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein. 2015. Palette-based photo recoloring. ACM Trans. Graph. 34, 4 (2015), 139:1–139:11. Google ScholarDigital Library
5. Qifeng Chen, Dingzeyu Li, and Chi-Keung Tang. 2013. KNN matting. IEEE Trans. Pattern Anal. Mach. Intell. 35, 9 (2013), 2175–2188. Google ScholarDigital Library
6. Yung-Yu Chuang, Brian Curless, David H. Salesin, and Richard Szeliski. 2001. A Bayesian approach to digital matting. In Proc. CVPR.Google Scholar
7. Yu Feng and Greg Hamerly. 2006. PG-means: Learning the number of clusters in data. In Proc. NIPS. Google ScholarDigital Library
8. Eduardo S. L. Gastal and Manuel M. Oliveira. 2010. Shared sampling for real-time alpha matting. Comput. Graph. Forum 29, 2 (2010), 575–584.Google ScholarCross Ref
9. Greg Hamerly and Charles Elkan. 2003. Learning the K in K-means. In Proc. NIPS. Google ScholarDigital Library
10. Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35, 6 (2013), 1397–1409. Google ScholarDigital Library
11. Anat Levin, Dani Lischinski, and Yair Weiss. 2008a. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2 (2008), 228–242. Google ScholarDigital Library
12. Anat Levin, Alex Rav-Acha, and Dani Lischinski. 2008b. Spectral matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 10 (2008), 1699–1712. Google ScholarDigital Library
13. Ido Omer and Michael Werman. 2004. Color lines: Image specific color representation. In Proc. CVPR. Google ScholarDigital Library
14. Jinshan Pan, Zhe Hu, Zhixun Su, Hsin-Ying Lee, and Min-Hsuan Yang. 2016. Soft-segmentation guided object motion deblurring. In Proc. CVPR.Google ScholarCross Ref
15. J. Pont-Tuset and F. Marques. 2015. Supervised evaluation of image segmentation and object proposal techniques. IEEE Trans. Pattern Anal. Mach. Intell. 38, 7 (2015), 1465–1478.Google ScholarCross Ref
16. Thomas Porter and Tom Duff. 1984. Compositing digital images. SIGGRAPH Comput. Graph. 18, 3 (1984), 253–259. Google ScholarDigital Library
17. Iulia Posirca, Yunmei Chen, and Celia Z. Barcelos. 2011. A new stochastic variational PDE model for soft Mumford-Shah segmentation. J. Math. Anal. Appl. 384, 1 (2011), 104–114.Google ScholarCross Ref
18. C. Richardt, J. Lopez-Moreno, A. Bousseau, M. Agrawala, and G. Drettakis. 2014. Vectorising bitmaps into semi-transparent gradient layers. Comput. Graph. Forum 33, 4 (2014), 11–19.Google ScholarDigital Library
19. Mark A. Ruzon and Carlo Tomasi. 2000. Alpha estimation in natural images. In Proc. CVPR.Google Scholar
20. Mark Schmidt. 2007. UGM: A Matlab toolbox for probabilistic undirected graphical models. Retrieved from http://www.cs.ubc.ca/∼schmidtm/Software/UGM.html.Google Scholar
21. YiChang Shih, Dilip Krishnan, Fredo Durand, and William T. Freeman. 2015. Reflection removal using ghosting cues. In Proc. CVPR.Google Scholar
22. D. Singaraju and R. Vidal. 2011. Estimation of alpha mattes for multiple image layers. IEEE Trans. Pattern Anal. Mach. Intell. 33, 7 (2011), 1295–1309. Google ScholarDigital Library
23. Yu-Wing Tai, Jiaya Jia, and Chi-Keung Tang. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In Proc. CVPR. Google ScholarDigital Library
24. Yu-Wing Tai, Jiaya Jia, and Chi-Keung Tang. 2007. Soft color segmentation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 29, 9 (2007), 1520–1537. Google ScholarDigital Library
25. Jianchao Tan, Jyh-Ming Lien, and Yotam Gingold. 2016. Decomposing images into layers via RGB-space geometry. ACM Trans. Graph. 36, 1 (2016), 7:1–7:14. Google ScholarDigital Library
26. F. Yang, H. Lu, and Y. W. Chen. 2010b. Robust tracking based on boosted color soft segmentation and ICA-R. In Proc. ICIP.Google Scholar
27. Lei Yang, Pedro V. Sander, Jason Lawrence, and Hugues Hoppe. 2011. Antialiasing recovery. ACM Trans. Graph. 30, 3 (2011), 22:1–22:9. Google ScholarDigital Library
28. W. Yang, J. Cai, J. Zheng, and J. Luo. 2010a. User-friendly interactive image segmentation through unified combinatorial user inputs. IEEE Trans. Image Process. 19, 9 (2010), 2470–2479. Google ScholarDigital Library
29. Sai-Kit Yeung, Tai-Pang Wu, and Chi-Keung Tang. 2008. Extracting smooth and transparent layers from a single image. In Proc. CVPR.Google Scholar
