“Palette-based photo recoloring” by Chang, Fried, Lin, DiVerdi and Finkelstein

Image editing applications offer a wide array of tools for color manipulation. Some of these tools are easy to understand but offer a limited range of expressiveness. Other more powerful tools are time consuming for experts and inscrutable to novices. Researchers have described a variety of more sophisticated methods but these are typically not interactive, which is crucial for creative exploration. This paper introduces a simple, intuitive and interactive tool that allows non-experts to recolor an image by editing a color palette. This system is comprised of several components: a GUI that is easy to learn and understand, an efficient algorithm for creating a color palette from an image, and a novel color transfer algorithm that recolors the image based on a user-modified palette. We evaluate our approach via a user study, showing that it is faster and easier to use than two alternatives, and allows untrained users to achieve results comparable to those of experts using professional software.

1. An, X., and Pellacini, F. 2008. Appprop: All-pairs appearance-space edit propagation. In ACM SIGGRAPH 2008 Papers, ACM, SIGGRAPH ’08, 40:1–40:9. Google ScholarDigital Library
2. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input / output image pairs. In The Twenty-Fourth IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
3. Chang, Y., Saito, S., Uchikawa, K., and Nakajima, M. 2005. Example-based color stylization of images. ACM Transactions on Applied Perception 2, 3 (July), 322345. Google ScholarDigital Library
4. Chen, X., Zou, D., Zhao, Q., and Tan, P. 2012. Manifold preserving edit propagation. ACM Trans. Graph. 31, 6 (Nov), 132:1–132:7. Google ScholarDigital Library
5. Chen, X., Zou, D., Li, J., Cao, X., Zhao, Q., and Zhang, H., 2014. Sparse dictionary learning for edit propagation of high-resolution images. Computer Vision and Pattern Recognition (CVPR), June. Google ScholarDigital Library
6. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. Association for Computing Machinery, Inc.Google Scholar
7. Csurka, G., Skaff, S., Marchesotti, L., and Saunders, C. 2010. Learning moods and emotions from color combinations. In Proceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing, ACM, 298–305. Google ScholarDigital Library
8. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. In ACM Transactions on Graphics (TOG), vol. 21, ACM, 249–256. Google ScholarDigital Library
9. Hacohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D., 2011. Nrdc: Non-rigid dense correspondence with applications for image enhancement. ACM SIGGRAPH 2011 papers, Article No. 70. Google ScholarDigital Library
10. Hacohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2013. Optimizing color consistency in photo collections. ACM Trans. Graph. 32, 4 (July), 38:1–38:10. Google ScholarDigital Library
11. Hou, X., and Zhang, L. 2007. Color conceptualization. In Proceedings of the 15th International Conference on Multimedia, ACM, MULTIMEDIA ’07, 265–268. Google ScholarDigital Library
12. Kanungo, T., Mount, D., Netanyahu, N., Piatko, C., Silverman, R., and Wu, A. 2002. An efficient k-means clustering algorithm: analysis and implementation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 24, 7 (Jul), 881–892. Google ScholarDigital Library
13. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. In ACM SIGGRAPH 2004 Papers, ACM, SIGGRAPH ’04, 689–694. Google ScholarDigital Library
14. Li, C., and Chen, T. 2009. Aesthetic visual quality assessment of paintings. Selected Topics in Signal Processing, IEEE Journal of 3, 2, 236–252.Google Scholar
15. Li, Y., Adelson, E., and Agarwala, A. 2008. Scribbleboost: Adding classification to edge-aware interpolation of local image and video adjustments. In Computer Graphics Forum, vol. 27, Wiley Online Library, 1255–1264. Google ScholarDigital Library
16. Li, Y., Ju, T., and Hu, S.-M. 2010. Instant propagation of sparse edits on images and videos. In Computer Graphics Forum, vol. 29, Wiley Online Library, 2049–2054.Google Scholar
17. Lin, S., and Hanrahan, P. 2013. Modeling how people extract color themes from images. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13. Google ScholarDigital Library
18. Lin, S., Ritchie, D., Fisher, M., and Hanrahan, P. 2013. Probabilistic color-by-numbers: Suggesting pattern colorizations using factor graphs. vol. 32, 37:1–37:12. Google ScholarDigital Library
19. Liu, Y., Cohen, M., Uyttendaele, M., and Rusinkiewicz, S. 2014. Autostyle: Automatic style transfer from image collections to users images. Computer Graphics Forum 33, 4, 21–31. Google ScholarDigital Library
20. Luo, M. R., Cui, G., and Rigg, B. 2001. The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Research & Application 26, 5, 340–350.Google ScholarCross Ref
21. Marks, J., Andalman, B., Beardsley, P. A., Freeman, W., Gibson, S., Hodgins, J., Kang, T., Mirtich, B., Pfister, H., Ruml, W., Ryall, K., Seims, J., and Shieber, S. 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’97, 389–400. Google ScholarDigital Library
22. McLachlan, G., and Peel, D. 2004. Finite mixture models. John Wiley & Sons.Google Scholar
23. Mit-Adobe Fivek Dataset, 2011. http://groups.csail.mit.edu/graphics/fivek_dataset/.Google Scholar
24. Mojsilovic, A. 2005. A computational model for color naming and describing color composition of images. Image Processing, IEEE Transactions on 14, 5, 690–699. Google ScholarDigital Library
25. Mori, G. 2005. Guiding model search using segmentation. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, vol. 2, 1417–1423 Vol. 2. Google ScholarDigital Library
26. O’Donovan, P., Agarwala, A., and Hertzmann, A. 2011. Color Compatibility From Large Datasets. ACM Transactions on Graphics 30, 4. Google ScholarDigital Library
27. Pelleg, D., and Moore, A. 2000. X-means: Extending k-means with efficient estimation of the number of clusters. In Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann, San Francisco, 727–734. Google ScholarDigital Library
28. Pitie, F., Kokaram, A. C., and Dahyot, R. 2005. N-dimensional probability density function transfer and its application to color transfer. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on, vol. 2, IEEE, 1434–1439. Google ScholarDigital Library
29. Qu, Y., Wong, T.-T., and Heng, P.-A. 2006. Manga colorization. ACM Transactions on Graphics (SIGGRAPH 2006 issue) 25, 3 (July), 1214–1220. Google ScholarDigital Library
30. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications 21, 5 (September), 3441. Google ScholarDigital Library
31. Secord, A. 2002. Weighted voronoi stippling. In Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, ACM, 37–43. Google ScholarDigital Library
32. Shapira, L., Shamir, A., and Cohen-Or, D. 2009. Image appearance exploration by model-based navigation. In Computer Graphics Forum, vol. 28, 629–638.Google ScholarCross Ref
33. Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. 2004. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Transactions on 13, 4, 600–612. Google ScholarDigital Library
34. Wang, B., Yu, Y., Wong, T.-T., Chen, C., and Xu, Y.-Q. 2010. Data-driven image color theme enhancement. In ACM SIGGRAPH Asia 2010 Papers, ACM, SIGGRAPH ASIA ’10, 146:1–146:10. Google ScholarDigital Library
35. Wang, X., Jia, J., and Cai, L. 2013. Affective image adjustment with a single word. Vis. Comput. 29, 11 (Nov.), 1121–1133. Google ScholarDigital Library
36. Wing Tai, Y., Jia, J., and Keung Tang, C. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In Proc. Computer Vision and Pattern Recognition, 747–754. Google ScholarDigital Library
37. Xu, K., Li, Y., Ju, T., Hu, S.-M., and Liu, T.-Q. 2009. Efficient affinity-based edit propagation using k-d tree. In ACM SIGGRAPH Asia 2009 Papers, ACM, SIGGRAPH Asia ’09, 118:1–118:6. Google ScholarDigital Library
38. Yoo, J.-D., Park, M.-K., Cho, J.-H., and Lee, K. H. 2013. Local color transfer between images using dominant colors. J. Electron. Imaging 22, 3 (July).Google ScholarCross Ref