“Example-based image color and tone style enhancement” by Wang, Yu and Xu

Color and tone adjustments are among the most frequent image enhancement operations. We define a color and tone style as a set of explicit or implicit rules governing color and tone adjustments. Our goal in this paper is to learn implicit color and tone adjustment rules from examples. That is, given a set of examples, each of which is a pair of corresponding images before and after adjustments, we would like to discover the underlying mathematical relationships optimally connecting the color and tone of corresponding pixels in all image pairs. We formally define tone and color adjustment rules as mappings, and propose to approximate complicated spatially varying nonlinear mappings in a piecewise manner. The reason behind this is that a very complicated mapping can still be locally approximated with a low-order polynomial model. Parameters within such low-order models are trained using data extracted from example image pairs. We successfully apply our framework in two scenarios, low-quality photo enhancement by transferring the style of a high-end camera, and photo enhancement using styles learned from photographers and designers.

1. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. In SIGGRAPH ’06: ACM SIGGRAPH2006 Papers, ACM, New York, NY, USA, 637–645. Google Scholar
2. Battiato, S., Bosco, A., Castorina, A., and Messina, G. 2004. Automatic image enhancement by content dependent exposure correction. EURASIP Journal on Applied Signal Processing 12. Google Scholar
3. Bhat, P., Curless, B., Cohen, M., and Zitnick, L. 2008. Fourier analysis of the 2d screened poisson equation for gradient domain problems. In European Conference on Computer Vision (ECCV). Google Scholar
4. Chang, Y., Saito, S., Uchikawa, K., and Nakajima, M. 2005. Example-based color stylization of images. ACM Trans. Appl. Percept. 2, 3, 322–345. Google ScholarDigital Library
5. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, ACM, New York, NY, USA, 624–630. Google Scholar
6. Dale, K., Johnson, M., Sunkavalli, K., Matusik, W., and Pfister, H. 2009. Image restoration using online photo collections. In International Conference on Computer Vision (ICCV).Google Scholar
7. Ding, C., and He, X. 2002. Cluster merging and splitting in hierarchical clustering algorithms. In Proceedings of the 2002 IEEE International Conference on Data Mining, IEEE Computer Society, Washington, DC, USA, ICDM ’02. Google ScholarDigital Library
8. Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Trans. on Graphics 21, 3, 249–256. Google ScholarDigital Library
9. Hardoon, D. R., Szedmak, S. R., and Shawe-taylor, J. R. 2004. Canonical correlation analysis: An overview with application to learning methods. Neural Comput. 16 (December), 2639–2664. Google ScholarDigital Library
10. Jobson, D., Rahman, Z., and Woodell, G. 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing 6, 7, 965–976. Google ScholarDigital Library
11. Kang, S., Kapoor, A., and Lischinski, D. 2010. Personalization of image enhancement. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
12. Li, Y., Ju, T., and Hu, S.-M. 2010. Instant propagation of sparse edits on images and videos. Computer Graphics Forum 29, 2049–2054.Google ScholarCross Ref
13. Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3, 646–653. Google ScholarDigital Library
14. Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. on Graphics 22, 3, 313–318. Google ScholarDigital Library
15. Piti, F., and Kokaram, A. 2007. The linear monge-kantorovitch linear colour mapping for example-based colour transfer. Visual Media Production, 4th European Conference on Visual Media Production, London, UK, 1–9.Google Scholar
16. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5, 34–41. Google ScholarDigital Library
17. Shapira, L., Shamir, A., and Cohen-Or, D. 2009. Image appearance exploration by model-based navigation. Comput. Graph. Forum 28, 2, 629–638.Google ScholarCross Ref
18. Shinagawa, Y., and Kunii, T. L. 1998. Unconstrained automatic image matching using multiresolutional critical-point filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 994–1010. Google ScholarDigital Library
19. Siddiqui, H., and Bouman, C. 2008. Hierarchical color correction for camera cell phone images. IEEE Transactions on Image Processing 17, 11, 2138–2155. Google ScholarDigital Library
20. Toledo, S., Rotkin, V., and Chen, D. 2003. Taucs: A library of sparse linear solvers. version 2.2. Tech. rep., Tel-Aviv University.Google Scholar
21. Vapnik, V. 1995. The Nature of Statistical Learning Theory. Springer, New York. Google Scholar
22. Wang, B., Yu, Y., Wong, T.-T., Chen, C., and Xu, Y.-Q. 2010. Data-driven image color theme enhancement. ACM Trans. Graph. 29, 5. Google ScholarDigital Library