“Colorization using optimization” by Levin, Lischinski and Weiss

Colorization is a computer-assisted process of adding color to a monochrome image or movie. The process typically involves segmenting images into regions and tracking these regions across image sequences. Neither of these tasks can be performed reliably in practice; consequently, colorization requires considerable user intervention and remains a tedious, time-consuming, and expensive task.In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise; neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.

1. BURNS, G. Colorization. Museum of Broadcast Communications: Encyclopedia of Television, http://www.museum.tv/archives/etv/index.html.Google Scholar
2. COOPER, R. 1991. Colorization and moral rights: Should the United States adopt unified protection for artists? Journalism Quarterly (Urbana, Illinois), Autumn.Google Scholar
3. JACK, K. 2001. Video Demystified, 3rd edition ed. Elsevier Science & Technology.Google Scholar
4. LUCAS, B., AND KANADE, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. Int. Joint Conf. AI 674–679.Google Scholar
5. MARKLE, W., AND HUNT, B., 1987. Coloring a black and white signal using motion detection. Canadian patent no. 1291260, Dec.Google Scholar
6. NEURALTEK, 2003. BlackMagic photo colorization software, version 2.8 http://www.timebrush.com/blackmagic.Google Scholar
7. PERONA, P., AND MALIK, J. 1989. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on PAMI 8, 5, 565–593.Google Scholar
8. PRESS, W., TEUKOLSKY, S., VETTERLING, W., AND FLANNERY, B. 1992. Numerical Recipes in C: The art of scientific computing. Cambridge University Press. Google ScholarDigital Library
9. SHI, J., AND MALIK, J. 1997. Normalized cuts and image segmentation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 731–737. Google ScholarDigital Library
10. SILBERG, J., 1998. The Pleasantville post production team that focussed on the absence of color. Cinesite Press Article, http://www.cinesite.com/core/press/articles/1998/10_00_98-team.html.Google Scholar
11. TANG, B., SAPIRO, G., AND CASSELES, V. 2001. Color image enhancement via chromaticity diffusion. IEEE Transactions on Image Processing 10, 5, 701–708. Google ScholarDigital Library
12. TORRALBA, A., AND FREEMAN, W. T. 2003. Properties and applications of shape recipes. In IEEE Computer Vision and Pattern Recognition (CVPR).Google Scholar
13. WEISS, Y. 1999. Segmentation using eigenvectors: A unifying view. In Proceedings ICCV, 975–982. Google ScholarDigital Library
14. WELSH, T., ASHIKHMIN, M., AND MUELLER, K. 2002. Transferring color to greyscale images. ACM Transactions on Graphics 21, 3 (July), 277–280. Google ScholarDigital Library
15. ZOMET, A., AND PELEG, S. 2002. Multi-sensor super resolution. In Proceedings of the IEEE Workshop on Applications of Computer Vision. Google ScholarDigital Library