“Semantic colorization with internet images”
Conference:
Type(s):
Title:
- Semantic colorization with internet images
Session/Category Title: Image Mix and Match
Presenter(s)/Author(s):
Abstract:
Colorization of a grayscale photograph often requires considerable effort from the user, either by placing numerous color scribbles over the image to initialize a color propagation algorithm, or by looking for a suitable reference image from which color information can be transferred. Even with this user supplied data, colorized images may appear unnatural as a result of limited user skill or inaccurate transfer of colors. To address these problems, we propose a colorization system that leverages the rich image content on the internet. As input, the user needs only to provide a semantic text label and segmentation cues for major foreground objects in the scene. With this information, images are downloaded from photo sharing websites and filtered to obtain suitable reference images that are reliable for color transfer to the given grayscale photo. Different image colorizations are generated from the various reference images, and a graphical user interface is provided to easily select the desired result. Our experiments and user study demonstrate the greater effectiveness of this system in comparison to previous techniques.
References:
1. Belongie, S., Malik, J., and Puzhica, J. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. PAMI 24, 4, 509–532. Google ScholarDigital Library
2. Charpiat, G., Hofmann, M., and Schölkopf, B. 2008. Automatic image colorization via multimodal predictions. In Proc. ECCV, 126–139. Google ScholarDigital Library
3. Chen, T., Cheng, M.-M., Tan, P., Shamir, A., and Hu, S.-M. 2009. Sketch2photo: internet image montage. In ACM Trans. Graph., vol. 28, 1–10. Google ScholarDigital Library
4. Comanicu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. In IEEE Trans. PAMI. Google ScholarDigital Library
5. Hasler, D., and Strunk, S. 2003. Measuring colourfulness in natural images. In Human Vision and Electronic Imaging.Google Scholar
6. Hays, J., and Efros, A. 2007. Scene completion using millions of photographs. ACM Trans. Graph. 26, 87–94. Google ScholarDigital Library
7. He, K., Sun, J., and Tang, X. 2010. Guided image filtering. In Proc. ECCV, 1–14. Google ScholarDigital Library
8. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In SIGGRAPH. Google ScholarDigital Library
9. Huang, Y.-C., Tung, Y.-S., Chen, J.-C., Wang, S.-W., and Wu, J.-L. 2005. An adaptive edge detection based colorization algorithm and its applications. In Proc. ACM Multimedia. Google ScholarDigital Library
10. Irony, R., Cohen-Or, D., and Lischinski, D. 2005. Colorization by example. In Proc. EGSR, 201–210. Google ScholarDigital Library
11. Komodakis, N., and Tziritas, G. 2007. Approximate labeling via graph-cuts based on linear programming. IEEE Trans. PAMI 29, 8, 1436–1453. Google ScholarDigital Library
12. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 689–694. Google ScholarDigital Library
13. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. In ACM Trans. Graph., 303–308. Google ScholarDigital Library
14. Liu, T., Sun, J., Zheng, N.-N., Tang, X., and Shum, H.-Y. 2007. Learning to detect a salient object. In Proc. CVPR, 1–8.Google Scholar
15. Liu, X., Wan, L., Qu, Y., Wong, T.-T., Lin, S., Leung, C.-S., and Heng, P.-A. 2008. Intrinsic colorization. ACM Trans. Graph. 27, 1–9. Google ScholarDigital Library
16. Lowe, D. G. 1999. Object recognition from local scale-invariant features. In Proc. ICCV, 1150–1157. Google ScholarDigital Library
17. Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. 2007. Natural image colorization. In Proc. EGSR, 309–320. Google ScholarDigital Library
18. Oliva, A., and Torralba, A. 2006. Building the gist of a scene: The role of global image features in recognition. In Visual Perception, Progress in Brain Research.Google Scholar
19. Oliva, A., and Torralba, A. 2007. The role of context in object recognition. Trends in Cognitive Sciences 11, 12.Google ScholarCross Ref
20. Qu, Y., Wong, T.-T., and Heng, P.-A. 2006. Manga colorization. ACM Trans. Graph. 25, 1214–1220. Google ScholarDigital Library
21. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41. Google ScholarDigital Library
22. Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314. Google ScholarDigital Library
23. Tai, Y.-W., Jia, J., and Tang, C.-K. 2005. Local color transfer via probabilistic segmentation by expectation-maximization. In Proc. CVPR, 747–754. Google ScholarDigital Library
24. Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. In Proc. ACM SIGGRAPH. Google ScholarDigital Library
25. Yatziv, L., and Sapiro, G. 2006. Fast image and video colorization using chrominance blending. In IEEE Trans. PAMI.Google Scholar
26. Zhu, J., Hoi, S., Lyu, M., and Yan, S. 2011. Near-duplicate keyframe retrieval by semi-supervised learning and nonrigid image matching. ACM Trans. Multimedia Comput. Commun. Appl.. Google ScholarDigital Library
