“Invertible grayscale”
Conference:
Type(s):
Title:
- Invertible grayscale
Session/Category Title: Image processing
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Once a color image is converted to grayscale, it is a common belief that the original color cannot be fully restored, even with the state-of-the-art colorization methods. In this paper, we propose an innovative method to synthesize invertible grayscale. It is a grayscale image that can fully restore its original color. The key idea here is to encode the original color information into the synthesized grayscale, in a way that users cannot recognize any anomalies. We propose to learn and embed the color-encoding scheme via a convolutional neural network (CNN). It consists of an encoding network to convert a color image to grayscale, and a decoding network to invert the grayscale to color. We then design a loss function to ensure the trained network possesses three required properties: (a) color invertibility, (b) grayscale conformity, and (c) resistance to quantization error. We have conducted intensive quantitative experiments and user studies over a large amount of color images to validate the proposed method. Regardless of the genre and content of the color input, convincing results are obtained in all cases.
References:
1. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A system for large-scale machine learning.. In OSDI, Vol. 16. 265–283. Google ScholarDigital Library
2. Xiaobo An and Fabio Pellacini. 2008. AppProp: all-pairs appearance-space edit propagation. ACM Transactions on Graphics (TOG) 27, 3 (2008), 40. Google ScholarDigital Library
3. Mohammad Haris Baig and Lorenzo Torresani. 2017. Multiple hypothesis colorization and its application to image compression. Computer Vision and Image Understanding (CVIU 164 (2017), 111–123.Google Scholar
4. Raja Bala and Reiner Eschbach. 2004. Spatial color-to-grayscale transform preserving chrominance edge information. In Color and Imaging Conference.Google Scholar
5. Miguel A Carreira-Perpinán and Ramin Raziperchikolaei. 2015. Hashing with binary autoencoders. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
6. Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein. 2015. Palette-based photo recoloring. ACM Transactions on Graphics (TOG) 34, 4 (2015), 139. Google ScholarDigital Library
7. Guillaume Charpiat, Matthias Hofmann, and Bernhard Schölkopf. 2008. Automatic image colorization via multimodal predictions. In European Conference on Computer Vision (ECCV). Google ScholarDigital Library
8. Zezhou Cheng, Qingxiong Yang, and Bin Sheng. 2015. Deep colorization. In IEEE International Conference on Computer Vision (ICCV). Google ScholarDigital Library
9. Alex Yong-Sang Chia, Shaojie Zhuo, Raj Kumar Gupta, Yu-Wing Tai, Siu-Yeung Cho, Ping Tan, and Stephen Lin. 2011. Semantic colorization with internet images. ACM Transactions on Graphics (TOG) 30, 6 (2011), 156. Google ScholarDigital Library
10. Aditya Deshpande, Jason Rock, and David Forsyth. 2015. Learning large-scale automatic image colorization. In IEEE International Conference on Computer Vision (ICCV). Google ScholarDigital Library
11. Sovann En, Bruno Crémilleux, and Frédéric Jurie. 2017. Unsupervised deep hashing with stacked convolutional autoencoders. (2017).Google Scholar
12. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2012. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html. (2012).Google Scholar
13. Amy A Gooch, Sven C Olsen, Jack Tumblin, and Bruce Gooch. 2005. Color2gray: salience-preserving color removal. ACM Transactions on Graphics (TOG) 24, 3 (2005), 634–639. Google ScholarDigital Library
14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In IEEE Conference Computer Vision and Pattern Recognition (CVPR).Google Scholar
15. Yi-Chin Huang, Yi-Shin Tung, Jun-Cheng Chen, Sung-Wen Wang, and Ja-Ling Wu. 2005. An adaptive edge detection based colorization algorithm and its applications. In ACM International Conference on Multimedia (MM). Google ScholarDigital Library
16. Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. 2016. Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Transactions on Graphics (TOG) 35, 4 (2016), 110. Google ScholarDigital Library
17. Revital Irony, Daniel Cohen-Or, and Dani Lischinski. 2005. Colorization by Example. In Rendering Techniques. Google ScholarDigital Library
18. Yongjin Kim, Cheolhun Jang, Julien Demouth, and Seungyong Lee. 2009. Robust color-to-gray via nonlinear global mapping. ACM Transactions on Graphics (TOG) 28, 5 (2009), 161. Google ScholarDigital Library
19. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1511.06349 (2014).Google Scholar
20. Jung Gap Kuk, Jae Hyun Ahn, and Nam Ik Cho. 2010. A color to grayscale conversion considering local and global contrast. In Asian Conference on Computer Vision (ACCV).Google Scholar
21. Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2016. Learning representations for automatic colorization. In European Conference on Computer Vision (ECCV).Google ScholarCross Ref
22. Anat Levin, Dani Lischinski, and Yair Weiss. 2004. Colorization using optimization. ACM Transactions on Graphics (TOG) 23, 3 (2004), 689–694. Google ScholarDigital Library
23. Qiegen Liu, Peter X Liu, Weisi Xie, Yuhao Wang, and Dong Liang. 2015. GcsDecolor: gradient correlation similarity for efficient contrast preserving decolorization. IEEE Transactions on Image Processing (TIP) 24, 9 (2015), 2889–2904.Google ScholarDigital Library
24. Xiaopei Liu, Liang Wan, Yingge Qu, Tien-Tsin Wong, Stephen Lin, Chi-Sing Leung, and Pheng-Ann Heng. 2008. Intrinsic colorization. ACM Transactions on Graphics (TOG) 27, 5 (2008), 152. Google ScholarDigital Library
25. Cewu Lu, Li Xu, and Jiaya Jia. 2012. Contrast preserving decolorization. In IEEE International Conference on Computational Photography (ICCP).Google Scholar
26. Qing Luan, Fang Wen, Daniel Cohen-Or, Lin Liang, Ying-Qing Xu, and Heung-Yeung Shum. 2007. Natural image colorization. In Eurographics Conference on Rendering Techniques. Google ScholarDigital Library
27. Laszlo Neumann, M Čadík, and Antal Nemcsics. 2007. An efficient perception-based adaptive color to gray transformation. In Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging. Google ScholarDigital Library
28. Yingge Qu, Tien-Tsin Wong, and Pheng-Ann Heng. 2006. Manga colorization. ACM Transactions on Graphics (TOG) 25, 3 (2006), 1214–1220. Google ScholarDigital Library
29. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention.Google ScholarCross Ref
30. Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. 2017. Scribbler: Controlling deep image synthesis with sketch and color. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
31. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
32. Mingli Song, Dacheng Tao, Chun Chen, Xuelong Li, and Chang Wen Chen. 2010. Color to gray: Visual cue preservation. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 9 (2010), 1537–1552. Google ScholarDigital Library
33. Baoyuan Wang, Yizhou Yu, Tien-Tsin Wong, Chun Chen, and Ying-Qing Xu. 2010. Data-driven image color theme enhancement. ACM Transactions on Graphics (TOG) 29, 6 (2010), 146. Google ScholarDigital Library
34. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing (TIP) 13, 4 (2004), 600–612. Google ScholarDigital Library
35. Tomihisa Welsh, Michael Ashikhmin, and Klaus Mueller. 2002. Transferring color to greyscale images. ACM Transactions on Graphics (TOG) 21, 3 (2002), 277–280. Google ScholarDigital Library
36. Kun Xu, Yong Li, Tao Ju, Shi-Min Hu, and Tian-Qiang Liu. 2009. Efficient affinity-based edit propagation using kd tree. ACM Transactions on Graphics (TOG) 28, 5 (2009), 118. Google ScholarDigital Library
37. Richard Zhang, Phillip Isola, and Alexei A Efros. 2016. Colorful image colorization. In European Conference on Computer Vision (ECCV).Google ScholarCross Ref
38. Richard Zhang, Jun-Yan Zhu, Phillip Isola, Xinyang Geng, Angela S Lin, Tianhe Yu, and Alexei A Efros. 2017. Real-time user-guided image colorization with learned deep priors. ACM Transactions on Graphics (TOG) 36, 4 (2017), 119. Google ScholarDigital Library
39. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision (ICCV).Google ScholarCross Ref
