“ColorfulCurves: Palette-Aware Lightness Control and Color Editing via Sparse Optimization” by Chao, Klein, Tan, Echevarria and Gingold
Conference:
Type:
Title:
- ColorfulCurves: Palette-Aware Lightness Control and Color Editing via Sparse Optimization
Session/Category Title: Colorful Topics in Imaging
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Color editing in images often consists of two main tasks: changing hue and saturation, and editing lightness or tone curves. State-of-the-art palette-based recoloring approaches entangle these two tasks. A user’s only lightness control is changing the lightness of individual palette colors. This is inferior to state-of-the-art commercial software, where lightness editing is based on flexible tone curves that remap lightness. However, tone curves are only provided globally or per color channel (e.g., RGB). They are unrelated to the image content. Neither tone curves nor palette-based approaches support direct image-space edits—changing a specific pixel to a desired hue, saturation, and lightness. ColorfulCurves solves both of these problems by uniting palette-based and tone curve editing. In ColorfulCurves, users directly edit palette colors’ hue and saturation, per-palette tone curves, or image pixels (hue, saturation, and lightness). ColorfulCurves solves an L2,1 optimization problem in real-time to find a sparse edit that satisfies all user constraints. Our expert study found overwhelming support for ColorfulCurves over experts’ preferred tools.
References:
1. Adobe. 2020. What is Color Grading in Lightroom? https://blog.adobe.com/en/publish/2020/10/20/introducing-color-grading. [Online; accessed 24-January-2023].
2. Adobe. 2022. Adjust the colors in your photos on your phone or tablet. https://helpx.adobe.com/lightroom-cc/how-to/color-adjustment.html. [Online; accessed 24-January-2023].
3. Naofumi Akimoto, Huachun Zhu, Yanghua Jin, and Yoshimitsu Aoki. 2020. Fast Soft Color Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8277–8286.
4. Yağiz Aksoy, Tunç Ozan Aydin, Marc Pollefeys, and Aljoša Smolić. 2016. Interactive high-quality green-screen keying via color unmixing. ACM Transactions on Graphics (TOG) 35, 5 (2016), 152.
5. Yağiz Aksoy, Tunç Ozan Aydin, Aljoša Smolić, and Marc Pollefeys. 2017. Unmixing-based soft color segmentation for image manipulation. ACM Transactions on Graphics (TOG) 36, 2 (2017), 19.
6. Pontus Andersson, Jim Nilsson, Tomas Akenine-Möller, Magnus Oskarsson, Kalle Åström, and Mark D Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. Proc. ACM Comput. Graph. Interact. Tech. 3, 2 (2020), 15–1.
7. Omri Avrahami, Dani Lischinski, and Ohad Fried. 2022. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18208–18218.
8. Reynold J Bailey and Cindy Grimm. 2006. Perceptually meaningful image editing: Depth. (2006).
9. Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel. 2022. Text2live: Text-driven layered image and video editing. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XV. Springer, 707–723.
10. Blackmagic Design. 2020. DaVinci Resolve 18. https://www.blackmagicdesign.com/products/davinciresolve/color. [Online; accessed 24-January-2023].
11. Ivaylo Boyadzhiev, Kavita Bala, Sylvain Paris, and Edward Adelson. 2015. Band-sifting decomposition for image-based material editing. ACM Transactions on Graphics (TOG) 34, 5 (2015), 1–16.
12. Vladimir Bychkovsky, Sylvain Paris, Eric Chan, and Frédo Durand. 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In CVPR 2011. IEEE, 97–104.
13. Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein. 2015a. Palette-based Photo Recoloring. ACM Trans. Graph. 34, 4 (July 2015).
14. Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein. 2015b. Palette-based photo recoloring. ACM Trans. Graph. 34, 4 (2015), 139–1.
15. Color Grading LLC. 2022. Cinema Grade. https://www.cinemagrade.com/. [Online; accessed 24-January-2023].
16. Paul Debevec and Simon Gibson. 2002. A tone mapping algorithm for high contrast images. In 13th eurographics workshop on rendering: Pisa, Italy. Citeseer.
17. Zheng-Jun Du, Kai-Xiang Lei, Kun Xu, Jianchao Tan, and Yotam Gingold. 2021. Video Recoloring via Spatial-Temporal Geometric Palettes. ACM Transactions on Graphics (TOG) 40, 4 (Aug. 2021).
18. Fredo Durand and Julie Dorsey. 2000. Interactive tone mapping. In Eurographics Workshop on Rendering Techniques. Springer, 219–230.
19. Marie Gardiner. 2022. How to Use the Primaries Color Wheels in DaVinci Resolve. https://photography.tutsplus.com/tutorials/primaries-color-wheels-resolve-cms-41775. [Online; accessed 24-January-2023].
20. Amy A Gooch, Sven C Olsen, Jack Tumblin, and Bruce Gooch. 2005. Color2gray: salience-preserving color removal. ACM Transactions on Graphics (TOG) 24, 3 (2005), 634–639.
21. Mairéad Grogan and Aljosa Smolic. 2020. Image Decomposition Using Geometric Region Colour Unmixing. In European Conference on Visual Media Production (Virtual Event, United Kingdom) (CVMP ’20). Association for Computing Machinery, New York, NY, USA, Article 2, 10 pages.
22. Daichi Horita, Kiyoharu Aizawa, Ryohei Suzuki, Taizan Yonetsuji, and Huachun Zhu. 2022. Fast Nonlinear Image Unblending. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2051–2059.
23. Erum Arif Khan, Erik Reinhard, Roland W Fleming, and Heinrich H Bülthoff. 2006. Image-based material editing. ACM Transactions on Graphics (TOG) 25, 3 (2006), 654–663.
24. Yuki Koyama and Masataka Goto. 2018. Decomposing images into layers with advanced color blending. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 397–407.
25. Life After Photoshop. 2020. Lightroom tone curves explained: Tone Curve vs Point Curve vs Target Curve adjustments. https://www.lifeafterphotoshop.com/lightroomtone-curves-explained-tone-curve-vs-point-curve-vs-target-curve/. [Online; accessed 24-January-2023].
26. Hailong Ma, Sibo Feng, Xi Xiao, Chenyu Dong, and Xingyue Cheng. 2022. Cascade Luminance and Chrominance for Image Retouching: More Like Artist. arXiv preprint arXiv:2205.15999 (2022).
27. Rafał Mantiuk, Scott Daly, and Louis Kerofsky. 2008. Display adaptive tone mapping. In ACM SIGGRAPH 2008 papers. 1–10.
28. Dan Margulis. 2022. Wiki (Dan Margulis). (2022). https://en.wikipedia.org/wiki/Dan_Margulis#cite_note-1
29. S Mahdi H Miangoleh, Sebastian Dille, Long Mai, Sylvain Paris, and Yagiz Aksoy. 2021. Boosting monocular depth estimation models to high-resolution via content-adaptive multi-resolution merging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9685–9694.
30. Jianchao Tan, Jose Echevarria, and Yotam Gingold. 2018a. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–10.
31. Jianchao Tan, Jose Echevarria, and Yotam Gingold. 2018b. Palette-based image decomposition, harmonization, and color transfer. arXiv preprint arXiv:1804.01225 (2018).
32. Jianchao Tan, Jyh Ming Lien, and Yotam Gingold. 2016. Decomposing Images into Layers via RGB-Space Geometry. ACM Transactions on Graphics 36, 1 (2016), 1–14.
33. Yael Vinker, Inbar Huberman-Spiegelglas, and Raanan Fattal. 2021. Unpaired learning for high dynamic range image tone mapping. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14657–14666.
34. Yili Wang, Yifan Liu, and Kun Xu. 2019. An Improved Geometric Approach for Palette-based Image Decomposition and Recoloring. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 11–22.
35. Qing Zhang, Yongwei Nie, Lei Zhu, Chunxia Xiao, and Wei-Shi Zheng. 2021a. A Blind Color Separation Model for Faithful Palette-based Image Recoloring. IEEE Transactions on Multimedia (2021), 1–1.
36. Q. Zhang, Y. Nie, L. Zhu, C. Xiao, and W.-S. Zheng. 2021b. A Blind Color Separation Model for Faithful Palette-based Image Recoloring. IEEE Transactions on Multimedia (2021), 1–1. Conference Name: IEEE Transactions on Multimedia.
