“Pyramid Texture Filtering” by Zhang, Jiang, Nie and Zheng
Conference:
Type:
Title:
- Pyramid Texture Filtering
Session/Category Title: Material Rendering
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present a simple but effective technique to smooth out textures while preserving the prominent structures. Our method is built upon a key observation—the coarsest level in a Gaussian pyramid often naturally eliminates textures and summarizes the main image structures. This inspires our central idea for texture filtering, which is to progressively upsample the very low-resolution coarsest Gaussian pyramid level to a full-resolution texture smoothing result with well-preserved structures, under the guidance of each fine-scale Gaussian pyramid level and its associated Laplacian pyramid level. We show that our approach is effective to separate structure from texture of different scales, local contrasts, and forms, without degrading structures or introducing visual artifacts. We also demonstrate the applicability of our method on various applications including detail enhancement, image abstraction, HDR tone mapping, inverse halftoning, and LDR image enhancement. Code is available at https://rewindl.github.io/pyramid_texture_filtering/.
References:
1. Linchao Bao, Yibing Song, Qingxiong Yang, Hao Yuan, and Gang Wang. 2013. Tree filtering: Efficient structure-preserving smoothing with a minimum spanning tree. IEEE Transactions on Image Processing 23, 2 (2013), 555–569.
2. Sai Bi, Xiaoguang Han, and Yizhou Yu. 2015. An L1 image transform for edge-preserving smoothing and scene-level intrinsic decomposition. ACM Transactions on Graphics 34, 4 (2015), 1–12.
3. Peter J Burt and Edward H Adelson. 1983. The Laplacian Pyramid as a Compact Image Code. IEEE Transactions on Communications 3, 4 (1983).
4. Jiawen Chen, Sylvain Paris, and Frédo Durand. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Transactions on Graphics 26, 3 (2007), 103.
5. Hojin Cho, Hyunjoon Lee, Henry Kang, and Seungyong Lee. 2014. Bilateral texture filtering. ACM Transactions on Graphics 33, 4 (2014), 1–8.
6. Antonio Criminisi, Toby Sharp, Carsten Rother, and Patrick Pérez. 2010. Geodesic image and video editing. ACM Transactions on Graphics 29, 5 (2010), 134–1.
7. Hui Du, Xiaogang Jin, and Philip J Willis. 2016. Two-level joint local laplacian texture filtering. The Visual Computer 32 (2016), 1537–1548.
8. Frédo Durand and Julie Dorsey. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Transactions on Graphics 21, 3 (2002), 257–266.
9. Elmar Eisemann and Frédo Durand. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (2004), 673–678.
10. Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, and Xin Tong. 2018. Image smoothing via unsupervised learning. ACM Transactions on Graphics 37, 6 (2018), 1–14.
11. Zeev Farbman, Raanan Fattal, Dani Lischinski, and Richard Szeliski. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Transactions on Graphics 27, 3 (2008), 1–10.
12. Raanan Fattal. 2009. Edge-avoiding wavelets and their applications. ACM Transactions on Graphics 28, 3 (2009), 1–10.
13. Eduardo SL Gastal and Manuel M Oliveira. 2011. Domain transform for edge-aware image and video processing. ACM Transactions on Graphics 30, 4 (2011), 1–12.
14. Eduardo SL Gastal and Manuel M Oliveira. 2012. Adaptive manifolds for real-time high-dimensional filtering. ACM Transactions on Graphics 31, 4 (2012), 1–13.
15. Michaël Gharbi, Jiawen Chen, Jonathan T Barron, Samuel W Hasinoff, and Frédo Durand. 2017. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics 36, 4 (2017), 1–12.
16. Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing 26, 2 (2016), 982–993.
17. Bumsub Ham, Minsu Cho, and Jean Ponce. 2015. Robust image filtering using joint static and dynamic guidance. In CVPR. 4823–4831.
18. Junho Jeon, Hyunjoon Lee, Henry Kang, and Seungyong Lee. 2016. Scale-aware structure-preserving texture filtering. Computer Graphics Forum 35, 7 (2016), 77–86.
19. Levent Karacan, Erkut Erdem, and Aykut Erdem. 2013. Structure-preserving image smoothing via region covariances. ACM Transactions on Graphics 32, 6 (2013), 1–11.
20. Michael Kass and Justin Solomon. 2010. Smoothed local histogram filters. ACM Transactions on Graphics 29, 4 (2010), 1–10.
21. Youngjung Kim, Bumsub Ham, Minh N Do, and Kwanghoon Sohn. 2018. Structure-texture image decomposition using deep variational priors. IEEE Transactions on Image Processing 28, 6 (2018), 2692–2704.
22. Johannes Kopf, Michael F Cohen, Dani Lischinski, and Matt Uyttendaele. 2007. Joint bilateral upsampling. ACM Transactions on Graphics 26, 3 (2007), 96.
23. Sifei Liu, Jinshan Pan, and Ming-Hsuan Yang. 2016. Learning recursive filters for low-level vision via a hybrid neural network. In ECCV. 560–576.
24. Wei Liu, Xiaogang Chen, Chuanhua Shen, Zhi Liu, and Jie Yang. 2017. Semi-global weighted least squares in image filtering. In ICCV. 5861–5869.
25. Wei Liu, Pingping Zhang, Xiaolin Huang, Jie Yang, Chunhua Shen, and Ian Reid. 2020. Real-time image smoothing via iterative least squares. ACM Transactions on Graphics 39, 3 (2020), 1–24.
26. Wei Liu, Pingping Zhang, Yinjie Lei, Xiaolin Huang, Jie Yang, and Michael Kwok-Po Ng. 2021. A generalized framework for edge-preserving and structure-preserving image smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
27. Kaiyue Lu, Shaodi You, and Nick Barnes. 2018. Deep texture and structure aware filtering network for image smoothing. In ECCV. 217–233.
28. Dongbo Min, Sunghwan Choi, Jiangbo Lu, Bumsub Ham, Kwanghoon Sohn, and Minh N Do. 2014. Fast global image smoothing based on weighted least squares. IEEE Transactions on Image Processing 23, 12 (2014), 5638–5653.
29. Sylvain Paris and Frédo Durand. 2006. A fast approximation of the bilateral filter using a signal processing approach. In ECCV. 568–580.
30. Sylvain Paris, Samuel W Hasinoff, and Jan Kautz. 2011. Local laplacian filters: edge-aware image processing with a laplacian pyramid. ACM Transactions on Graphics 30, 4 (2011), 68.
31. Pietro Perona and Jitendra Malik. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 7 (1990), 629–639.
32. Georg Petschnigg, Richard Szeliski, Maneesh Agrawala, Michael Cohen, Hugues Hoppe, and Kentaro Toyama. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (2004), 664–672.
33. Kartic Subr, Cyril Soler, and Frédo Durand. 2009. Edge-preserving multiscale image decomposition based on local extrema. ACM Transactions on Graphics 28, 5 (2009), 1–9.
34. Carlo Tomasi and Roberto Manduchi. 1998. Bilateral Filtering for Gray and Color Images. In ICCV. 839–839.
35. Xing Wei, Qingxiong Yang, and Yihong Gong. 2018. Joint contour filtering. International Journal of Computer Vision 126, 11 (2018), 1245–1265.
36. Ben Weiss. 2006. Fast median and bilateral filtering. ACM Transactions on Graphics 25, 3 (2006), 519–526.
37. Li Xu, Cewu Lu, Yi Xu, and Jiaya Jia. 2011. Image smoothing via L0 gradient minimization. ACM Transactions on Graphics 30, 6 (2011), 1–12.
38. Li Xu, Jimmy Ren, Qiong Yan, Renjie Liao, and Jiaya Jia. 2015. Deep edge-aware filters. In ICML. 1669–1678.
39. Li Xu, Qiong Yan, Yang Xia, and Jiaya Jia. 2012. Structure extraction from texture via relative total variation. ACM Transactions on Graphics 31, 6 (2012), 1–10.
40. Feihu Zhang, Longquan Dai, Shiming Xiang, and Xiaopeng Zhang. 2015. Segment graph based image filtering: fast structure-preserving smoothing. In ICCV. 361–369.
41. Qi Zhang, Xiaoyong Shen, Li Xu, and Jiaya Jia. 2014. Rolling guidance filter. In ECCV. 815–830.
42. Lei Zhu, Chi-Wing Fu, Yueming Jin, Mingqiang Wei, Jing Qin, and Pheng-Ann Heng. 2016. Non-Local Sparse and Low-Rank Regularization for Structure-Preserving Image Smoothing. Computer Graphics Forum 35, 7 (2016), 217–226.
43. Maria Zontak, Inbar Mosseri, and Michal Irani. 2013. Separating signal from noise using patch recurrence across scales. In CVPR. 1195–1202.
