“Bilateral guided upsampling” by Chen, Adams, Wadhwa and Hasinoff
Conference:
Type(s):
Title:
- Bilateral guided upsampling
Session/Category Title: Filtering Images
Presenter(s)/Author(s):
Abstract:
We present an algorithm to accelerate a large class of image processing operators. Given a low-resolution reference input and output pair, we model the operator by fitting local curves that map the input to the output. We can then produce a full-resolution output by evaluating these low-resolution curves on the full-resolution input. We demonstrate that this faithfully models state-of-the-art operators for tone mapping, style transfer, and recoloring. The curves are computed by lifting the input into a bilateral grid and then solving for the 3D array of affine matrices that best maps input color to output color per x, y, intensity bin. We enforce a smoothness term on the matrices which prevents false edges and noise amplification. We can either globally optimize this energy, or quickly approximate a solution by locally fitting matrices and then enforcing smoothness by blurring in grid space. This latter option reduces to joint bilateral upsampling [Kopf et al. 2007] or the guided filter [He et al. 2013], depending on the choice of parameters. The cost of running the algorithm is reduced to the cost of running the original algorithm at greatly reduced resolution, as fitting the curves takes about 10 ms on mobile devices, and 1–2 ms on desktop CPUs, and evaluating the curves can be done with a simple GPU shader.
References:
1. Adams, A., Baek, J., and Davis, M. A. 2010. Fast high-dimensional filtering using the permutohedral lattice. In Computer Graphics Forum, vol. 29, 753–762. Cross Ref
2. Adams, A. 2011. High-dimensional gaussian filtering for computational photography. PhD thesis, Stanford University.
3. Aubry, M., Paris, S., Hasinoff, S. W., Kautz, J., and Durand, F. 2014. Fast local Laplacian filters: theory and applications. ACM Trans. Graph. 33, 5 (Sept.), 167:1–167:14.
4. Barron, J. T., and Poole, B. 2016. The fast bilateral solver. ECCV.
5. Barron, J. T., Adams, A., Shih, Y., and Hernandez, C. 2015. Fast bilateral-space stereo for synthetic defocus. CVPR.
6. Bousseau, A., Paris, S., and Durand, F. 2009. User-assisted intrinsic images. ACM Trans. Graph. 28, 5 (Dec.), 130:1–130:10.
7. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 3.
8. Chen, Q., Li, D., and Tang, C.-K. 2013. KNN matting. IEEE TPAMI 35, 9, 2175–2188.
9. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. 27, 3, 67:1–67:10.
10. Farbman, Z., Fattal, R., and Lischinski, D. 2011. Convolution pyramids. ACM Trans. Graph. 30, 6 (Dec.), 175:1–175:8.
11. Gastal, E. S. L., and Oliveira, M. M. 2011. Domain transform for edge-aware image and video processing. ACM Trans. Graph. 30, 4 (July), 69:1–69:12.
12. Gastal, E. S. L., and Oliveira, M. M. 2012. Adaptive manifolds for real-time high-dimensional filtering. ACM Trans. Graph. 31, 4 (July), 33:1–33:13.
13. Gharbi, M., Shih, Y., Chaurasia, G., Ragan-Kelley, J., Paris, S., and Durand, F. 2015. Transform recipes for efficient cloud photo enhancement. ACM Trans. Graph. 34, 6.
14. He, K., and Sun, J. 2015. Fast guided filter. CoRR abs/1505.00996.
15. He, K., Sun, J., and Tang, X. 2013. Guided image filtering. IEEE TPAMI 35, 6, 1397–1409.
16. Jeong, W.-K., Johnson, M. K., Yu, I., Kautz, J., Pfister, H., and Paris, S. 2011. Display-aware image editing. ICCP.
17. Kim, J.-H., Jang, W.-D., Sim, J.-Y., and Kim, C.-S. 2013. Optimized contrast enhancement for real-time image and video dehazing. Vis. Comm. and Image Representation 24, 3, 410–425.
18. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Trans. Graph. 26, 3.
19. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689–694.
20. Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. ECCV.
21. Paris, S., Hasinoff, S. W., and Kautz, J. 2011. Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans. Graph. 30, 4 (July), 68:1–68:12.
22. Ragan-Kelley, J., Barnes, C., Adams, A., Paris, S., Durand, F., and Amarasinghe, S. 2013. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. ACM SIGPLAN 48, 6.
23. Shih, Y., Paris, S., Barnes, C., Freeman, W. T., and Durand, F. 2014. Style transfer for headshot portraits. ACM Trans. Graph. 33, 4 (July), 148:1–148:14.
24. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. IEEE ICCV.
25. Xu, L., Lu, C., Xu, Y., and Jia, J. 2011. Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30, 6, 174:1–174:12.
26. Yuan, L., and Sun, J. 2011. High quality image reconstruction from RAW and JPEG image pair. IEEE ICCV.
