“A sparse control model for image and video editing”
Conference:
Type(s):
Title:
- A sparse control model for image and video editing
Session/Category Title: Image & Video Editing
Presenter(s)/Author(s):
Abstract:
It is common that users draw strokes, as control samples, to modify color, structure, or tone of a picture. We discover inherent limitation of existing methods for their implicit requirement on where and how the strokes are drawn, and present a new system that is principled on minimizing the amount of work put in user interaction. Our method automatically determines the influence of edit samples across the whole image jointly considering spatial distance, sample location, and appearance. It greatly reduces the number of samples that are needed, while allowing for a decent level of global and local manipulation of resulting effects and reducing propagation ambiguity. Our method is broadly beneficial to applications adjusting visual content.
References:
1. Adams, A., Gelfand, N., Dolson, J., and Levoy, M. 2009. Gaussian kd-trees for fast high-dimensional filtering. ACM Trans. Graph. 28, 3.
2. Adams, A., Baek, J., and Davis, M. A. 2010. Fast high-dimensional filtering using the permutohedral lattice. Comput. Graph. Forum 29, 2, 753–762.
3. An, X., and Pellacini, F. 2008. Appprop: all-pairs appearance-space edit propagation. ACM Trans. Graph. 27, 3.
4. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. 25, 3, 637–645.
5. Bousseau, A., Paris, S., and Durand, F. 2009. User-assisted intrinsic images. ACM Trans. Graph. 28, 5.
6. Boyadzhiev, I., Bala, K., Paris, S., and Durand, F. 2012. User-guided white balance for mixed lighting conditions. ACM Trans. Graph. 31, 6, 200.
7. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 3, 103.
8. Chen, Q., Li, D., and Tang, C.-K. 2012. Knn matting. In CVPR, 869–876.
9. Chen, X., Zou, D., Zhao, Q., and Tan, P. 2012. Manifold preserving edit propagation. ACM Trans. Graph. 31, 6, 132.
10. Criminisi, A., Sharp, T., Rother, C., and Pérez, P. 2010. Geodesic image and video editing. ACM Trans. Graph. 29, 5, 134.
11. Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21, 3, 257–266.
12. Farbman, Z., Fattal, R., Lischinski, D., and Szeliski, R. 2008. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. 27, 3.
13. Farbman, Z., Fattal, R., and Lischinski, D. 2010. Diffusion maps for edge-aware image editing. ACM Trans. Graph. 29, 6, 145.
14. Fattal, R., Agrawala, M., and Rusinkiewicz, S. 2007. Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26, 3, 51.
15. Fattal, R. 2009. Edge-avoiding wavelets and their applications. ACM Trans. Graph. 28, 3.
16. Fowlkes, C., Belongie, S., Chung, F. R. K., and Malik, J. 2004. Spectral grouping using the nyström method. IEEE Trans. Pattern Anal. Mach. Intell. 26, 2, 214–225.
17. Gastal, E. S. L., and Oliveira, M. M. 2011. Domain transform for edge-aware image and video processing. ACM Trans. Graph. 30, 4, 69.
18. Gastal, E. S. L., and Oliveira, M. M. 2012. Adaptive manifolds for real-time high-dimensional filtering. ACM Trans. Graph. 31, 4, 33.
19. Kass, M., and Solomon, J. 2010. Smoothed local histogram filters. ACM Trans. Graph. 29, 4.
20. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Trans. Graph. 26, 3, 96.
21. Krähenbühl, P., and Koltun, V. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. In NIPS.
22. Lang, M., Wang, O., Aydin, T., Smolic, A., and Gross, M. H. 2012. Practical temporal consistency for image-based graphics applications. ACM Trans. Graph. 31, 4, 34.
23. Lee, P., and Wu, Y. 2011. Nonlocal matting. In CVPR, 2193–2200.
24. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689–694.
25. Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2, 228–242.
26. Levin, A., Rav-Acha, A., and Lischinski, D. 2008. Spectral matting. IEEE Trans. Pattern Anal. Mach. Intell. 30, 10, 1699–1712.
27. Li, Y., Adelson, E. H., and Agarwala, A. 2008. Scribble-boost: Adding classification to edge-aware interpolation of local image and video adjustments. Comput. Graph. Forum 27, 4, 1255–1264.
28. Li, Y., Ju, T., and Hu, S.-M. 2010. Instant propagation of sparse edits on images and videos. Comput. Graph. Forum 29, 7, 2049–2054.
29. Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3, 646–653.
30. Liu, R., Li, Z., and Jia, J. 2008. Image partial blur detection and classification. In CVPR.
31. Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. In ECCV (4), 568–580.
32. Paris, S., Hasinoff, S. W., and Kautz, J. 2011. Local laplacian filters: Edge-aware image processing with a laplacian pyramid. ACM Trans. Graph. 30, 4, 68.
33. Pellacini, F., and Lawrence, J. 2007. Appwand: editing measured materials using appearance-driven optimization. ACM Trans. Graph. 26, 3, 54.
34. Ragan-Kelley, J., Adams, A., Paris, S., Levoy, M., Amarasinghe, S., and Durand, F. 2012. Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Trans. Graph. 31, 4, 32.
35. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., and Rott, P. 2009. A perceptually motivated online benchmark for image matting. In CVPR, 1826–1833.
36. Rother, C., Kolmogorov, V., and Blake, A. 2004. “grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 3, 309–314.
37. Shepard, D. 1968. A two-dimensional interpolation function for irregularly-spaced data. In ACM national conference, 517–524.
38. Subr, K., Soler, C., and Durand, F. 2009. Edge-preserving multiscale image decomposition based on local extrema. ACM Trans. Graph. 28, 5.
39. Wang, J., and Cohen, M. F. 2005. An iterative optimization approach for unified image segmentation and matting. In ICCV, 936–943.
40. Wang, J., and Cohen, M. F. 2007. Optimized color sampling for robust matting. In CVPR.
41. Xu, K., Li, Y., Ju, T., Hu, S.-M., and Liu, T.-Q. 2009. Efficient affinity-based edit propagation using k-d tree. ACM Trans. Graph. 28, 5.
42. Xu, K., Wang, J., Tong, X., Hu, S.-M., and Guo, B. 2009. Edit propagation on bidirectional texture functions. Comput. Graph. Forum 28, 7.
43. Xu, L., Lu, C., Xu, Y., and Jia, J. 2011. Image smoothing via L0 gradient minimization. ACM Trans. Graph. 30, 6.
44. Xu, L., Yan, Q., Xia, Y., and Jia, J. 2012. Structure extraction from texture via relative total variation. ACM Trans. Graph. 31, 6, 139.
