“Multi-scale label-map extraction for texture synthesis”
Conference:
Type(s):
Title:
- Multi-scale label-map extraction for texture synthesis
Session/Category Title: TEXTURE
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Texture synthesis is a well-established area, with many important applications in computer graphics and vision. However, despite their success, synthesis techniques are not used widely in practice because the creation of good exemplars remains challenging and extremely tedious. In this paper, we introduce an unsupervised method for analyzing texture content across multiple scales that automatically extracts good exemplars from natural images. Unlike existing methods, which require extensive manual tuning, our method is fully automatic. This allows the user to focus on using texture palettes derived from their own images, rather than on manual interactions dictated by the needs of an underlying algorithm.Most natural textures exhibit patterns at multiple scales that may vary according to the location (non-stationarity). To handle such textures many synthesis algorithms rely on an analysis of the input and a guidance of the synthesis. Our new analysis is based on a labeling of texture patterns that is both (i) multi-scale and (ii) unsupervised — that is, patterns are labeled at multiple scales, and the scales and the number of labeled clusters are selected automatically. Our method works in two stages. The first builds a hierarchical extension of superpixels and the second labels the superpixels based on random walk in a graph of similarity between superpixels and a nonnegative matrix factorization. Our label-maps provide descriptors for pixels and regions that benefit state-of-the-art texture synthesis algorithms. We show several applications including guidance of non-stationary synthesis, content selection and texture painting. Our method is designed to treat large inputs and can scale to many megapixels. In addition to traditional exemplar inputs, our method can also handle natural images containing different textured regions.
References:
1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Susstrunk, S. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. Pattern Analysis and Machine Intelligence, IEEE Transactions on 34, 11 (November), 2274–2282. Google ScholarDigital Library
2. Arbeláez, P., Maire, M., Fowlkes, C., and Malik, J. 2011. Contour detection and hierarchical image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on 33, 5 (May), 898–916. Google ScholarDigital Library
3. Ashikhmin, M. 2001. Synthesizing natural textures. In Proceedings of the 2001 Symposium on Interactive 3D Graphics, ACM, I3D ’01, 217–226. Google ScholarDigital Library
4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (July), 24:1–24:11. Google ScholarDigital Library
5. Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., and Plemmons, R. J. 2007. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis 52, 1, 155–173.Google ScholarCross Ref
6. Cohen, M. F., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. ACM Trans. Graph. 22, 3 (July), 287–294. Google ScholarDigital Library
7. Dai, D., Riemenschneider, H., and Van Gool, L. 2014. The synthesizability of texture examples. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3027–3034. Google ScholarDigital Library
8. Diamanti, O., Barnes, C., Paris, S., Shechtman, E., and Sorkine-Hornung, O. 2015. Synthesis of complex image appearance from limited exemplars. ACM Trans. Graph. 34, 2 (Mar.), 22:1–22:14. Google ScholarDigital Library
9. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, ACM, SIGGRAPH ’01, 341–346. Google ScholarDigital Library
10. Gillis, N. 2011. Nonnegative Matrix Factorization: Complexity, Algorithms and Applications. PhD thesis, Université Catholique de Louvain, École Polytechnique de Louvain.Google Scholar
11. Gillis, N. 2014. The why and how of nonnegative matrix factorization. In Regularization, Optimization, Kernels, and Support Vector Machines. Chapman & Hall/CRC, 257–291.Google Scholar
12. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, ACM, SIGGRAPH ’01, 327–340. Google ScholarDigital Library
13. Kaspar, A., Neubert, B., Lischinski, D., Pauly, M., and Kopf, J. 2015. Self tuning texture optimization. Computer Graphics Forum 34, 2, 349–359. Google ScholarDigital Library
14. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. In ACM SIGGRAPH 2003 Papers, ACM, SIGGRAPH ’03, 277–286. Google ScholarDigital Library
15. Lefebvre, S., and Hoppe, H. 2006. Appearance-space texture synthesis. In ACM SIGGRAPH 2006 Papers, ACM, SIGGRAPH ’06, 541–548. Google ScholarDigital Library
16. Lockerman, Y. D., Xue, S., Dorsey, J., and Rushmeier, H. 2013. Creating texture exemplars from unconstrained images. In Proceedings of the 2013 International Conference on Computer-Aided Design and Computer Graphics, IEEE Computer Society, CADGRAPHICS ’13, 397–398. Google ScholarDigital Library
17. Lu, J., Dorsey, J., and Rushmeier, H. 2009. Dominant texture and diffusion distance manifolds. Computer Graphics Forum 28, 2, 667–676.Google ScholarCross Ref
18. Lu, J., Garr-Schultz, A., Dorsey, J., and Rushmeier, H. 2009. A psychophysical study of dominant texture detection. In Proceedings of the 6th Symposium on Applied Perception in Graphics and Visualization, ACM, APGV ’09, 133–133. Google ScholarDigital Library
19. Mertens, T., Kautz, J., Chen, J., Bekaert, P., and Durand, F. 2006. Texture transfer using geometry correlation. In Proceedings of the 17th Eurographics Conference on Rendering Techniques, Eurographics Association, EGSR ’06, 273–284. Google ScholarDigital Library
20. Mount, D. M. 2010. ANN programming manual. Tech. rep., University of Maryland, College Park, Maryland.Google Scholar
21. Narain, R., Kwatra, V., Lee, H.-P., Kim, T., Carlson, M., and Lin, M. C. 2007. Feature-guided dynamic texture synthesis on continuous flows. In Proceedings of the 18th Eurographics Conference on Rendering Techniques, Eurographics Association, EGSR’07, 361–370. Google ScholarDigital Library
22. Rosenberger, A., Cohen-Or, D., and Lischinski, D. 2009. Layered shape synthesis: Automatic generation of control maps for non-stationary textures. ACM Trans. Graph. 28, 5 (dec), 107:1–107:9. Google ScholarDigital Library
23. Vanhoey, K., Sauvage, B., Larue, F., and Dischler, J.-M. 2013. On-the-fly multi-scale infinite texturing from example. ACM Trans. Graph. 32, 6, 208. Google ScholarDigital Library
24. Wang, W., and Hua, M. 2013. Extracting dominant textures in real time with multi-scale hue-saturation-intensity histograms. Image Processing, IEEE Transactions on 22, 11, 4237–4248. Google ScholarDigital Library
25. Wang, B., and Tu, Z. 2012. Affinity learning via self-diffusion for image segmentation and clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2312–2319. Google ScholarDigital Library
26. Wu, F., Dong, W., Kong, Y., Mei, X., Yan, D.-M., Zhang, X., and Paul, J.-C. 2016. Feature-aware natural texture synthesis. The Visual Computer 32, 1, 43–55. Google ScholarDigital Library
27. Zalesny, A., Ferrari, V., Caenen, G., and Van Gool, L. 2005. Composite texture synthesis. International Journal of Computer Vision 62, 1-2, 161–176. Google ScholarDigital Library
28. Zelnik-Manor, L., and Perona, P. 2004. Self-tuning spectral clustering. In Advances in neural information processing systems, 1601–1608.Google Scholar
29. Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. ACM Trans. Graph. 22, 3 (July), 295–302. Google ScholarDigital Library
