“Deep Correlations for Texture Synthesis” by Cohen-Or and Sendik
Conference:
Type(s):
Title:
- Deep Correlations for Texture Synthesis
Session/Category Title: Image Texture & Completion
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Example-based texture synthesis has been an active research problem for over two decades. Still, synthesizing textures with nonlocal structures remains a challenge. In this article, we present a texture synthesis technique that builds upon convolutional neural networks and extracted statistics of pretrained deep features. We introduce a structural energy, based on correlations among deep features, which capture the self-similarities and regularities characterizing the texture. Specifically, we show that our technique can synthesize textures that have structures of various scales, local and nonlocal, and the combination of the two.
References:
1. M. Aittala, T. Aila, and J. Lehtinen 2016. Reflectance modeling by neural texture synthesis. ACM Transactions on Graphics 35, 4 (2016). Google ScholarDigital Library
2. R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu 1995. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing 16, 5 (1995), 1190–1208. Google ScholarDigital Library
3. S. Darabi, E. Shechtman, C. Barnes, D. B. Goldman, and P. Sen 2012. Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics (TOG) (Proceedings of SIGGRAPH 2012) 31, 4 (2012), 82:1–82:10. Google ScholarDigital Library
4. J. S. De Bonet 1997. Multiresolution sampling procedure for analysis and synthesis of texture images. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. ACM/Addison-Wesley Publishing Co., 361–368. Google ScholarDigital Library
5. A. A. Efros and W. T. Freeman 2001. Image quilting for texture synthesis and transfer. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 341–346. Google ScholarDigital Library
6. A. A. Efros and T. K. Leung 1999. Texture synthesis by non-parametric sampling. In Proceedings of the 7th IEEE International Conference on Computer Vision. Vol. 2. IEEE, 1033–1038. Google ScholarDigital Library
7. B. Galerne, Y. Gousseau, and J.-M. Morel 2011. Random phase textures: Theory and synthesis. IEEE Transactions on Image Processing 20, 1 (2011), 257–267. Google ScholarDigital Library
8. L. Gatys, A. S. Ecker, and M. Bethge 2015. Texture synthesis using convolutional neural networks. In Advances in Neural Information Processing Systems. 262–270. Google ScholarDigital Library
9. D. J. Heeger and J. R. Bergen 1995. Pyramid-based texture analysis/synthesis. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 229–238. Google ScholarDigital Library
10. J. Johnson, A. Alahi, and F. Li 2016. Perceptual losses for real-time style transfer and super-resolution. CoRRabs/1603.08155.Google Scholar
11. A. Kaspar, B. Neubert, D. Lischinski, M. Pauly, and J. Kopf 2015. Self tuning texture optimization. In Computer Graphics Forum. Vol. 34. Wiley Online Library, 349–359. Google ScholarDigital Library
12. V. Kwatra, I. Essa, A. Bobick, and N. Kwatra 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics (ToG) 24, 3 (2005), 795–802. Google ScholarDigital Library
13. S. Lefebvre and H. Hoppe 2005. Parallel controllable texture synthesis. ACM Transactions on Graphics (ToG) 24, 3 (2005), 777–786. Google ScholarDigital Library
14. S. Lefebvre and H. Hoppe 2006. Appearance-space texture synthesis. ACM Transactions on Graphics (TOG) 25, 3 (2006), 541–548. Google ScholarDigital Library
15. C. Li and M. Wand 2016. Combining Markov random fields and convolutional neural networks for image synthesis. arXiv preprint arXiv:1601.04589.Google Scholar
16. G. Liu, Y. Gousseau, and G. Xia 2016. Texture synthesis through convolutional neural networks and spectrum constraints. CoRRabs/1605.01141.Google Scholar
17. Y. Liu, W.-C. Lin, and J. Hays 2004. Near-regular texture analysis and manipulation. ACM Transactions on Graphics (TOG) 23, 3 (Aug. 2004), 368–376. Google ScholarDigital Library
18. J. Portilla and E. P. Simoncelli 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40, 1 (2000), 49–70. Google ScholarDigital Library
19. E. Risser, C. Han, R. Dahyot, and E. Grinspun 2010. Synthesizing structured image hybrids. ACM Transactions on Graphics (TOG). 29, 4 , Article 85 (July (2010). Google ScholarDigital Library
20. K. Simonyan and A. Zisserman 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
21. K. Valkealahti and E. Oja 1998. Reduced multidimensional co-occurrence histograms in texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 1 (1998), 90–94. Google ScholarDigital Library
22. A. Vedaldi and K. Lenc 2014. MatConvNet-convolutional neural networks for matlab. arXiv preprint arXiv:1412.4564.Google Scholar
23. M. Vijfwinkel 2005. Cg texture database. {Online; accessed 22-August-2016}.Google Scholar
24. L.-Y. Wei, S. Lefebvre, V. Kwatra, and G. Turk 2009. State of the art in example-based texture synthesis. In Eurographics 2009, State of the Art Report, EG-STAR. Eurographics Association, 93–117.Google Scholar
25. L.-Y. Wei and M. Levoy 2000. Fast texture synthesis using tree-structured vector quantization. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. ACM/Addison-Wesley Publishing Co., 479–488. Google ScholarDigital Library
26. Y. Wexler, E. Shechtman, and M. Irani 2004. Space-time video completion. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04). Vol. 1. IEEE, I–120.Google ScholarCross Ref
