“Texture optimization for example-based synthesis” by Bobick, Kwatra, Essa and Kwatra

We present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the synthesis problem as minimization of an energy function, which is optimized using an Expectation Maximization (EM)-like algorithm. In contrast to most example-based techniques that do region-growing, ours is a joint optimization approach that progressively refines the entire texture. Additionally, our approach is ideally suited to allow for controllable synthesis of textures. Specifically, we demonstrate controllability by animating image textures using flow fields. We allow for general two-dimensional flow fields that may dynamically change over time. Applications of this technique include dynamic texturing of fluid animations and texture-based flow visualization.

1. Ashikhmin, M. 2001. Synthesizing natural textures. 2001 ACM Symposium on Interactive 3D Graphics (March), 217–226. Google ScholarDigital Library
2. Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3 (August). Google ScholarDigital Library
3. Bregler, C., Covell, M., and Slaney, M. 1997. Video rewrite: Driving visual speech with audio. Proceedings of SIGGRAPH 97 (August), 353–360. ISBN 0-89791-896-7. Held in Los Angeles, California. Google ScholarDigital Library
4. Cohen, M. F., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 287–294. Google ScholarDigital Library
5. Coleman, D., Holland, P., Kaden, N., Klema, V., and Peters, S. C. 1980. A system of subroutines for iteratively reweighted least squares computations. ACM Trans. Math. Softw. 6, 3, 327–336. Google ScholarDigital Library
6. DeBonet, J. S. 1997. Multiresolution sampling procedure for analysis and synthesis of texture images. Proceedings of ACM SIGGRAPH 97 (August), 361–368. Google ScholarDigital Library
7. Dellaert, F., Kwatra, V., and Oh, S. M. 2005. Mixture trees for modeling and fast conditional sampling with applications in vision and graphics. In IEEE Computer Vision and Pattern Recognition. Google ScholarDigital Library
8. Doretto, G., and Soatto, S. 2003. Editable dynamic textures. In IEEE Computer Vision and Pattern Recognition, II: 137–142.Google Scholar
9. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001, 341–346. Google ScholarDigital Library
10. Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision, 1033–1038. Google ScholarDigital Library
11. Elkan, C. 2003. Using the triangle inequality to accelerate k-means. In International Conference on Machine Learning.Google Scholar
12. Ezzat, T., Geiger, G., and Poggio, T. 2002. Trainable videorealistic speech animation. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM Press, 388–398. Google ScholarDigital Library
13. Fitzgibbon, A., Wexler, Y., and Zisserman, A. 2003. Image-based rendering using image-based priors. In International Conference on Computer Vision. Google ScholarDigital Library
14. Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 2, 56–65. Google ScholarDigital Library
15. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. Proceedings of ACM SIGGRAPH 95 (August), 229–238. Google ScholarDigital Library
16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. Proceedings of SIGGRAPH 2001 (August), 327–340. ISBN 1-58113-292-1. Google ScholarDigital Library
17. Johnson, S. C. 1967. Hierarchical clustering schemes. Psychometrika 2, 241–254.Google ScholarCross Ref
18. Jojic, N., Frey, B., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In International Conference on Computer Vision. Google ScholarDigital Library
19. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3 (July), 277–286. Google ScholarDigital Library
20. Liang, L., Liu, C., Xu, Y.-Q., Guo, B., and Shum, H.-Y. 2001. Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics Vol. 20, No. 3 (July), 127–150. Google ScholarDigital Library
21. McLachlan, G., and Krishnan, T. 1997. The EM algorithm and extensions. Wiley series in probability and statistics. John Wiley & Sons.Google Scholar
22. Neyret, F. 2003. Advected textures. Symposium on Computer Animation’03 (July). Google ScholarDigital Library
23. Paget, R., and Longstaff, I. D. 1998. Texture synthesis via a non-causal nonparametric multiscale markov random field. IEEE Transactions on Image Processing 7, 6 (June), 925–931. Google ScholarDigital Library
24. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 313–318. Google ScholarDigital Library
25. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40, 1 (October), 49–70. Google ScholarDigital Library
26. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. Proceedings of ACM SIGGRAPH 2000 (July), 489–498. Google ScholarDigital Library
27. Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. Proceedings of ACM SIGGRAPH 2000 (July), 479–488. ISBN 1-58113-208-5. Google ScholarDigital Library
28. Wei, L.-Y., and Levoy, M. 2002. Order-independent texture synthesis. Tech. Rep. TR-2002-01, Stanford University CS Department.Google Scholar
29. Wexler, Y., Shechtman, E., and Irani, M. 2004. Space-time video completion. In CVPR 2004, 120–127.Google Scholar
30. Wu, Q., and Yu, Y. 2004. Feature matching and deformation for texture synthesis. ACM Transactions on Graphics (SIGGRAPH 2004) (August). Google ScholarDigital Library
31. Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 295–302. Google ScholarDigital Library
32. Zhang, E., Mischaikow, K., and Turk, G. 2004. Vector field design on surfaces. Tech. Rep. 04–16, Georgia Institute of Technology.Google Scholar