“Texture optimization for example-based synthesis” by Bobick, Kwatra, Essa and Kwatra

  • ©Aaron Bobick, Vivek Kwatra, Irfan Essa, and Nipun Kwatra




    Texture optimization for example-based synthesis



    We present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the synthesis problem as minimization of an energy function, which is optimized using an Expectation Maximization (EM)-like algorithm. In contrast to most example-based techniques that do region-growing, ours is a joint optimization approach that progressively refines the entire texture. Additionally, our approach is ideally suited to allow for controllable synthesis of textures. Specifically, we demonstrate controllability by animating image textures using flow fields. We allow for general two-dimensional flow fields that may dynamically change over time. Applications of this technique include dynamic texturing of fluid animations and texture-based flow visualization.


    1. Ashikhmin, M. 2001. Synthesizing natural textures. 2001 ACM Symposium on Interactive 3D Graphics (March), 217–226. Google ScholarDigital Library
    2. Bhat, K. S., Seitz, S. M., Hodgins, J. K., and Khosla, P. K. 2004. Flow-based video synthesis and editing. ACM Transactions on Graphics (SIGGRAPH 2004) 23, 3 (August). Google ScholarDigital Library
    3. Bregler, C., Covell, M., and Slaney, M. 1997. Video rewrite: Driving visual speech with audio. Proceedings of SIGGRAPH 97 (August), 353–360. ISBN 0-89791-896-7. Held in Los Angeles, California. Google ScholarDigital Library
    4. Cohen, M. F., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 287–294. Google ScholarDigital Library
    5. Coleman, D., Holland, P., Kaden, N., Klema, V., and Peters, S. C. 1980. A system of subroutines for iteratively reweighted least squares computations. ACM Trans. Math. Softw. 6, 3, 327–336. Google ScholarDigital Library
    6. DeBonet, J. S. 1997. Multiresolution sampling procedure for analysis and synthesis of texture images. Proceedings of ACM SIGGRAPH 97 (August), 361–368. Google ScholarDigital Library
    7. Dellaert, F., Kwatra, V., and Oh, S. M. 2005. Mixture trees for modeling and fast conditional sampling with applications in vision and graphics. In IEEE Computer Vision and Pattern Recognition. Google ScholarDigital Library
    8. Doretto, G., and Soatto, S. 2003. Editable dynamic textures. In IEEE Computer Vision and Pattern Recognition, II: 137–142.Google Scholar
    9. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001, 341–346. Google ScholarDigital Library
    10. Efros, A., and Leung, T. 1999. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision, 1033–1038. Google ScholarDigital Library
    11. Elkan, C. 2003. Using the triangle inequality to accelerate k-means. In International Conference on Machine Learning.Google Scholar
    12. Ezzat, T., Geiger, G., and Poggio, T. 2002. Trainable videorealistic speech animation. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM Press, 388–398. Google ScholarDigital Library
    13. Fitzgibbon, A., Wexler, Y., and Zisserman, A. 2003. Image-based rendering using image-based priors. In International Conference on Computer Vision. Google ScholarDigital Library
    14. Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 2, 56–65. Google ScholarDigital Library
    15. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. Proceedings of ACM SIGGRAPH 95 (August), 229–238. Google ScholarDigital Library
    16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. Proceedings of SIGGRAPH 2001 (August), 327–340. ISBN 1-58113-292-1. Google ScholarDigital Library
    17. Johnson, S. C. 1967. Hierarchical clustering schemes. Psychometrika 2, 241–254.Google ScholarCross Ref
    18. Jojic, N., Frey, B., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In International Conference on Computer Vision. Google ScholarDigital Library
    19. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3 (July), 277–286. Google ScholarDigital Library
    20. Liang, L., Liu, C., Xu, Y.-Q., Guo, B., and Shum, H.-Y. 2001. Real-time texture synthesis by patch-based sampling. ACM Transactions on Graphics Vol. 20, No. 3 (July), 127–150. Google ScholarDigital Library
    21. McLachlan, G., and Krishnan, T. 1997. The EM algorithm and extensions. Wiley series in probability and statistics. John Wiley & Sons.Google Scholar
    22. Neyret, F. 2003. Advected textures. Symposium on Computer Animation’03 (July). Google ScholarDigital Library
    23. Paget, R., and Longstaff, I. D. 1998. Texture synthesis via a non-causal nonparametric multiscale markov random field. IEEE Transactions on Image Processing 7, 6 (June), 925–931. Google ScholarDigital Library
    24. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 313–318. Google ScholarDigital Library
    25. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40, 1 (October), 49–70. Google ScholarDigital Library
    26. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. Proceedings of ACM SIGGRAPH 2000 (July), 489–498. Google ScholarDigital Library
    27. Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. Proceedings of ACM SIGGRAPH 2000 (July), 479–488. ISBN 1-58113-208-5. Google ScholarDigital Library
    28. Wei, L.-Y., and Levoy, M. 2002. Order-independent texture synthesis. Tech. Rep. TR-2002-01, Stanford University CS Department.Google Scholar
    29. Wexler, Y., Shechtman, E., and Irani, M. 2004. Space-time video completion. In CVPR 2004, 120–127.Google Scholar
    30. Wu, Q., and Yu, Y. 2004. Feature matching and deformation for texture synthesis. ACM Transactions on Graphics (SIGGRAPH 2004) (August). Google ScholarDigital Library
    31. Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. ACM Transactions on Graphics, SIGGRAPH 2003 22, 3, 295–302. Google ScholarDigital Library
    32. Zhang, E., Mischaikow, K., and Turk, G. 2004. Vector field design on surfaces. Tech. Rep. 04–16, Georgia Institute of Technology.Google Scholar

ACM Digital Library Publication: