“Inverse texture synthesis” by Wei, Han, Zhou, Bao, Guo, et al. …

  • ©Li-Yi Wei, Jianwei Han, Kun Zhou, Hujun Bao, Baining Guo, and Heung-Yeung Shum




    Inverse texture synthesis



    The quality and speed of most texture synthesis algorithms depend on a 2D input sample that is small and contains enough texture variations. However, little research exists on how to acquire such sample. For homogeneous patterns this can be achieved via manual cropping, but no adequate solution exists for inhomogeneous or globally varying textures, i.e. patterns that are local but not stationary, such as rusting over an iron statue with appearance conditioned on varying moisture levels.We present inverse texture synthesis to address this issue. Our inverse synthesis runs in the opposite direction with respect to traditional forward synthesis: given a large globally varying texture, our algorithm automatically produces a small texture compaction that best summarizes the original. This small compaction can be used to reconstruct the original texture or to re-synthesize new textures under user-supplied controls. More important, our technique allows real-time synthesis of globally varying textures on a GPU, where the texture memory is usually too small for large textures. We propose an optimization framework for inverse texture synthesis, ensuring that each input region is properly encoded in the output compaction. Our optimization process also automatically computes orientation fields for anisotropic textures containing both low- and high-frequency regions, a situation difficult to handle via existing techniques.


    1. Ashikhmin, M. 2001. Synthesizing natural textures. In Symposium on Interactive 3D graphics, 217–226. Google ScholarDigital Library
    2. Avidan, S., and Shamir, A. 2007. Seam carving for contentaware image resizing. In SIGGRAPH papers, 10. Google ScholarDigital Library
    3. Bargteil, A. W., Sin, F., Michaels, J. E., Goktekin, T. G., and O’Brien, J. F. 2006. A texture synthesis method for liquid animations. In Symposium on Computer animation, 345–351. Google ScholarDigital Library
    4. Brooks, S., and Dodgson, N. 2002. Self-similarity based texture editing. In SIGGRAPH papers, 653–656. Google ScholarDigital Library
    5. Cabral, B., and Leedom, L. C. 1993. Imaging vector fields using line integral convolution. In SIGGRAPH papers, 263–270. Google ScholarDigital Library
    6. Cohen, M. F., Shade, J., Hiller, S., and Deussen, O. 2003. Wang tiles for image and texture generation. In SIGGRAPH papers, 287–294. Google ScholarDigital Library
    7. Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. In SIGGRAPH papers, 303–312. Google ScholarDigital Library
    8. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In SIGGRAPH papers, 341–346. Google ScholarDigital Library
    9. Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV ’99, 1033. Google ScholarDigital Library
    10. Fang, H., and Hart, J. C. 2004. Textureshop: texture synthesis as a photograph editing tool. In SIGGRAPH papers, 354–359. Google ScholarDigital Library
    11. Gersho, A., and Gray, R. M. 1991. Vector quantization and signal compression. Kluwer Academic Publishers. Google ScholarDigital Library
    12. Gu, J., Tu, C.-I., Ramamoorthi, R., Belhumeur, P., Matusik, W., and Nayar, S. 2006. Time-varying surface appearance: acquisition, modeling and rendering. In SIGGRAPH papers, 762–771. Google ScholarDigital Library
    13. Han, J., Zhou, K., Wei, L.-Y., Gong, M., Bao, H., Zhang, X., and Guo, B. 2006. Fast example-based surface texture synthesis via discrete optimization. Vis. Comput. 22, 9, 918–925. Google ScholarDigital Library
    14. Hanrahan, P., and Haeberli, P. 1990. Direct wysiwyg painting and texturing on 3d shapes. In SIGGRAPH papers, 215–223. Google ScholarDigital Library
    15. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In SIGGRAPH papers, 229–238. Google ScholarDigital Library
    16. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In SIGGRAPH papers, 327–340. Google ScholarDigital Library
    17. Jojic, N., Frey, B. J., and Kannan, A. 2003. Epitomic analysis of appearance and shape. In ICCV ’03, 34. Google ScholarDigital Library
    18. Kannan, A., Winn, J., and Rother, C. 2007. Clustering appearance and shape by learning jigsaws. In Advances in Neural Information Processing Systems 19.Google Scholar
    19. Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D., and Wong, T.-T. 2007. Solid texture synthesis from 2d exemplars. In SIGGRAPH papers, 2. Google ScholarDigital Library
    20. Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. In SIGGRAPH papers, 795–802. Google ScholarDigital Library
    21. Kwatra, V., Adalsteinsson, D., Kim, T., Kwatra, N., Carlson, M., and Lin, M. 2007. Texturing fluids. IEEE Trans. Visualization and Computer Graphics 13, 5, 939–952. Google ScholarDigital Library
    22. Lefebvre, S., and Hoppe, H. 2005. Parallel controllable texture synthesis. In SIGGRAPH papers, 777–786. Google ScholarDigital Library
    23. Lefebvre, S., and Hoppe, H. 2006. Appearance-space texture synthesis. In SIGGRAPH papers, 541–548. Google ScholarDigital Library
    24. Leung, T., and Malik, J. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1, 29–44. Google ScholarDigital Library
    25. Liu, Y., Lin, W.-C., and Hays, J. 2004. Near-regular texture analysis and manipulation. In SIGGRAPH papers, 368–376. Google ScholarDigital Library
    26. Lu, J., Georghiades, A. S., Glaser, A., Wu, H., Wei, L.-Y., Guo, B., Dorsey, J., and Rushmeier, H. 2007. Context-aware textures. ACM Trans. Graph. 26, 1, 3. Google ScholarDigital Library
    27. Matusik, W., Zwicker, M., and Durand, F. 2005. Texture design using a simplicial complex of morphable textures. In SIGGRAPH papers, 787–794. Google ScholarDigital Library
    28. Paris, S., Briceno, H. M., and Sillion, F. X. 2004. Capture of hair geometry from multiple images. In SIGGRAPH papers, 712–719. Google ScholarDigital Library
    29. Perona, P., and Malik, J. 1990. Detecting and localizing edges composed of steps, peaks and roofs. In ICCV ’90, 52–57.Google Scholar
    30. Popat, K., and Picard, R. W. 1997. Cluster based probability model and its application to image and texture processing. IEEE Trans. Image Processing 6, 2, 268–284. Google ScholarDigital Library
    31. Portilla, J., and Simoncelli, E. P. 2000. A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vision 40, 1, 49–70. Google ScholarDigital Library
    32. Qin, X., and Yang, Y.-H. 2005. Basic gray level aura matrices: Theory and its application to texture synthesis. In ICCV ’05, 128–135. Google ScholarDigital Library
    33. Ritter, L., Li, W., Curless, B., Agrawala, M., and Salesin, D. 2006. Painting with texture. In Eurographics Symposium on Rendering, 371–376. Google ScholarCross Ref
    34. Tong, X., Zhang, J., Liu, L., Wang, X., Guo, B., and Shum, H.-Y. 2002. Synthesis of bidirectional texture functions on arbitrary surfaces. In SIGGRAPH papers, 665–672. Google ScholarDigital Library
    35. Turk, G. 2001. Texture synthesis on surfaces. In SIGGRAPH papers, 347–354. Google ScholarDigital Library
    36. Wang, J., Tong, X., Lin, S., Pan, M., Wang, C., Bao, H., Guo, B., and Shum, H.-Y. 2006. Appearance manifolds for modeling time-variant appearance of materials. In SIGGRAPH papers, 754–761. Google ScholarDigital Library
    37. Wang, H., Wexler, Y., Ofek, E., and Hoppe, H. 2008. Factoring repeated content within and among images. In SIGGRAPH papers. Google ScholarDigital Library
    38. Wei, L.-Y., and Levoy, M. 2000. Fast texture synthesis using tree-structured vector quantization. In SIGGRAPH papers, 479–488. Google ScholarDigital Library
    39. Wei, L.-Y., and Levoy, M. 2001. Texture synthesis over arbitrary manifold surfaces. In SIGGRAPH papers, 355–360. Google ScholarDigital Library
    40. Zhang, J., Zhou, K., Velho, L., Guo, B., and Shum, H.-Y. 2003. Synthesis of progressively-variant textures on arbitrary surfaces. In SIGGRAPH papers, 295–302. Google ScholarDigital Library
    41. Ziou, D., and Tabbone, S. 1998. Edge detection techniques – an overview. International Journal of Pattern Recognition and Image Analysis 8, 537–559.Google Scholar

ACM Digital Library Publication: