“StyLit: illumination-guided example-based stylization of 3D renderings” by Fišer, Jamriška, Lukáč, Shechtman, Asente, et al. …

  • ©Jakub Fišer, Ondřej Jamriška, Michal Lukáč, Eli Shechtman, Paul Asente, Jingwan Lu, and Daniel Sýkora




    StyLit: illumination-guided example-based stylization of 3D renderings

Session/Category Title: IMAGE & SHAPE MANIPULATION




    We present an approach to example-based stylization of 3D renderings that better preserves the rich expressiveness of hand-created artwork. Unlike previous techniques, which are mainly guided by colors and normals, our approach is based on light propagation in the scene. This novel type of guidance can distinguish among context-dependent illumination effects, for which artists typically use different stylization techniques, and delivers a look closer to realistic artwork. In addition, we demonstrate that the current state of the art in guided texture synthesis produces artifacts that can significantly decrease the fidelity of the synthesized imagery, and propose an improved algorithm that alleviates them. Finally, we demonstrate our method’s effectiveness on a variety of scenes and styles, in applications like interactive shading study or autocompletion.


    1. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics 28, 3, 24. Google ScholarDigital Library
    2. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized PatchMatch correspondence algorithm. In Proceedings of European Conference on Computer Vision, 29–43. Google ScholarDigital Library
    3. Barnes, C., Zhang, F.-L., Lou, L., Wu, X., and Hu, S.-M. 2015. PatchTable: Efficient patch queries for large datasets and applications. ACM Transactions on Graphics 34, 4, 97. Google ScholarDigital Library
    4. Bénard, P., Lagae, A., Vangorp, P., Lefebvre, S., Drettakis, G., and Thollot, J. 2010. A dynamic noise primitive for coherent stylization. Computer Graphics Forum 29, 4, 1497–1506. Google ScholarDigital Library
    5. Bénard, P., Cole, F., Kass, M., Mordatch, I., Hegarty, J., Senn, M. S., Fleischer, K., Pesare, D., and Breeden, K. 2013. Stylizing animation by example. ACM Transactions on Graphics 32, 4, 119. Google ScholarDigital Library
    6. Blinn, J. F., and Newell, M. E. 1976. Texture and reflection in computer generated images. Communications of the ACM 19, 10, 542–547. Google ScholarDigital Library
    7. Bousseau, A., Kaplan, M., Thollot, J., and Sillion, F. 2006. Interactive watercolor rendering with temporal coherence and abstraction. In International Symposium on Non-Photorealistic Animation and Rendering, 141–149. Google ScholarDigital Library
    8. Chen, J., and Wang, B. 2010. High quality solid texture synthesis using position and index histogram matching. The Visual Computer 26, 4, 253–262. Google ScholarDigital Library
    9. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-generated watercolor. In SIGGRAPH Conference Proceedings, 421–430. Google ScholarDigital Library
    10. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image Melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics 31, 4, 82. Google ScholarDigital Library
    11. Diamanti, O., Barnes, C., Paris, S., Shechtman, E., and Sorkine-Hornung, O. 2015. Synthesis of complex image appearance from limited exemplars. ACM Transactions on Graphics 34, 2, 22. Google ScholarDigital Library
    12. Fišer, J., Lukáč, M., Jamriška, O., Č adík, M., Gingold, Y., Asente, P., and Sýkora, D. 2014. Color Me Noisy: Example-based rendering of hand-colored animations with temporal noise control. Computer Graphics Forum 33, 4, 1–10.Google ScholarCross Ref
    13. Gatys, L. A., Ecker, A. S., and Bethge, M. 2015. A neural algorithm of artistic style. CoRR abs/1508.06576.Google Scholar
    14. Haeberli, P. 1990. Paint by numbers: Abstract image representations. SIGGRAPH Computer Graphics 24, 4, 207–214. Google ScholarDigital Library
    15. Haevre, W. V., Laerhoven, T. V., Fiore, F. D., and Reeth, F. V. 2007. From Dust Till Drawn: A real-time bidirectional pastel simulation. The Visual Computer 23, 9-11, 925–934. Google ScholarDigital Library
    16. Han, J., Zhou, K., Wei, L.-Y., Gong, M., Bao, H., Zhang, X., and Guo, B. 2006. Fast example-based surface texture synthesis via discrete optimization. The Visual Computer 22, 9-11, 918–925. Google ScholarDigital Library
    17. Hashimoto, R., Johan, H., and Nishita, T. 2003. Creating various styles of animations using example-based filtering. In Proceedings of Computer Graphics International, 312–317.Google Scholar
    18. Heckbert, P. S. 1990. Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH Computer Graphics 24, 4, 145–154. Google ScholarDigital Library
    19. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In SIGGRAPH Conference Proceedings, 327–340. Google ScholarDigital Library
    20. Jamriška, O., Fišer, J., Asente, P., Lu, J., Shechtman, E., and Sýkora, D. 2015. LazyFluids: Appearance transfer for fluid animations. ACM Transactions on Graphics 34, 4, 92. Google ScholarDigital Library
    21. Kajiya, J. T. 1986. The rendering equation. SIGGRAPH Computer Graphics 20, 4, 143–150. Google ScholarDigital Library
    22. Kaspar, A., Neubert, B., Lischinski, D., Pauly, M., and Kopf, J. 2015. Self tuning texture optimization. Computer Graphics Forum 34, 2, 349–360. Google ScholarDigital Library
    23. Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D., and Wong, T.-T. 2007. Solid texture synthesis from 2D exemplars. ACM Transactions on Graphics 26, 3, 2. Google ScholarDigital Library
    24. Kwatra, V., Essa, I. A., Bobick, A. F., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics 24, 3, 795–802. Google ScholarDigital Library
    25. Kyprianidis, J. E., Collomosse, J., Wang, T., and Isenberg, T. 2013. State of the “art”: A taxonomy of artistic stylization techniques for images and video. IEEE Transactions on Visualization and Computer Graphics 19, 5, 866–885. Google ScholarDigital Library
    26. Lee, H., Seo, S., Ryoo, S., and Yoon, K. 2010. Directional texture transfer. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 43–48. Google ScholarDigital Library
    27. Lefebvre, S., and Hoppe, H. 2006. Appearance-space texture synthesis. ACM Transactions on Graphics 25, 3, 541–548. Google ScholarDigital Library
    28. Lu, C., Xu, L., and Jia, J. 2012. Combining sketch and tone for pencil drawing production. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 65–73. Google ScholarDigital Library
    29. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., and Pérez, P. 2014. Video inpainting of complex scenes. SIAM Journal of Imaging Science 7, 4, 1993–2019.Google ScholarCross Ref
    30. Rong, G., and Tan, T.-S. 2006. Jump flooding in GPU with applications to Voronoi diagram and distance transform. In Proceedings of Symposium on Interactive 3D Graphics and Games, 109–116. Google ScholarDigital Library
    31. Rosenberger, A., Cohen-Or, D., and Lischinski, D. 2009. Layered Shape Synthesis: Automatic generation of control maps for non-stationary textures. ACM Transactions on Graphics 28, 5, 107. Google ScholarDigital Library
    32. Salisbury, M. P., Wong, M. T., Hughes, J. F., and Salesin, D. H. 1997. Orientable textures for image-based pen-and-ink illustration. In SIGGRAPH Conference Proceedings, 401–406. Google ScholarDigital Library
    33. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
    34. Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556.Google Scholar
    35. Sloan, P.-P. J., Martin, W., Gooch, A., and Gooch, B. 2001. The Lit Sphere: A model for capturing NPR shading from art. In Proceedings of Graphics Interface, 143–150. Google ScholarDigital Library
    36. Tu, Z., Chen, X., Yuille, A. L., and Zhu, S.-c. 2005. Image Parsing: Unifying segmentation, detection, and recognition. International Journal of Computer Vision 63, 2, 113–140. Google ScholarDigital Library
    37. Wang, B., Wang, W., Yang, H., and Sun, J. 2004. Efficient example-based painting and synthesis of 2D directional texture. IEEE Transactions on Visualization and Computer Graphics 10, 3, 266–277. Google ScholarDigital Library
    38. Wei, L.-Y., Han, J., Zhou, K., Bao, H., Guo, B., and Shum, H.-Y. 2008. Inverse texture synthesis. ACM Transactions on Graphics 27, 3. Google ScholarDigital Library
    39. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 3, 463–476. Google ScholarDigital Library
    40. Winnemöller, H., Kyprianidis, J. E., and Olsen, S. C. 2012. XDoG: An extended difference-of-gaussians compendium including advanced image stylization. Computers & Graphics 36, 6, 740–753. Google ScholarDigital Library
    41. Xing, J., Chen, H.-T., and Wei, L.-Y. 2014. Autocomplete painting repetitions. ACM Transactions on Graphics 33, 6, 172. Google ScholarDigital Library
    42. Xing, J., Wei, L.-Y., Shiratori, T., and Yatani, K. 2015. Autocomplete hand-drawn animations. ACM Transactions on Graphics 34, 6, 169. Google ScholarDigital Library
    43. Zeng, K., Zhao, M., Xiong, C., and Zhu, S.-C. 2009. From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1, 2. Google ScholarDigital Library
    44. Zeng, Q., Chen, W., Wang, H., Tu, C., Cohen-Or, D., Lischinski, D., and Chen, B. 2015. Hallucinating stereoscopy from a single image. Computer Graphics Forum 34, 2, 1–12. Google ScholarDigital Library
    45. Zhao, M., and Zhu, S.-C. 2011. Portrait painting using active templates. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 117–124. Google ScholarDigital Library

ACM Digital Library Publication:

Overview Page: