“StyLit: illumination-guided example-based stylization of 3D renderings” by Fišer, Jamriška, Lukáč, Shechtman, Asente, et al. …
Conference:
Type:
Title:
- StyLit: illumination-guided example-based stylization of 3D renderings
Session/Category Title: IMAGE & SHAPE MANIPULATION
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We present an approach to example-based stylization of 3D renderings that better preserves the rich expressiveness of hand-created artwork. Unlike previous techniques, which are mainly guided by colors and normals, our approach is based on light propagation in the scene. This novel type of guidance can distinguish among context-dependent illumination effects, for which artists typically use different stylization techniques, and delivers a look closer to realistic artwork. In addition, we demonstrate that the current state of the art in guided texture synthesis produces artifacts that can significantly decrease the fidelity of the synthesized imagery, and propose an improved algorithm that alleviates them. Finally, we demonstrate our method’s effectiveness on a variety of scenes and styles, in applications like interactive shading study or autocompletion.
References:
1. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics 28, 3, 24. Google ScholarDigital Library
2. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. 2010. The generalized PatchMatch correspondence algorithm. In Proceedings of European Conference on Computer Vision, 29–43. Google ScholarDigital Library
3. Barnes, C., Zhang, F.-L., Lou, L., Wu, X., and Hu, S.-M. 2015. PatchTable: Efficient patch queries for large datasets and applications. ACM Transactions on Graphics 34, 4, 97. Google ScholarDigital Library
4. Bénard, P., Lagae, A., Vangorp, P., Lefebvre, S., Drettakis, G., and Thollot, J. 2010. A dynamic noise primitive for coherent stylization. Computer Graphics Forum 29, 4, 1497–1506. Google ScholarDigital Library
5. Bénard, P., Cole, F., Kass, M., Mordatch, I., Hegarty, J., Senn, M. S., Fleischer, K., Pesare, D., and Breeden, K. 2013. Stylizing animation by example. ACM Transactions on Graphics 32, 4, 119. Google ScholarDigital Library
6. Blinn, J. F., and Newell, M. E. 1976. Texture and reflection in computer generated images. Communications of the ACM 19, 10, 542–547. Google ScholarDigital Library
7. Bousseau, A., Kaplan, M., Thollot, J., and Sillion, F. 2006. Interactive watercolor rendering with temporal coherence and abstraction. In International Symposium on Non-Photorealistic Animation and Rendering, 141–149. Google ScholarDigital Library
8. Chen, J., and Wang, B. 2010. High quality solid texture synthesis using position and index histogram matching. The Visual Computer 26, 4, 253–262. Google ScholarDigital Library
9. Curtis, C. J., Anderson, S. E., Seims, J. E., Fleischer, K. W., and Salesin, D. H. 1997. Computer-generated watercolor. In SIGGRAPH Conference Proceedings, 421–430. Google ScholarDigital Library
10. Darabi, S., Shechtman, E., Barnes, C., Goldman, D. B., and Sen, P. 2012. Image Melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on Graphics 31, 4, 82. Google ScholarDigital Library
11. Diamanti, O., Barnes, C., Paris, S., Shechtman, E., and Sorkine-Hornung, O. 2015. Synthesis of complex image appearance from limited exemplars. ACM Transactions on Graphics 34, 2, 22. Google ScholarDigital Library
12. Fišer, J., Lukáč, M., Jamriška, O., Č adík, M., Gingold, Y., Asente, P., and Sýkora, D. 2014. Color Me Noisy: Example-based rendering of hand-colored animations with temporal noise control. Computer Graphics Forum 33, 4, 1–10.Google ScholarCross Ref
13. Gatys, L. A., Ecker, A. S., and Bethge, M. 2015. A neural algorithm of artistic style. CoRR abs/1508.06576.Google Scholar
14. Haeberli, P. 1990. Paint by numbers: Abstract image representations. SIGGRAPH Computer Graphics 24, 4, 207–214. Google ScholarDigital Library
15. Haevre, W. V., Laerhoven, T. V., Fiore, F. D., and Reeth, F. V. 2007. From Dust Till Drawn: A real-time bidirectional pastel simulation. The Visual Computer 23, 9-11, 925–934. Google ScholarDigital Library
16. Han, J., Zhou, K., Wei, L.-Y., Gong, M., Bao, H., Zhang, X., and Guo, B. 2006. Fast example-based surface texture synthesis via discrete optimization. The Visual Computer 22, 9-11, 918–925. Google ScholarDigital Library
17. Hashimoto, R., Johan, H., and Nishita, T. 2003. Creating various styles of animations using example-based filtering. In Proceedings of Computer Graphics International, 312–317.Google Scholar
18. Heckbert, P. S. 1990. Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH Computer Graphics 24, 4, 145–154. Google ScholarDigital Library
19. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In SIGGRAPH Conference Proceedings, 327–340. Google ScholarDigital Library
20. Jamriška, O., Fišer, J., Asente, P., Lu, J., Shechtman, E., and Sýkora, D. 2015. LazyFluids: Appearance transfer for fluid animations. ACM Transactions on Graphics 34, 4, 92. Google ScholarDigital Library
21. Kajiya, J. T. 1986. The rendering equation. SIGGRAPH Computer Graphics 20, 4, 143–150. Google ScholarDigital Library
22. Kaspar, A., Neubert, B., Lischinski, D., Pauly, M., and Kopf, J. 2015. Self tuning texture optimization. Computer Graphics Forum 34, 2, 349–360. Google ScholarDigital Library
23. Kopf, J., Fu, C.-W., Cohen-Or, D., Deussen, O., Lischinski, D., and Wong, T.-T. 2007. Solid texture synthesis from 2D exemplars. ACM Transactions on Graphics 26, 3, 2. Google ScholarDigital Library
24. Kwatra, V., Essa, I. A., Bobick, A. F., and Kwatra, N. 2005. Texture optimization for example-based synthesis. ACM Transactions on Graphics 24, 3, 795–802. Google ScholarDigital Library
25. Kyprianidis, J. E., Collomosse, J., Wang, T., and Isenberg, T. 2013. State of the “art”: A taxonomy of artistic stylization techniques for images and video. IEEE Transactions on Visualization and Computer Graphics 19, 5, 866–885. Google ScholarDigital Library
26. Lee, H., Seo, S., Ryoo, S., and Yoon, K. 2010. Directional texture transfer. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 43–48. Google ScholarDigital Library
27. Lefebvre, S., and Hoppe, H. 2006. Appearance-space texture synthesis. ACM Transactions on Graphics 25, 3, 541–548. Google ScholarDigital Library
28. Lu, C., Xu, L., and Jia, J. 2012. Combining sketch and tone for pencil drawing production. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 65–73. Google ScholarDigital Library
29. Newson, A., Almansa, A., Fradet, M., Gousseau, Y., and Pérez, P. 2014. Video inpainting of complex scenes. SIAM Journal of Imaging Science 7, 4, 1993–2019.Google ScholarCross Ref
30. Rong, G., and Tan, T.-S. 2006. Jump flooding in GPU with applications to Voronoi diagram and distance transform. In Proceedings of Symposium on Interactive 3D Graphics and Games, 109–116. Google ScholarDigital Library
31. Rosenberger, A., Cohen-Or, D., and Lischinski, D. 2009. Layered Shape Synthesis: Automatic generation of control maps for non-stationary textures. ACM Transactions on Graphics 28, 5, 107. Google ScholarDigital Library
32. Salisbury, M. P., Wong, M. T., Hughes, J. F., and Salesin, D. H. 1997. Orientable textures for image-based pen-and-ink illustration. In SIGGRAPH Conference Proceedings, 401–406. Google ScholarDigital Library
33. Simakov, D., Caspi, Y., Shechtman, E., and Irani, M. 2008. Summarizing visual data using bidirectional similarity. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
34. Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556.Google Scholar
35. Sloan, P.-P. J., Martin, W., Gooch, A., and Gooch, B. 2001. The Lit Sphere: A model for capturing NPR shading from art. In Proceedings of Graphics Interface, 143–150. Google ScholarDigital Library
36. Tu, Z., Chen, X., Yuille, A. L., and Zhu, S.-c. 2005. Image Parsing: Unifying segmentation, detection, and recognition. International Journal of Computer Vision 63, 2, 113–140. Google ScholarDigital Library
37. Wang, B., Wang, W., Yang, H., and Sun, J. 2004. Efficient example-based painting and synthesis of 2D directional texture. IEEE Transactions on Visualization and Computer Graphics 10, 3, 266–277. Google ScholarDigital Library
38. Wei, L.-Y., Han, J., Zhou, K., Bao, H., Guo, B., and Shum, H.-Y. 2008. Inverse texture synthesis. ACM Transactions on Graphics 27, 3. Google ScholarDigital Library
39. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 3, 463–476. Google ScholarDigital Library
40. Winnemöller, H., Kyprianidis, J. E., and Olsen, S. C. 2012. XDoG: An extended difference-of-gaussians compendium including advanced image stylization. Computers & Graphics 36, 6, 740–753. Google ScholarDigital Library
41. Xing, J., Chen, H.-T., and Wei, L.-Y. 2014. Autocomplete painting repetitions. ACM Transactions on Graphics 33, 6, 172. Google ScholarDigital Library
42. Xing, J., Wei, L.-Y., Shiratori, T., and Yatani, K. 2015. Autocomplete hand-drawn animations. ACM Transactions on Graphics 34, 6, 169. Google ScholarDigital Library
43. Zeng, K., Zhao, M., Xiong, C., and Zhu, S.-C. 2009. From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1, 2. Google ScholarDigital Library
44. Zeng, Q., Chen, W., Wang, H., Tu, C., Cohen-Or, D., Lischinski, D., and Chen, B. 2015. Hallucinating stereoscopy from a single image. Computer Graphics Forum 34, 2, 1–12. Google ScholarDigital Library
45. Zhao, M., and Zhu, S.-C. 2011. Portrait painting using active templates. In Proceedings of International Symposium on Non-Photorealistic Animation and Rendering, 117–124. Google ScholarDigital Library
