“Realistic Post-processing of Rendered 3D Scenes” by Feygina, Ignatov and Makarov


Entry Number: 42


    Realistic Post-processing of Rendered 3D Scenes



    In this talk, we show a realistic post-processing rendering based on generative adversarial network CycleWGAN. We propose to use CycleGAN architecture and Wasserstein loss function with additional identity component in order to transfer graphics from Grand Theft Auto V to the older version of GTA video-game, Grand Theft Auto: San Andreas. We aim to present the application of modern art style transfer and unpaired image-to-image translations methods for graphics improvement using deep neural networks with adversarial loss.


    • Qifeng Chen and Vladlen Koltun. 2017. Photographic image synthesis with cascaded refinement networks. In International Conference on Computer Vision (ICCV), Vol. 1. IEEE. 
    • Marc-André Gardner et al. 2017. Learning to Predict Indoor Illumination from a Single Image. arXiv preprint arXiv:1704.00090 (2017). 
    • Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2414–2423. 
    • Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. NIPS, 2672–2680. 
    • Agrim Gupta, Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2017. Characterizing and Improving Stability in Neural Style Transfer. arXiv preprint arXiv:1705.02092 (2017). 
    • Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. 2001. Image analogies. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 327–340. 
    • Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2016. Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004 (2016). 
    • Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jungkwon Lee, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. arXiv preprint arXiv:1703.05192 (2017). 
    • Ming-Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. In Advances in neural information processing systems. NIPS, 469–477. 
    • Rockstar North Rockstar Games. 2013. Grand Theft Auto: V. http://www.rockstargames. com/V/. [Online; accessed 17-09-2013]. 
    • War Drum Studios Rockstar North. 2004. Grand Theft Auto: San Andreas. http: //www.rockstargames.com/sanandreas/. [Online; accessed 26-10-2004]. 
    • Libin Sun and James Hays. 2017. Super-resolution Using Constrained Deep Texture Synthesis. arXiv preprint arXiv:1701.07604 (2017). 
    • Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2017. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs. arXiv preprint arXiv:1711.11585 (2017). 
    • Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired imageto-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017).



    The work of D.I. Ignatov and I. Makarov was supported by the Russian Science Foundation under grant 17-11-01294 and performed at National Research University Higher School of Economics, Russia.