“Learning generative models for rendering specular microgeometry” by Kuznetsov, Hašan, Xu, Yan, Walter, et al. … – ACM SIGGRAPH HISTORY ARCHIVES

“Learning generative models for rendering specular microgeometry” by Kuznetsov, Hašan, Xu, Yan, Walter, et al. …

  • 2019 SA Technical Papers_Kuznetsov_Learning generative models for rendering specular microgeometry

Conference:


Type(s):


Title:

    Learning generative models for rendering specular microgeometry

Session/Category Title:   Samples & Speckles

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Rendering specular material appearance is a core problem of computer graphics. While smooth analytical material models are widely used, the high-frequency structure of real specular highlights requires considering discrete, finite microgeometry. Instead of explicit modeling and simulation of the surface microstructure (which was explored in previous work), we propose a novel direction: learning the high-frequency directional patterns from synthetic or measured examples, by training a generative adversarial network (GAN). A key challenge in applying GAN synthesis to spatially varying BRDFs is evaluating the reflectance for a single location and direction without the cost of evaluating the whole hemisphere. We resolve this using a novel method for partial evaluation of the generator network. We are also able to control large-scale spatial texture using a conditional GAN approach. The benefits of our approach include the ability to synthesize spatially large results without repetition, support for learning from measured data, and evaluation performance independent of the complexity of the dataset synthesis or measurement.

References:


    1. Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance Modeling by Neural Texture Synthesis. ACM Trans. Graph. 35, 4, Article 65 (2016), 65:1–65:13 pages.Google ScholarDigital Library
    2. Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), Vol. 70. PMLR, International Convention Centre, Sydney, Australia, 214–223. http://proceedings.mlr.press/v70/arjovsky17a.htmlGoogle ScholarDigital Library
    3. Matt Jen-Yuan Chiang, Benedikt Bitterli, Chuck Tappan, and Brent Burley. 2016. A Practical and Controllable Hair and Fur Model for Production Path Tracing. Computer Graphics Forum 35, 2 (2016), 275–283.Google ScholarCross Ref
    4. R. L. Cook and K. E. Torrance. 1982. A Reflectance Model for Computer Graphics. ACM Trans. Graph. 1, 1 (1982), 7–24.Google ScholarDigital Library
    5. Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. 2018. Single-image SVBRDF Capture with a Rendering-aware Deep Network. ACM Trans. Graph. 37, 4, Article 128 (July 2018), 128:1–128:15 pages.Google ScholarDigital Library
    6. Chris Donahue, Julian McAuley, and Miller Puckette. 2018. Synthesizing Audio with Generative Adversarial Networks. CoRR abs/1802.04208 (2018). arXiv:1802.04208 http://arxiv.org/abs/1802.04208Google Scholar
    7. Zhao Dong, Bruce Walter, Steve Marschner, and Donald P. Greenberg. 2015. Predicting Appearance from Measured Microgeometry of Metal Surfaces. ACM Trans. Graph. 35, 1, Article 9 (2015), 13 pages. Google ScholarDigital Library
    8. Alexei A. Efros and William T. Freeman. 2001. Image Quilting for Texture Synthesis and Transfer. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01). 341–346.Google Scholar
    9. Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafał K. Mantiuk, and Jonas Unger. 2017. HDR Image Reconstruction from a Single Exposure Using Deep CNNs. ACM Trans. Graph. 36, 6, Article 178 (Nov. 2017), 15 pages.Google ScholarDigital Library
    10. Bruno Galerne, Ares Lagae, Sylvain Lefebvre, and George Drettakis. 2012. Gabor Noise by Example. ACM Trans. Graph. 31, 4, Article 73 (July 2012), 9 pages.Google ScholarDigital Library
    11. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27. 2672–2680.Google Scholar
    12. Paul Graham, Borom Tunwattanapong, Jay Busch, Xueming Yu, Andrew Jones, Paul Debevec, and Abhijeet Ghosh. 2013. Measurement-Based Synthesis of Facial Microgeometry. Computer Graphics Forum 32, 2pt3 (2013), 335–344.Google Scholar
    13. Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. 2017. Globally and Locally Consistent Image Completion. ACM Transactions on Graphics (Proc. of SIGGRAPH 2017) 36, 4, Article 107 (2017), 107:1–107:14 pages.Google Scholar
    14. Wenzel Jakob, Miloš Hašan, Ling-Qi Yan, Jason Lawrence, Ravi Ramamoorthi, and Steve Marschner. 2014. Discrete Stochastic Microfacet Models. ACM Trans. Graph. 33, 4 (2014).Google ScholarDigital Library
    15. Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations. https://openreview.net/forum?id=Hk99zCeAbGoogle Scholar
    16. Jonathan Leaf, Rundong Wu, Eston Schweickart, Doug L. James, and Steve Marschner. 2018. Interactive Design of Periodic Yarn-level Cloth Patterns. ACM Trans. Graph. 37, 6, Article 202 (Dec. 2018), 15 pages.Google ScholarDigital Library
    17. Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
    18. Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling Surface Appearance from a Single Photograph Using Self-augmented Convolutional Neural Networks. ACM Trans. Graph. 36, 4, Article 45 (July 2017), 11 pages.Google ScholarDigital Library
    19. Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chandraker. 2018. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In Computer Vision – ECCV 2018 – 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part III (Lecture Notes in Computer Science), Vol. 11207. 74–90.Google Scholar
    20. Mehdi Mirza and Simon Osindero. 2014. Conditional Generative Adversarial Nets. CoRR abs/1411.1784 (2014). http://arxiv.org/abs/1411.1784Google Scholar
    21. Koki Nagano, Graham Fyffe, Oleg Alexander, Jernej Barbiç, Hao Li, Abhijeet Ghosh, and Paul Debevec. 2015. Skin Microstructure Deformation with Displacement Map Convolution. ACM Trans. Graph. 34, 4, Article 109 (July 2015), 10 pages.Google ScholarDigital Library
    22. Giljoo Nam, Joo Ho Lee, Hongzhi Wu, Diego Gutierrez, and Min H. Kim. 2016. Simultaneous Acquisition of Microscale Reflectance and Normals. ACM Trans. Graph. 35, 6, Article 185 (Nov. 2016), 11 pages.Google ScholarDigital Library
    23. Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. CoRR abs/1511.06434 (2015). arXiv:1511.06434 http://arxiv.org/abs/1511.06434Google Scholar
    24. Gilles Rainer, Wenzel Jakob, Abhijeet Ghosh, and Tim Weyrich. 2019. Neural BTF Compression and Interpolation. Computer Graphics Forum (2019). Google ScholarCross Ref
    25. Boris Raymond, Gael Guennebaud, and Pascal Barla. 2016. Multi-Scale Rendering of Scratched Materials using a Structured SV-BRDF Model. ACM Transactions on Graphics (July 2016). Google ScholarDigital Library
    26. Szymon Rusinkiewicz. 1998. A New Change of Variables for Efficient BRDF Representation.. In Rendering Techniques (Eurographics). Springer, 11–22.Google Scholar
    27. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in neural information processing systems. 2234–2242.Google Scholar
    28. Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. 2018. MoCoGAN: Decomposing Motion and Content for Video Generation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
    29. Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction Through Rough Surfaces (EGSR 07). 195–206.Google ScholarDigital Library
    30. Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-Resolution Image Synthesis and Semantic Manipulation With Conditional GANs. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
    31. Sebastian Werner, Zdravko Velinov, Wenzel Jakob, and Matthias B. Hullin. 2017. Scratch Iridescence: Wave-optical Rendering of Diffractive Surface Structure. ACM Trans. Graph. 36, 6, Article 207 (2017), 14 pages. Google ScholarDigital Library
    32. Ling-Qi Yan, Miloš Hašan, Wenzel Jakob, Jason Lawrence, Steve Marschner, and Ravi Ramamoorthi. 2014. Rendering Glints on High-resolution Normal-mapped Specular Surfaces. ACM Trans. Graph. 33, 4, Article 116 (2014), 9 pages. Google ScholarDigital Library
    33. Ling-Qi Yan, Miloš Hašan, Steve Marschner, and Ravi Ramamoorthi. 2016. Positionnormal Distributions for Efficient Rendering of Specular Microstructure. ACM Trans. Graph. 35, 4, Article 56 (2016), 9 pages. Google ScholarDigital Library
    34. Ling-Qi Yan, Miloš Hašan, Bruce Walter, Steve Marschner, and Ravi Ramamoorthi. 2018. Rendering Specular Microgeometry with Wave Optics. ACM Trans. Graph. 37, 4 (2018).Google ScholarDigital Library
    35. Jinsong Zhang and Jean-Francois Lalonde. 2017. Learning High Dynamic Range From Outdoor Panoramas. In The IEEE International Conference on Computer Vision (ICCV).Google Scholar
    36. Shuang Zhao, Fujun Luan, and Kavita Bala. 2016. Fitting Procedural Yarn Models for Realistic Cloth Rendering. ACM Trans. Graph. 35, 4, Article 51 (July 2016), 11 pages.Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org