“An Introduction to Neural Shading” by Duca, He, Bitterli, Cumings, Bekin, et al. …
Conference:
Type(s):
Title:
- An Introduction to Neural Shading
Organizer(s):
Presenter(s)/Author(s):
Abstract:
The past five years has seen rapid growth in techniques that replace physically based shading algorithms with learned neural approximations. These techniques compress data and code to provide compact, high-quality approximations to complex algorithms. As of recently, these algorithms can now be hardware accelerated using cross-platform APIs in Vulkan and Direct3D to access AI acceleration hardware. Combined with modern differentiable shading languages, there is now a complete development toolchain for building and deploying neural shaders. The course will take attendees on a theory-to-practice journey through the neural shading space, starting with a survey of the types of techniques and their applications. We will then teach the math behind neural shading, starting small and adding detail from there. After that, the course will cover hardware acceleration concepts and share tips on bringing models from training into a production C-plus-plus environment. We close by reviewing a few “full” neural graphics models and discuss how they advance the state-of-the-art in real-time graphics. After completing this course, participants will understand neural shading fundamentals, and be able to build and deploy hardware-accelerated neural shading models in modern renderers.
Additional Information:
- List of topics and approximate times:
- Introduction (20 min)
- Intro and Overview and Applications (Lefohn/Bitterli, 15 min)
- Plan for next few hours / mechanics (Lefohn/Duca, 15min)
- Fundamentals (30 min)
- Initial Example – Chris Cummings (15 min)
- Formalisms – Benedikt Bitterli (15 min) Core Concepts (50 min)
- Your First MLP in a Shader – Chris Cummings (20 min)
- Learning a Neural Texture – Benedikt Bitterli (20 min)
- Autodiff Usage – Yong He (10 min)
- Hardware Acceleration (20 min, Yong He)
- Deployment (35 min)
- Neural Shading in Graphics Engines – Sergey Bekin (15 min)
- Performance Tuning – Anton Panteleev (20 min) Applications (20 min)
- Neural Texture Compression – Anton Panteleev (10 min)
- Neural Materials – Benedikt Bitterli (10 min) Wrap-up (5min)
Beginner
Prerequisite: We assume attendees understand the basics of realtime graphics and shading languages
Topics: GPU, Neural Networks, Shading
Additional Info: This course teaches the fundamentals of neural shading, wherein traditional graphics algorithms are replaced with simple neural networks. Both theory and practical implementation will be covered, along with hardware acceleration, and production deployment. Follow along with the instructors using interactive samples written in Python and Slang!
