“From Reconstruction to Generation: State-Of-Art Approaches for 3D Visualization” by Mohan and Kim
Conference:
Type(s):
Title:
- From Reconstruction to Generation: State-Of-Art Approaches for 3D Visualization
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Our course provides an essential overview of 3D reconstruction and generative AI-based 3D generation approaches that have transformed the field. The course provides an understanding of the fundamentals of photogrammetry, neural radiance fields-based techniques such as Instant NeRF, and generative AI-based 3D generation such as text-to-3D, with a focus on reflecting on these approaches, their applications in 3D world building, and how participants can integrate them into their research. The course firstly introduces the core concepts of photogrammetry, through which participants can understand its importance in reconstructing accurate 3D models. Through case studies, participants will reflect on the applications of photogrammetry in various domains, such as cultural heritage preservation and VR experiences, and can view some examples themselves. We then delve into a gentle introduction to neural rendering with a broad overview of NVIDIA Instant NeRF, a state-of-the-art technique for real-time 3D reconstruction from multiple images. Participants will have the opportunity to watch several demonstrations of InstantNGP and discover its applications in interactive 3D world building. Furthermore, participants will be introduced to emerging generative AI approaches that simplify 3D content generation. In particular, we explore the use of text-to-3D techniques, which leverage natural language descriptions to automatically generate 3D models. We will also examine the impact of generative AI techniques on the creation of realistic and immersive 3D environments. By the end of the course, participants will have a solid understanding of the basics of two salient 3D reconstruction methods, as well as other generative AI techniques for 3D content. We also hope to enable participants to critically evaluate and harness the potential of these approaches in 3D world building.
