“Interactive Object Insertion With Differentiable Rendering” by Peng, Taira, Careaga and Aksoy
Conference:
Type(s):
Title:
- Interactive Object Insertion With Differentiable Rendering
Session/Category Title:
- Rendering
Presenter(s)/Author(s):
Abstract:
We develop an object insertion pipeline and interface that enables iterative editing of illumination-aware composite images. Our pipeline leverages off-the-shelf computer vision methods and differentiable rendering to reconstruct a 3D representation of a given scene. Users can add 3D objects and render them with physically accurate lighting effects.
References:
[1] Chris Careaga and Yağız Aksoy. 2024. Colorful Diffuse Intrinsic Image Decomposition in the Wild. ACM Trans. Graph. 43, 6, Article 178 (2024), 12 pages.
[2] Chris Careaga and Yağız Aksoy. 2025. Physically Controllable Relighting of Photographs. In Proc. SIGGRAPH.
[3] Paul Debevec. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proc. SIGGRAPH.
[4] Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. 2022. Mitsuba 3 renderer. https://mitsuba-renderer.org.
[5] Kevin Karsch, Varsha Hedau, David Forsyth, and Derek Hoiem. 2011. Rendering synthetic objects into legacy photographs. ACM Trans. Graph. (2011).
[6] Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. 2025. MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision.
