“Compositing Light field Video Using Multiplane Images” by DuVall, Broxton, Flynn and Debevec
Conference:
Type(s):
Entry Number: 67
Title:
- Compositing Light field Video Using Multiplane Images
Presenter(s)/Author(s):
Abstract:
We present a variety of new compositing techniques using Multi-plane Images (MPI’s) [Zhou et al. 2018] derived from footage shot with an inexpensive and portable light field video camera array. The effects include camera stabilization, foreground object removal, synthetic depth of field, and deep compositing. Traditional compositing is based around layering RGBA images to visually integrate elements into the same scene, and often requires manual 2D and/or 3D artist intervention to achieve realism in the presence of volumetric effects such as smoke or splashing water. We leverage the newly introduced DeepView solver [Flynn et al. 2019] and a light field camera array to generate MPIs stored in the DeepEXR format for compositing with realistic spatial integration and a simple work-flow which offers new creative capabilities. We demonstrate using this technique by combining footage that would otherwise be very challenging and time intensive to achieve when using traditional techniques, with minimal artist intervention.
References:
John Flynn, Michael Broxton, Paul Debevec, Matthew DuVall, Graham Fyffe, Ryan
Overbeck, Noah Snavely, and Richard Tucker. 2019. DeepView: View synthesis
with learned gradient descent. CVPR (2019).
Florian Kainz. 2013. Interpreting OpenEXR Deep Pixels. (2013). Retrieved from https://lists.nongnu.org/archive/html/openexr-devel/2013-
09/pdfPyDOYBTkje.pdf.
Tom Lokovic and Eric Veach. 2000. Deep shadow maps. Proceedings of the 27th
Annual Conference on Computer Graphics and Interactive Techniques (2000). https:
//doi.org/10.1145/344779.344958
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. 2018.
Stereo Magnification: Learning view synthesis using multiplane images. Siggraph
(2018).
