“Compositing Light field Video Using Multiplane Images” by DuVall, Broxton, Flynn and Debevec

  • ©Matthew DuVall, Michael Broxton, John Flynn, and Paul E. Debevec

  • ©Matthew DuVall, Michael Broxton, John Flynn, and Paul E. Debevec

  • ©Matthew DuVall, Michael Broxton, John Flynn, and Paul E. Debevec



Entry Number: 67


    Compositing Light field Video Using Multiplane Images



    We present a variety of new compositing techniques using Multi-plane Images (MPI’s) [Zhou et al. 2018] derived from footage shot with an inexpensive and portable light field video camera array. The effects include camera stabilization, foreground object removal, synthetic depth of field, and deep compositing. Traditional compositing is based around layering RGBA images to visually integrate elements into the same scene, and often requires manual 2D and/or 3D artist intervention to achieve realism in the presence of volumetric effects such as smoke or splashing water. We leverage the newly introduced DeepView solver [Flynn et al. 2019] and a light field camera array to generate MPIs stored in the DeepEXR format for compositing with realistic spatial integration and a simple work-flow which offers new creative capabilities. We demonstrate using this technique by combining footage that would otherwise be very challenging and time intensive to achieve when using traditional techniques, with minimal artist intervention.


    John Flynn, Michael Broxton, Paul Debevec, Matthew DuVall, Graham Fyffe, Ryan
    Overbeck, Noah Snavely, and Richard Tucker. 2019. DeepView: View synthesis
    with learned gradient descent. CVPR (2019).
    Florian Kainz. 2013. Interpreting OpenEXR Deep Pixels. (2013). Retrieved from https://lists.nongnu.org/archive/html/openexr-devel/2013-
    Tom Lokovic and Eric Veach. 2000. Deep shadow maps. Proceedings of the 27th
    Annual Conference on Computer Graphics and Interactive Techniques (2000). https:
    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. 2018.
    Stereo Magnification: Learning view synthesis using multiplane images. Siggraph



Overview Page: