“Directing Cinematographic Drones” by Galvane, Lino, Christie and Fleureau

  • ©Quentin Galvane, Christophe Lino, Marc Christie, and Julien Fleureau

Conference:


Type:


Title:

    Directing Cinematographic Drones

Presenter(s)/Author(s):


Session Title: Taking Flight

Moderator(s):


Abstract:


    Quadrotor drones equipped with high-quality cameras have rapidly raised as novel, cheap, and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth—and cinematographic—control of a camera drone remains challenging for most users, despite recent tools that either automate part of the process or enable the manual design of waypoints to create drone trajectories.

    This article moves a step further by offering high-level control of cinematographic drones for the specific task of framing dynamic targets. We propose techniques to automatically and interactively plan quadrotor drone motions in dynamic three-dimensional (3D) environments while satisfying both cinematographic and physical quadrotor constraints. We first propose the Drone Toric Space, a dedicated camera parameter space with embedded constraints, and derive some intuitive on-screen viewpoint manipulators. Second, we propose a dedicated path planning technique that ensures both that cinematographic properties can be enforced along the path and that the path is physically feasible by a quadrotor drone. At last, we build on the Drone Toric Space and the specific path planning technique to coordinate the motion of multiple drones around dynamic targets. A number of results demonstrate the interactive and automated capacities of our approaches on different use-cases.

References:


    1. D. Arijon. 1976. Grammar of the Film Language. Hastings House Publishers.Google Scholar
    2. W. Bares, S. McDermott, C. Boudreaux, and S. Thainimit. 2000a. Virtual 3D camera composition from frame constraints. In Proceedings of the 8th ACM International Conference on Multimedia (MULTIMEDIA’00). ACM Press, 177–186. Google ScholarDigital Library
    3. W. Bares, S. Thainimit, S. McDermott, and C. Boudreaux. 2000b. A model for constraint-based camera planning. In Smart Graphics AAAI Spring Symposium. Stanford, California, 84–91.Google Scholar
    4. Jim Blinn. 1988. Where am I? What am I looking at?(cinematography). IEEE Comput. Graph. Appl. 8, 4 (1988), 76–81. Google ScholarDigital Library
    5. Marc Christie and Patrick Olivier. 2009. Camera control in computer graphics: Models, techniques and applications. In ACM SIGGRAPH ASIA 2009 Courses. ACM, 3. Google ScholarDigital Library
    6. Marc Christie, Patrick Olivier, and Jean-Marie Normand. 2008. Camera control in computer graphics. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 2197–2218.Google Scholar
    7. Robin Deits and Russ Tedrake. 2015. Efficient mixed-integer planning for UAVs in cluttered environments. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA’15). IEEE, 42–49.Google ScholarCross Ref
    8. Steven M. Drucker and David Zeltzer. 1994. Intelligent camera control in a virtual environment. In Graphics Interface. Canadian Information Processing Society, 190–190.Google Scholar
    9. O. Dunkley, J. Engel, J. Sturm, and D. Cremers. 2014. Visual-inertial navigation for a camera-equipped 25g nano-quadrotor. In Proceedings of the IROS2014 Aerial Open Source Robotics Workshop.Google Scholar
    10. Julien Fleureau, Quentin Galvane, Francois-Louis Tariolle, and Philippe Guillotel. 2016. Generic drone control platform for autonomous capture of cinema scenes. In Proceedings of the 2nd Workshop on Micro Aerial Vehicle Networks, Systems, and Applications for Civilian Use (DroNet’16). ACM, New York, NY, 35–40. Google ScholarDigital Library
    11. Quentin Galvane, Julien Fleureau, Francois-Louis Tariolle, and Philippe Guillotel. 2016. Automated cinematography with unmanned aerial vehicles. In Eurographics Workshop on Intelligent Cinematography and Editing, M. Christie, Q. Galvane, A. Jhala, and R. Ronfard (Eds.). The Eurographics Association. Google ScholarDigital Library
    12. Quentin Galvane, Rémi Ronfard, Marc Christie, and Nicolas Szilas. 2014. Narrative-driven camera control for cinematic replay of computer games. In Proceedings of the 7th International Conference on Motion in Games. ACM, 109–117. Google ScholarDigital Library
    13. Quentin Galvane, Rémi Ronfard, Christophe Lino, and Marc Christie. 2015. Continuity editing for 3D animation. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Press. Google ScholarDigital Library
    14. Christoph Gebhardt, Benjamin Hepp, Tobias Nägeli, Stefan Stevšić, and Otmar Hilliges. 2016. Airways: Optimization-based planning of quadrotor trajectories according to high-level user goals. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM, New York, NY, 2508–2519. Google ScholarDigital Library
    15. M. Gleicher and A. Witkin. 1992. Through-the-lens camera control. In Proceedings of ACM SIGGRAPH’92. 331–340. Google ScholarDigital Library
    16. Nicolas Halper, Ralf Helbing, and Thomas Strothotte. 2001. A camera engine for computer games: Managing the trade-off between constraint satisfaction and frame coherence. Comput. Graph. Forum 20, 3 (2001), 174–183.Google ScholarCross Ref
    17. Niels Joubert, L. E. Jane, Dan B. Goldman, Floraine Berthouzoz, Mike Roberts, James A. Landay, and Pat Hanrahan. 2016. Towards a drone cinematographer: Guiding quadrotor cameras using visual composition principles. CoRR abs/1610.01691 (2016). http://arxiv.org/abs/1610.01691.Google Scholar
    18. Niels Joubert, Mike Roberts, Anh Truong, Floraine Berthouzoz, and Pat Hanrahan. 2015. An interactive tool for designing quadrotor camera shots. ACM Trans. Graph. 34, 6 (2015), 238. Google ScholarDigital Library
    19. Christophe Lino and Marc Christie. 2012. Efficient composition for virtual camera control. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’12). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland. Google ScholarDigital Library
    20. Christophe Lino and Marc Christie. 2015. Intuitive and efficient camera control with the toric space. ACM Trans. Graph. 34, 4 (2015), 82. Google ScholarDigital Library
    21. C. Lino, M. Christie, F. Lamarche, G. Schofield, and P. Olivier. 2010. A real-time cinematography system for interactive 3D environments. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Google ScholarDigital Library
    22. Daniel Mellinger and Vijay Kumar. 2011. Minimum snap trajectory generation and control for quadrotors. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA’11). IEEE, 2520–2525.Google ScholarCross Ref
    23. Ian Miguel. 2004. Dynamic Flexible Constraint Satisfaction and Its Application to AI Planning. Ph.D. Dissertation. University of Edinburgh.Google Scholar
    24. Steven Minton, Mark D. Johnston, Andrew B. Philips, and Philip Laird. 1990. Solving large-scale constraint-satisfaction and scheduling problems using a heuristic repair method. In Proceedings of the 8th National Conference on Artificial Intelligence. 17–24. http://www.aaai.org/Library/AAAI/1990/aaai90-003.php. Google ScholarDigital Library
    25. T. Nägeli, J. Alonso-Mora, A. Domahidi, D. Rus, and O. Hilliges. 2017. Real-time motion planning for aerial videography with dynamic obstacle avoidance and viewpoint optimization. IEEE Robot. Automat. Lett. 2, 3 (July 2017), 1696–1703.Google ScholarCross Ref
    26. Tobias Nägeli, Lukas Meier, Alexander Domahidi, Javier Alonso-Mora, and Otmar Hilliges. 2017. Real-time planning for automated multi-view drone cinematography. ACM Trans. Graph. 36, 4, Article 132 (July 2017), 10 pages. Google ScholarDigital Library
    27. Stephen Nuske, Sanjiban Choudhury, Sezal Jain, Andrew Chambers, Luke Yoder, Sebastian Scherer, Lyle Chamberlain, Hugh Cover, and Sanjiv Singh. 2015. Autonomous exploration and motion planning for an unmanned aerial vehicle navigating rivers. J. Field Robot. 32, 8 (2015), 1141–1162. Google ScholarDigital Library
    28. Thomas Oskam, Robert W. Sumner, Nils Thuerey, and Markus Gross. 2009. Visibility transition planning for dynamic camera control. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. ACM, 55–65. Google ScholarDigital Library
    29. Guilherme A. S. Pereira, Aveek K. Das, Vijay Kumar, and Mario Fernando Montenegro Campos. 2003. Formation control with configuration space constraints. In Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2755–2760.Google ScholarCross Ref
    30. R. Ranon and T. Urli. 2014. Improving the efficiency of viewpoint composition. IEEE Trans. Vis. Comput. Graph. 20, 5 (May 2014), 795–807. Google ScholarDigital Library
    31. Mike Roberts and Pat Hanrahan. 2016. Generating dynamically feasible trajectories for quadrotor cameras. ACM Trans. Graph. 35, 4 (2016). Google ScholarDigital Library
    32. Rémi Ronfard, Vineet Gandhi, and Laurent Boiron. 2015. The prose storyboard language: A tool for annotating and directing movies. CoRR abs/1508.07593 (2015). http://arxiv.org/abs/1508.07593.Google Scholar
    33. Brian Salomon, Maxim Garber, Ming C. Lin, and Dinesh Manocha. 2003. Interactive navigation in complex environments using path planning. In Proceedings of the 2003 Symposium on Interactive 3D Graphics. ACM, 41–50. Google ScholarDigital Library
    34. Fabrizio Schiano, Antonio Franchi, Daniel Zelazo, and Paolo Robuffo Giordano. 2016. A rigidity-based decentralized bearing formation controller for groups of quadrotor UAVs. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS’16). 5099–5106.Google ScholarDigital Library
    35. Nisha Sudarsanam, Cindy Grimm, and Karan Singh. 2009. CubeCam: A screen-space camera manipulation tool. In Computational Aesthetics in Graphics, Visualization, and Imaging. Google ScholarDigital Library
    36. Matthew Turpin, Kartik Mohta, Nathan Michael, and Vijay Kumar. 2013. Goal assignment and trajectory planning for large teams of aerial robots. In Robotics: Science and Systems.

ACM Digital Library Publication: