“The virtual cinematographer: a paradigm for automatic real-time camera control and directing” by He, Cohen and Salesin

This paper presents a paradigm for automatically generating complete camera specifications for capturing events in virtual 3D environments in real-time. We describe a fully implemented system, called the Virtual Cinematographer, and demonstrate its application in a virtual “party” setting. Cinematographic expertise, in the form of film idioms, is encoded as a set of small hierarchically organized finite state machines. Each idiom is responsible for capturing a particular type of scene, such as three virtual actors conversing or one actor moving across the environment. The idiom selects shot types and the timing of transitions between shots to best communicate events as they unfold. A set of camera modules, shared by the idioms, is responsible for the low-level geometric placement of specific cameras for each shot. The camera modules are also responsible for making subtle changes in the virtual actors’ positions to best frame each shot. In this paper, we discuss some basic heuristics of filmmaking and show how these ideas are encoded in the Virtual Cinematographer.

1. Daniel Arijon. Grammar of the Film Language. Communication Arts Books, Hastings House, Publishers, New York, 1976.
2. James Blinn. Where am I? What am I looking at? IEEE Computer Graphics and Applications, pages 76-81, 1988.
3. David B. Christianson, Sean E. Anderson, Li-wei He, David H. Salesin, Daniel S. Weld, and Michael F. Cohen. Declarative camera control for automatic cinematography. In Proceedings of the AAAI-96, August 1996.
4. Steven M. Drucker, Tinsley A. Galyean, and David Zeltzer. CINEMA: A system for procedural camera movements. In David Zeltzer, editor, Computer Graphics (1992 Symposium on Interactive 3D Graphics), volume 25, pages 67-70, March 1992.
5. Steven M. Drucker and David Zelter. CamDroid: A system for implementing intelligent camera control. In Michael Zyda, editor, Computer Graphics (1995 Symposium on Interactive 3D Graphics), volume 28, pages 139-144, April 1995.
6. Steven M. Drucker and David Zeltzer. Intelligent camera control in a virtual environment. In Proceedings of Graphics Interface ’94, pages 190-199, Banff, Alberta, Canada, May 1994. Canadian Information Processing Society.
7. James D. Foley, Andries van Dam, Steven K. Feiner, and John F. Hughes. Computer Graphics, Principles and Practice. Addison- Wesley Publishing Company, Reading, Massachusetts, second edition, 1990.
8. Michael Gleicher and Andrew Witkin. Through-the-lens camera control. In Edwin E. Catmull, editor, Computer Graphics (SIGGRAPH ’92 Proceedings), volume 26, pages 331-340, July 1992.
9. David Harel. Statecharts: A visual formalism for complex systems. Science of Computer Programming, pages 231-274, 1987.
10. Donald Hearn and M. Pauline Baker. Computer Graphics. Prentice Hall, Englewood Cliffs, New Jersey, second edition, 1994.
11. Peter Karp and Steven Feiner. Issues in the automated generation of animated presentations. In Proceedings of Graphics Interface ’90, pages 39-48, May 1990.
12. Peter Karp and Steven Feiner. Automated presentation planning of animation using task decomposition with heuristic reasoning. In Proceedings of Graphics Interface ’93, pages 118-127, Toronto, Ontario, Canada, May 1993. Canadian Information Processing Society.
13. Christopher Lukas. Directing for Film and Television. Anchor Press/Doubleday, Garden City, N.Y., 1985.
14. Jock D. Mackinlay, Stuart K. Card, and George G. Robertson. Rapid controlled movement through a virtual 3D workspace. In Forest Baskett, editor, Computer Graphics (SIGGRAPH ’90 Proceedings), volume 24, pages 171-176, August 1990.
15. Joseph V. Mascelli. The Five C’s of Cinematography. Cine/Grafic Publications, Hollywood, 1965.
16. Cary B. Phillips, Norman I. Badler, and John Granieri. Automatic viewing control for 3D direct manipulation. In David Zeltzer, editor, Computer Graphics (1992 Symposium on Interactive 3D Graphics), volume 25, pages 71-74, March 1992.
17. Warren Sack and Marc Davis. IDIC: Assembling video sequences from story plans and content annotations. In IEEE International Conference on Multimedia Computing and Systems, Boston, MA, May 1994.
18. Patrick Tucker. Secrets of Screen Acting. Routledge, New York, 1994.