“Computational time-lapse video” by Bennett and McMillan

We present methods for generating novel time-lapse videos that address the inherent sampling issues that arise with traditional photographic techniques. Starting with video-rate footage as input, our post-process downsamples the source material into a time-lapse video and provides user controls for retaining, removing, and resampling events. We employ two techniques for selecting and combining source frames to form the output. First, we present a non-uniform sampling method, based on dynamic programming, which optimizes the sampling of the input video to match the user’s desired duration and visual objectives. We present multiple error metrics for this optimization, each resulting in different sampling characteristics. To complement the non-uniform sampling, we present the virtual shutter, a non-linear filtering technique that synthetically extends the exposure time of time-lapse frames.

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D. H., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics, 294–302. Google ScholarDigital Library
2. Bennett, E. P., and McMillan, L. 2005. Video enhancement using per-pixel virtual exposures. ACM Transactions on Graphics 24, 3, 845–852. Google ScholarDigital Library
3. Bennett, E. P. 2007. Computational Video Enhancement. PhD thesis, The University of North Carolina at Chapel Hill. Google ScholarDigital Library
4. Braun, M. 1995. Picturing Time: The Work of Etienne-Jules Marey. U. C. Press.Google Scholar
5. Cohen, M. F., Colburn, R. A., and Drucker, S. 2003. Image stacks. Tech. Rep. MSR-TR-2003-40, Microsoft Research.Google Scholar
6. DeMenthon, D., Kobla, V., and Doermann, D. 1998. Video summarization by curve simplification. In Proc. ACM Multimedia, 211–218. Google ScholarDigital Library
7. Divakaran, A., Peker, K. A., Radhakrishnan, R., Xiong, Z., and Cabasson, R. 2003. Video summarization using MPEG-7 motion activity and audio descriptors. Tech. Rep. TR-2003-34, Mitsubishi Electric Research Laboratory.Google Scholar
8. Douglas, D., and Peucker, T. 1973. Algorithm for the reduction of the number of points required to represent a line or its caricature. The Canadian Cartographer 10, 2, 112–122.Google ScholarCross Ref
9. Edgerton, H. E., and Killian, J. R. 1979. Moments of Vision. MIT Press.Google Scholar
10. Hua, X.-S., Lu, L., and Zhang, H.-J. 2003. AVE: Automated home video editing. In Proc. ACM Multimedia, 490–497. Google ScholarDigital Library
11. Kinsman, E. 2006. The time-lapse photography FAQ. www.sciencephotography.com.Google Scholar
12. Muybridge, E. 1955. The Human Figure In Motion. Dover.Google Scholar
13. Perez, J.-C., and Vidal, E. 1994. Optimum polygonal approximation of digitized curves. Pattern Recognition Letters 15, 743–750. Google ScholarDigital Library
14. Porter, T., and Duff, T. 1984. Compositing digital images. Computer Graphics 19, 3, 253–259. Google ScholarDigital Library
15. Rav-Acha, A., Pritch, Y., and Peleg, S. 2006. Making a long video short: Dynamic video synopsis. In Proc. CVPR, 435–441. Google ScholarDigital Library
16. Riddle, P. 1979. Time-Lapse Cinemicroscopy. Academic Press.Google Scholar
17. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In Proc. ACM SIGGRAPH, 489–498. Google ScholarDigital Library
18. Smith, M. A., and Kanade, T. 1997. Video skimming and characterization through the combination of image and language understanding techniques. Tech. Rep. CMU-CS-97-111, Carnegie Mellon University.Google Scholar