“Automatic editing of footage from multiple social cameras” by Arev, Park, Sheikh, Hodgins and Shamir

  • ©Ido Arev, Hyun Soo Park, Yaser Sheikh, Jessica K. Hodgins, and Ariel Shamir

Conference:


Type:


Title:

    Automatic editing of footage from multiple social cameras

Session/Category Title:   Video Applications


Presenter(s)/Author(s):


Moderator(s):



Abstract:


    We present an approach that takes multiple videos captured by social cameras—cameras that are carried or worn by members of the group involved in an activity—and produces a coherent “cut” video of the activity. Footage from social cameras contains an intimate, personalized view that reflects the part of an event that was of importance to the camera operator (or wearer). We leverage the insight that social cameras share the focus of attention of the people carrying them. We use this insight to determine where the important “content” in a scene is taking place, and use it in conjunction with cinematographic guidelines to select which cameras to cut to and to determine the timing of those cuts. A trellis graph representation is used to optimize an objective function that maximizes coverage of the important content in the scene, while respecting cinematographic guidelines such as the 180-degree rule and avoiding jump cuts. We demonstrate cuts of the videos in various styles and lengths for a number of scenarios, including sports games, street performances, family activities, and social get-togethers. We evaluate our results through an in-depth analysis of the cuts in the resulting videos and through comparison with videos produced by a professional editor and existing commercial solutions.

References:


    1. Agarwal, S., Furukawa, Y., Snavely, N., Simon, I., Curless, B., Seitz, S. M., and Szeliski, R. 2011. Building rome in a day. Communications of the ACM. Google ScholarDigital Library
    2. Ballan, L., Brostow, G. J., Puwein, J., and Pollefeys, M. 2010. Unstructured video-based rendering: interactive exploration of casually captured videos. ACM Transactions on Graphics. Google ScholarDigital Library
    3. Bao, X., and Choudhury, R. R. 2010. Movi: Mobile phone based video highlights via collaborative sensing. In Proceedings of the International Conference on Mobile systems, Applications, and Services. Google ScholarDigital Library
    4. Barbieri, M., Agnihotri, L., and Dimitrova, N. 2003. Video summarization: methods and landscape. In Proceedings of the SPIE Internet Multimedia Management Systems.Google Scholar
    5. Berthouzoz, F., Li, W., and Agrawala, M. 2012. Tools for placing cuts and transitions in interview video. ACM Transactions on Graphics. Google ScholarDigital Library
    6. Cricri, F., Curcio, I. D. D., Mate, S., Dabov, K., and Gabbouj, M. 2012. Sensor-based analysis of user generated video for multi-camera video remixing. In Advances in Multimedia Modeling. Google ScholarDigital Library
    7. Dale, K., Shechtman, E., Avidan, S., and Pfister, H. 2012. Multi-video browsing and summarization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Large-Scale Video Search and Mining.Google Scholar
    8. Dmytryk, E. 1984. On Film Editing: An Introduction to the Art of Film Construction. Focal Press.Google Scholar
    9. Fathi, A., Hodgins, J., and Rehg, J. 2012. Social interactions: A first-person perspective. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
    10. Gleicher, M. L., and Liu, F. 2007. Re-cinematography: Improving the camera dynamics of casual video. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
    11. Hata, T., Hirose, T., and Tanaka, K. 2000. Skimming multiple perspective video using tempo-spatial importance measures. In Proceedings of Visual Database Systems. Google ScholarDigital Library
    12. He, L.-w., Cohen, M. F., and Salesin, D. H. 1996. The virtual cinematographer: A paradigm for automatic real-time camera control and directing. ACM Transactions on Graphics.Google Scholar
    13. Heck, R., Wallick, M., and Gleicher, M. 2007. Virtual videography. ACM Transactions on Multimedia Computing, Communications, and Applications. Google ScholarDigital Library
    14. Jain, P., Manweiler, J., Acharya, A., and Beaty, K. 2013. Focus: Clustering crowdsourced videos by line-of-sight. In Proceedings of the International Conference on Embedded Networked Sensor Systems. Google ScholarDigital Library
    15. Kim, K., Grundmann, M., Shamir, A., Matthews, I., Hodgins, J., and Essa, I. 2010. Motion field to predict play evolution in dynamic sport scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
    16. Kumar, K., Prasad, S., Banwral, S., and Semwa, V. 2010. Sports video summarization using priority curve algorithm. International Journal on Computer Science and Engineering.Google Scholar
    17. Lee, Y. J., Ghosh, J., and Grauman, K. 2012. Discovering important people and objects for egocentric video summarization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
    18. Lepetit, V., Moreno-Noguer, F., and Fua, P. 2009. EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision. Google ScholarDigital Library
    19. Lu, Z., and Grauman, K. 2013. Story-driven summarization for egocentric video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
    20. Machnicki, E. 2002. Virtual director: Automating a webcast. In Proceedings of the SPIE Multimedia Computing and Networking.Google Scholar
    21. Money, A., and Agius, H. 2008. Video summarisation: A conceptual framework and survey of the state of the art. Journal of Visual Communications and Image Representation. Google ScholarDigital Library
    22. Park, H. S., Jain, E., and Sheikh, Y. 2012. 3D social saliency from head-mounted cameras. In Advances in Neural Information Processing Systems.Google Scholar
    23. Ponto, K., Kohlmann, J., and Gleicher, M. 2012. Effective replays and summarization of virtual experiences. IEEE Transactions on Visualization and Computer Graphics. Google ScholarDigital Library
    24. Pundik, D., and Moses, Y. 2010. Video synchronization using temporal signals from epipolar lines. In Proceedings of the European Conference on Computer Vision. Google ScholarDigital Library
    25. Rui, Y., He, L., Gupta, A., and Liu, Q. 2001. Building an intelligent camera management system. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
    26. Shrestha, P., de With, P. H., Weda, H., Barbieri, M., and Aarts, E. H. 2010. Automatic mashup generation from multiple-camera concert recordings. In Proceedings of the ACM International Conference on Multimedia. Google ScholarDigital Library
    27. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics. Google ScholarDigital Library
    28. Sumec, S. 2006. Multi camera automatic video editing. Computer Vision and Graphics.Google Scholar
    29. Takemae, Y., Otsuka, K., and Mukawa, N. 2004. Impact of video editing based on participants’ gaze in multiparty conversation. In ACM CHI Extended Abstracts on Human Factors in Computing Systems. Google ScholarDigital Library
    30. Taskiran, C., and Delp, E. 2005. Video summarization. Digital Image Sequence Processing, Compression, and Analysis.Google Scholar
    31. Truong, B., and Venkatesh, S. 2007. Video abstraction: A systematic review and classification. ACM Transactions on Multimedia Computing, Communications, and Applications. Google ScholarDigital Library
    32. Wardrip-Fruin, N., and Harrigan, P. 2004. First person: New media as story, performance, and game. MIT Press. Google ScholarDigital Library
    33. Zsombori, V., Frantzis, M., Guimaraes, R. L., Ursu, M. F., Cesar, P., Kegel, I., Craigie, R., and Bulterman, D. C. 2011. Automatic generation of video narratives from shared UGC. In Proceedings of the ACM Conference on Hypertext and Hypermedia. Google ScholarDigital Library


ACM Digital Library Publication:



Overview Page: