“Video diff: highlighting differences between similar actions in videos”
Conference:
Type(s):
Title:
- Video diff: highlighting differences between similar actions in videos
Session/Category Title: Video Processing
Presenter(s)/Author(s):
Abstract:
When looking at videos of very similar actions with the naked eye, it is often difficult to notice subtle motion differences between them. In this paper we introduce video diffing, an algorithm that highlights the important differences between a pair of video recordings of similar actions. We overlay the edges of one video onto the frames of the second, and color the edges based on a measure of local dissimilarity between the videos. We measure dissimilarity by extracting spatiotemporal gradients from both videos and calculating how dissimilar histograms of these gradients are at varying spatial scales. We performed a user study with 54 people to compare the ease with which users could use our method to find differences. Users gave our method an average grade of 4.04 out of 5 for ease of use, compared to 3.48 and 2.08 for two baseline approaches. Anecdotal results also show that our overlays are useful in the specific use cases of professional golf instruction and analysis of animal locomotion simulations.
References:
1. Bae, S., et al. 2010. Computational rephotography. ACM Trans. Graph. 29, 3, 24:1–24:15.
2. Caspi, Y., and Irani, M. 2002. Spatio-temporal alignment of sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 11, 1409–1424.
3. Coach’s Eye, 2015. https://www.coachseye.com.
4. Dalal, N., and Triggs, B. 2005. Histograms of oriented gradients for human detection. 886–893.
5. Dartfish, 2015. http://www.dartfish.com/Express.
6. Diego, F., et al. 2013. Joint spatio-temporal alignment of sequences. IEEE Transactions on Multimedia 15, 6, 1377–1387.
7. Dollar, P., Rabaud, V., Cottrell, G., and Belongie, S. 2005. Behavior recognition via sparse spatio-temporal features. 65–72.
8. Evangelidis, G. D., and Bauckhage, C. 2013. Efficient subframe video alignment using short descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 10, 2371–2386.
9. Grauman, K., and Darrell, T. 2007. The pyramid match kernel: Efficient learning with sets of features. J. Mach. Learn. Res. 8, 725–760.
10. Kläser, A., Marszałek, M., and Schmid, C. 2008. A spatio-temporal descriptor based on 3d-gradients. 995–1004.
11. Laptev, I. 2005. On space-time interest points. Int. J. Comput. Vision 64, 2-3, 107–123.
12. Lazebnik, S., et al. 2006. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2169–2178.
13. Liu, C., et al. 2008. Sift flow: Dense correspondence across different scenes. Proceedings of the 10th European Conference on Computer Vision: Part III, 28–42.
14. Padua, F. L., et al. 2008. Linear sequence-to-sequence alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 2, 304–320.
15. Sand, P., and teller, S. 2004. Video matching. ACM Transactions on Graphics 22, 3, 592–599.
16. Ubersense, 2015. http://www.ubersense.com/.
17. Ukrainitz, Y., and Irani, M. 2006. Aligning sequences and actions by maximizing space-time correlations. ECCV (3), 538–550.
18. Wadhwa, N., et al. 2013. Phase-based video motion processing. ACM Trans. Graph. (Proceedings SIGGRAPH 2013) 32, 4.
19. Wampler, K., et al. 2014. Generalizing locomotion style to new animals with inverse optimal regression. ACM Trans. Graph. 33, 4, 1–11.
20. Wang, H., et al. 2009. Evaluation of local spatio-temporal features for action recognition. 124.1–124.11.
21. Wu, H.-Y., et al. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. (Proceedings SIGGRAPH 2012) 31, 4.
