“Discontinuity-aware video object cutout”
Conference:
Type(s):
Title:
- Discontinuity-aware video object cutout
Session/Category Title: Video and Image Manipulation
Presenter(s)/Author(s):
Abstract:
Existing video object cutout systems can only deal with limited cases. They usually require detailed user interactions to segment real-life videos, which often suffer from both inseparable statistics (similar appearance between foreground and background) and temporal discontinuities (e.g. large movements, newly-exposed regions following disocclusion or topology change).In this paper, we present an efficient video cutout system to meet this challenge. A novel directional classifier is proposed to handle temporal discontinuities robustly, and then multiple classifiers are incorporated to cover a variety of cases. The outputs of these classifiers are integrated via another classifier, which is learnt from real examples. The foreground matte is solved by a coherent matting procedure, and remaining errors can be removed easily by additive spatio-temporal local editing. Experiments demonstrate that our system performs more robustly and more intelligently than existing systems in dealing with various input types, thus saving a lot of user labor and time.
References:
1. Agarwala, A., Hertzmann, A., Salesin, D. H., and Seitz, S. M. 2004. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graphics 23 (August), 584–591.
2. Apostoloff, N., and Fitzgibbon, A. 2004. Bayesian Video Matting Using Learnt Image Priors. In CVPR, 407–414.
3. Bai, X., and Sapiro, G. 2007. A geodesic framework for fast interactive image and video segmentation and matting. In ICCV, 1–8.
4. Bai, X., Wang, J., Simons, D., and Sapiro, G. 2009. Video snapcut: robust video object cutout using localized classifiers. ACM Trans. Graphics 28 (July), 70:1–70:11.
5. Bai, X., Wang, J., and Sapiro, G. 2010. Dynamic color flow: a motion-adaptive color model for object segmentation in video. In ECCV, 617–630.
6. Bilmes, J. 1998. A gentle tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. Tech. rep., International Computer Science Institute, Berkeley.
7. Chuang, Y.-Y., Agarwala, A., Curless, B., Salesin, D. H., and Szeliski, R. 2002. Video matting of complex scenes. ACM Trans. Graph. 21 (July), 243–248.
8. Gong, M., Wang, L., Yang, R., and Yang, Y.-H. 2010. Real-Time Video matting using Multichannel Poisson Equations. In Graphics Interface, 89–96.
9. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Trans. Graph. 26 (July).
10. Levin, A., Lischinski, D., and Weiss, Y. 2008. A closedform solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 228–242.
11. Li, Y., Sun, J., and Shum, H.-Y. 2005. Video object cut and paste. ACM Trans. Graphics 24 (July), 595–600.
12. Mortensen, E. N., and Barrett, W. A. 1999. Toboggan-Based Intelligent Scissors with a Four Parameter Edge Model. In CVPR.
13. Price, B., Cohen, S., and Morse, B. 2009. Livecut: Learning-based interactive video segmentation by evaluation of multiple propagated cues. In ICCV, 779–786.
14. Rhemann, C., Rother, C., Rav-Acha, A., and Sharp, T. 2008. High resolution matting via interactive trimap segmentation. In CVPR, 1–8.
15. Ross, D. A., Lim, J., Lin, R.-S., and Yang, M.-H. 2008. Incremental learning for robust visual tracking. International Journal of Computer Vision 77 (May), 125–141.
16. Sarim, M., Hilton, A., and Guillemaut, J.-Y. 2009. Non-Parametric patch based video matting. In BMVC, British Machine Vision Association, 98.1–98.11.
17. Tang, Z., Miao, Z., Wan, Y., and Zhang, D. 2011. Video matting via opacity propagation. The Visual Computer (19 April), 1–15.
18. Tong, R.-F., Zhangy, Y., and Ding, M. 2011. Video brush: A novel interface for efficient video cutout. Computer Graphics Forum 30, 2049–2057.
19. Vilalta, R., and Drissi, Y. 2002. A perspective view and survey of meta-learning. Artificial Intelligence Review 18, 77–95.
20. Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graphics 24 (July), 585–594.
21. Yang, L., Sander, P. V., Lawrence, J., and Hoppe, H. 2011. Antialiasing recovery. ACM Trans. Graph. 30 (May), 22:1–22:9.
22. Yin, P., Criminisi, A., Winn, J., and Essa, I. 2007. Tree-based classifiers for bilayer video segmentation. In CVPR, 1–8.
23. Zhong, F., Qin, X., and Peng, Q. 2010. Transductive segmentation of live video with non-stationary background. In CVPR, 2189–2196.
