“AutoKey: human assisted key extraction” by Mitsunaga, Yokoyama and Totsuka

Key extraction is an inverse problem of finding the foreground, the background, and the alpha from an image and some hints. Although the chromakey solves this for a limited case (single background color), this is often too restrictive in practical situations. When the extraction from arbitrary background is necessary, this is currently done by a time consuming manual task. In order to reduce the operator load, attempts have been made to assist operators using either color space or image space information. However, existing approaches have their limitations. Especially, they leave too much work to operators. In this paper, we present a key extraction algorithm which for the first time, addresses the problem quantitatively. We first derive a partial differential equation that relates the gradient of an image to the alpha values. We then describe an efficient algorithm that provides the alpha values as the solution of the equation. Along with our accurate motion estimation technique, it produces correct alpha values almost everywhere, leaving little work to operators. We also show that a careful design of the algorithm and the data representation greatly improves human interaction. At every step of the algorithm, human interaction is possible and it is intuitive.

1. Anderson, T. W., Introduction to Multivariate Statistical Analysis, John Wiley & Sons, 1958.]]
2. Attneave, F., “Some Informational Aspects of Visual Perception”, Psychological Review, Vol.61, pp.183-193, 1954.]]
3. Dadourian, A., “Compositor for Video Images”, patent in U.S.A., US5343252-A, 1994.]]
4. Etoh, M. and Shirai, Y., “Automatic Extraction of Complex Objects Using Region Segmentation”, NICOGRAPH’92, pp.8-17, 1992.]]
5. Inui, T., “A Model of Human Visual Memory”, 6th Scandinavian Conference on Image Analysis, pp.325-332, 1989.]]
6. Inoue, S., “An Object Extraction Method for Image Synthesis”, IEICE Trans., Vol.J74-D-II, No.10, pp.1411-1418, 1991.]]
7. Inoue, S. and Koyama, H., “An Extraction and Composing Method for Moving Image Synthesis”, Journal of lTEJ, Vol.47, No.7, pp999.-1005, 1993.]]
8. Jain, A. K., Fundamentals of Digital Image Processing, Prentice-Hall, 1989.]]
9. Kass, M., Witikin, A. and Terzopoulos, D., “Snakes : Active Contour Models”,International Journal of Computer Vision, Vol.1, No.3, pp.321-331, 1988.]]
10. Klinker, G. J., Shafer, S. A. and Kanade, T., “The Measurement of Highlight in Color Images”, International Journal of Computer Vision, Vol.2, pp.7-32, 1988.]]
11. Mishima, Y., “A Software Chromakeyer Using Polyhedric Slice”, NICOGRAPH’92, pp.44-52, 1992.]]
12. Porter, T. and Duff, T., “Compositing Digital Images”, Computer Graphics, Vo1.18, No.3, pp.253-259, 1984.]]
13. Robinson, G. S., “Color Edge Detection”, Optical Engineering, Vo1.16, No.5, pp.479-484, 1977.]]
14. Sezan, M. I. and Lagendijk, R. L., Motion Analysis and Image Sequence Processing, Kluwer Academic Publishers, 1993.]]
15. Trahanias, E E. and Venetsanopoulos, A. N., “Color Edge Detection Using Vector Order S tatistics”,IEEE Trans. Image Processing, Vol.2, No.2, pp.259-264, 1993.]]
16. Ueda, N., Mase, K. and Suenaga, Y., “A Contour Tracking Method Using Elastic Contour Model and Energy Minimization Approach”, IEICE Trans., Vol.J75-D-II, No.l, pp.111- 120, 1992.]]
17. Vlahos, E, “Combining Method for Colour Video Signals”, patent in U.S.A., US4625231-A, 1986.]]
18. Wong, R. Y. and Hall, E. L., “Sequential Hierarchical Scene Matching”, IEEE Trans. Comput., Vol.C-27, No.4, pp.359- 366, 1978.]]
19. Zenzo, S. D., “A Note on the Gradient of a Multi-Image”, Computer Vision, Graphics, and Image Processing, Vol.33, pp.116-125, 1986.]]
20. MPEG Video Committee, “Information Technology- Generic Coding of Moving Pictures and Associated Audio”, ISO/IEC IS 13818-2, Nov. 1994.]]