“Blue screen matting” by Smith and Blinn

A classical problem of imaging—the matting problem—is separation of a non-rectangular foreground image from a (usually) rectangular background image—for example, in a film frame, extraction of an actor from a background scene to allow substitution of a different background. Of the several attacks on this difficult and persistent problem, we discuss here only the special case of separating a desired foreground image from a background of a constant, or almost constant, backing color. This backing color has often been blue, so the problem, and its solution, have been called blue screen matting. However, other backing colors, such as yellow or (increasingly) green, have also been used, so we often generalize to constant color matting. The mathematics of constant color matting is presented and proven to be unsolvable as generally practiced. This, of course, flies in the face of the fact that the technique is commonly used in film and video, so we demonstrate constraints on the general problem that lead to solutions, or at least significantly prune the search space of solutions. We shall also demonstrate that an algorithmic solution is possible by allowing the foreground object to be shot against two constant backing colors—in fact, against two completely arbitrary backings so long as they differ everywhere.

1. BEYER, W. Traveling Matte Photography and the Blue Screen System. American Cinematographer, May 1964, p. 266. The second of a four-part series.
2. BLINN, J. F. Jim Blinn’s Corner: Compositing Part 1: Theory. IEEE Computer Graphics & Applications, September 1994, pp. 83-87.
3. BLINN, J. F. Jim Blinn’s Corner: Compositing Part 2: Practice. IEEE Computer Graphics & Applications, November 1994, pp. 78-82.
4. DADOURIAN, A. Method and Apparatus for Compositing Video Images. U. S. Patent 5,343,252, August 30, 1994.
5. FELLINGER, D. F. Method and Apparatus for Applying Correction to a Signal Used to Modulate a Background Video Signal to be Combined with a Foreground Video Signal. U. S. Patent 5,202,762, April 13, 1993.
6. FIELDING, R. The Technique of Special Effects Cinematography. Focal/Hastings House, London, 3rd edition, 1972, pp. 220-243.
7. LANCZOS, C. Applied Analysis. Prentice Hall, Inc., Englewood Cliffs, NJ, 1964, pp. 156-161.
8. MISHIMA, Y. A Software Chromakeyer Using Polyhedric Slice. Proceedings of NICOGRAPH 92 (1992), pp. 44-52 (Japanese). See http://206.155.32.1/us/primatte/whitepaper.
9. PORTER, T. and DUFF, T. Compositing Digital Images. Proceedings of SIGGRAPH 84 (Minneapolis, Minnesota, July 23-27, 1984). In Computer Graphics 18, 3 (July 1984), pp. 253-259.
10. PORTER, T. Matte Box Design. Lucasfilm Technical Memo 63, November 1986. Not built.
11. PRESS, W. H., TEUKOLSKY, S. A., VETTERLING, W. T., and FLANNERY, B. P. Numerical Recipes in C. Cambridge University Press, Cambridge, 1988, p. 59.
12. SMITH, A. R. Analysis of the Color-Difference Technique. Technical Memo 30, Lucasfilm Ltd., March 1982.
13. SMITH, A. R. Math of Mattings. Technical Memo 32, Lucasfilm Ltd., April 1982.
14. SMITH, A. R. Image Compositing Fundamentals. Technical Memo 4, Microsoft Corporation, June 1995.
15. SMITH, A. R. Alpha and the History of Digital Compositing. Technical Memo 7, Microsoft Corporation, August 1995.
16. VLAHOS, P. Composite Photography Utilizing Sodium Vapor Illumination. U. S. Patent 3,095,304, May 15, 1958. Expired.
17. VLAHOS, P. Composite Color Photography. U. S. Patent 3,158,477, November 24, 1964. Expired.
18. VLAHOS, P. Electronic Composite Photography. U. S. Patent 3,595,987, July 27, 1971. Expired.
19. VLAHOS, P. Electronic Composite Photography with Color Control. U. S. Patent 4,007,487, February 8, 1977. Expired.
20. VLAHOS, P. Comprehensive Electronic Compositing System. U. S. Patent 4,100,569, July 11, 1978. Expired.
21. VLAHOS, P. and TAYLOR, B. Traveling Matte Composite Photography. American Cinematographer Manual. American Society of Cinematographers, Hollywood, 7th edition, 1993, pp. 430-445.
22. VLAHOS, P. Comprehensive Electronic Compositing System. U. S. Patent 4,344,085, August 10, 1982.
23. VLAHOS, P. Encoded Signal Color Image Compositing. U. S. Patent 4,409,611, October 11, 1983.
24. VLAHOS, P., VLAHOS, P. and FELLINGER, D. F. Automatic Encoded Signal Color Image Compositing. U. S. Patent 4,589,013, May 13, 1986.
25. VLAHOS, P. Comprehensive Electronic Compositing System. U. S. Patent 4,625,231, November 25, 1986.