“Shadow matting and compositing” by Chuang, Goldman, Curless, Salesin and Szeliski

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the shadow is cast onto an arbitrary planar background. We then acquire the photometric and geometric properties of the target scene by sweeping oriented linear shadows (cast by a straight object) across it. From these shadow scans, we can construct a shadow displacement map without requiring camera or light source calibration. This map can then be used to deform the original shadow matte. We demonstrate our approach for both indoor scenes with controlled lighting and for outdoor scenes using natural lighting.

1. BERTALMIO, M., SAPIRO, G., CASELLES, V., AND BALLESTER, C. 2000. Image inpainting. In Proceedings of SIGGRAPH 2000, 417–424. Google ScholarDigital Library
2. BOUGUET, J.-Y., AND PERONA, P. 1998. 3D photography on your desk. In Proceedings of IEEE International Conference on Computer Vision (ICCV 98), 43–50. Google Scholar
3. CHUANG, Y.-Y., ZONGKER, D. E., HINDORFF, J., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, 121–130. Google Scholar
4. CHUANG, Y.-Y., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2001. A Bayesian approach to digital matting. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2001), vol. II, 264–271.Google Scholar
5. CHUANG, Y.-Y., AGARWALA, A., CURLESS, B., SALESIN, D. H., AND SZELISKI, R. 2002. Video matting of complex scenes. ACM Transactions on Graphics 21, 3, 243–248. Google ScholarDigital Library
6. DEBEVEC, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of SIGGRAPH 98, 189–198. Google Scholar
7. FINLAYSON, G. D., HORDLEY, S. D., AND DREW, M. S. 2002. Removing shadows from images. In Proceedings of European Conference on Computer Vision (ECCV 2002), vol. 2353 of LNCS, 823–836. Google ScholarDigital Library
8. KUMAR, R., ANANDAN, P., AND HANNA, K. 1994. Direct recovery of shape from multiple views: A parallax based approach. In Twelfth International Conference on Pattern Recognition (ICPR’94), 685–688.Google Scholar
9. LORENSEN, W. E., AND CLINE, H. E. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In Computer Graphics (Proceedings of ACM SIGGRAPH ’90), 163–169. Google Scholar
10. MATSUSHITA, Y., KANG, S. B., LIN, S., SHUM, H.-Y., AND TONG, X. 2002. Lighting interpolation by shadow morphing using intrinsic lumigraphs. In Proceedings of Pacific Graphics 2002, 58–65. Google ScholarDigital Library
11. PELLACINI, F., TOLE, P., AND GREENBERG, D. P. 2002. A user interface for interactive cinematic shadow design. ACM Transactions on Graphics 21, 3, 563–566. Google ScholarDigital Library
12. PERONA, P., AND MALIK, J. 1990. Scale space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 7 (July), 629–639. Google ScholarDigital Library
13. PETROVIC, L., FUJITO, B., WILLIAMS, L., AND FINKELSTEIN, A. 2000. Shadows for cel animation. In Siggraph 2000, Computer Graphics Proceedings, K. Akeley, Ed., Annual Conference Series, 511–516. Google Scholar
14. PORTER, T., AND DUFF, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of ACM SIGGRAPH ’84), 253–259. Google Scholar
15. RUZON, M. A., AND TOMASI, C. 2000. Alpha estimation in natural images. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 18–25.Google ScholarCross Ref
16. SAVARESE, S., RUSHMEIER, H., BERNARDINI, F., AND PERONA, P. 2001. Shadow carving. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 190–197.Google ScholarCross Ref
17. SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Proceedings of ACM SIGGRAPH 96, 259–268. Google Scholar
18. SZELISKI, R., AVIDAN, S., AND ANANDAN, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proceedings of Computer Vision and Pattern Recognition (CVPR 2000), 246–253.Google ScholarCross Ref
19. WEISS, Y. 2001. Deriving intrinsic images from image sequences. In Proceedings of IEEE International Conference on Computer Vision (ICCV 2001), 68–75.Google ScholarCross Ref
20. WRIGHT, S. 2001. Digital Compositing for Film and Video. Focal Press. Google Scholar
21. ZONGKER, D. E., WERNER, D. M., CURLESS, B., AND SALESIN, D. H. 1999. Environment matting and compositing. In Proceedings of ACM SIGGRAPH 99, 205–214. Google Scholar