“Matting and Compositing of Transparent and Refractive Objects” by Yeung, Tang, Brown and Kang

  • ©

Conference:


Type(s):


Title:

    Matting and Compositing of Transparent and Refractive Objects

Presenter(s)/Author(s):


Abstract:


    This article introduces a new approach for matting and compositing transparent and refractive objects in photographs. The key to our work is an image-based matting model, termed the Attenuation-Refraction Matte (ARM), that encodes plausible refractive properties of a transparent object along with its observed specularities and transmissive properties. We show that an object’s ARM can be extracted directly from a photograph using simple user markup. Once extracted, the ARM is used to paste the object onto a new background with a variety of effects, including compound compositing, Fresnel effect, scene depth, and even caustic shadows. User studies find our results favorable to those obtained with Photoshop as well as perceptually valid in most cases. Our approach allows photo editing of transparent and refractive objects in a manner that produces realistic effects previously only possible via 3D models or environment matting.

References:


    1. Agarwal, S., Mallick, S. P., Kriegman, D. J., and Belongie, S. 2004. On refractive optical flow. In Proceedings of the European Conference on Computer Vision ECCV (2). 483–494.
    2. Ben Ezra, M. and Nayar, S. 2003. What does motion reveal about transparency? In Proceedings of the International Conference on Computer Vision (ICCV’03). 1025–1032. 
    3. Bookstein, F. 1989. Principal warps: Thin-plate splines and the decomposition of deformations. Patt. Anal. Math. Intell. 11, 6, 567–585. 
    4. Chuang, Y.-Y., Goldman, D. B., Curless, B., Salesin, D. H., and Szeliski, R. 2003. Shadow matting and compositing. ACM Trans. Graph. 22, 3, 494–500. 
    5. Chuang, Y.-Y., Zongker, D. E., Hindorff, J., Curless, B., Salesin, D. H., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of the Annual SIGGRAPH Conference. 121–130. 
    6. Foley, J. D., van Dam, A., Feiner, S. K., and Hughes, J. F. 1995. Computer Graphics: Principles and Practices. 2nd Ed. Addison-Wesley. 
    7. Gutierrez, D., Lopez-Moreno, J., Fandos, J., Seron, F., Sanchez, M. P., and Reinhard, E. 2008. Depicting procedural caustics in single images. ACM Trans. Graph. 27, 5. 
    8. Hecht, E. 1987. Optics 2nd Ed. Addison Wesley.
    9. Jagnow, R., Dorsey, J., and Rushmeier, H. 2008. Evaluation of methods for approximating shapes used to synthesize 3d solid textures. ACM Trans. Appl. Percept. 4, 4, 1–27. 
    10. Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-Drop pasting. ACM Trans. Graph. 25, 3, 631–637. 
    11. Jimenez, J., Sundstedt, V., and Gutierrez, D. 2009. Screen-Space perceptual rendering of human skin. ACM Trans. Appl. Percept. 6, 4, 1–15.
    12. Khan, E. A., Reinhard, E., Fleming, R. W., and Bulthoff, H. H. 2006. Image-Based material editing. ACM Trans. Graph. 25, 3, 654–663.
    13. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. Graph. 23, 3, 303–308. 
    14. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. 2002. Acquisition and rendering of transparent and refractive objects. In Proceedings of the Eurographics Workshop on Rendering. 
    15. Miyazaki, D. and Ikeuchi, K. 2007. Shape estimation of transparent objects by using inverse polarization ray tracing. Patt. Anal. Math. Intell. 29, 11, 2018–2030.
    16. Morris, N. and Kutulakos, K. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. In Proceedings of the International Conference on Computer Vision (ICCV’07).
    17. Peers, P. and Dutre, P. 2003. Wavelet environment matting. In Proceedings of the 14th Eurographics Workshop on Rendering. 157–166. 
    18. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313–318.
    19. Rother, C., Kolmogorov, V., and Blake, A. 2004. “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 3, 309–314.
    20. Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Trans. Graph. 23, 3, 315–321.
    21. Wang, J. and Cohen, M. F. 2008. Image and Video Matting. Now Publishers Inc., Hanover, MA.
    22. Wexler, Y., Fitzgibbon, A., and Zisserman, A. 2002. Image-Based environment matting. In Proceedings of the EuroGraphics Workshop on Rendering. 289–299.
    23. Wu, T.-P., Tang, C.-K., Brown, M., and Shum, H.-Y. 2007. Natural shadow matting. ACM Trans. Graph. 26, 2.
    24. Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment matting and compositing. In Proceedings of the Annual SIGGRAPH Conference. 205–214.

ACM Digital Library Publication:


Overview Page: