“Image-based rendering in the gradient domain” by Kopf, Langguth, Scharstein, Szeliski and Goesele – ACM SIGGRAPH HISTORY ARCHIVES

“Image-based rendering in the gradient domain” by Kopf, Langguth, Scharstein, Szeliski and Goesele

  • 2013 SA Technical Papers_Kopf_Image-Based Rendering in the Gradient Domain

Conference:


Type(s):


Title:

    Image-based rendering in the gradient domain

Session/Category Title:   HDR & IBR

Presenter(s)/Author(s):


Abstract:


    We propose a novel image-based rendering algorithm for handling complex scenes that may include reflective surfaces. Our key contribution lies in treating the problem in the gradient domain. We use a standard technique to estimate scene depth, but assign depths to image gradients rather than pixels. A novel view is obtained by rendering the horizontal and vertical gradients, from which the final result is reconstructed through Poisson integration using an approximate solution as a data term. Our algorithm is able to handle general scenes including reflections and similar effects without explicitly separating the scene into reflective and transmissive parts, as required by previous work. Our prototype renderer is fully implemented on the GPU and runs in real time on commodity hardware.

References:


    1. Beery, E., and Yeredor, A. 2008. Blind separation of superimposed shifted images using parameterized joint diagonalization. IEEE Transactions on Image Processing 17, 3, 340–353.
    2. Bergen, J. R., Burt, P. J., Hingorani, R., and Peleg, S. 1992. A three-frame algorithm for estimating two-component image motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 9, 886–896.
    3. Bhat, D. N., and Nayar, S. K. 1998. Stereo and specular reflection. International Journal of Computer Vision 26, 2, 91–106.
    4. Buehler, C., Bosse, M., McMillan, L., Gortler, S. J., and Cohen, M. F. 2001. Unstructured Lumigraph rendering. Proc. SIGGRAPH 2001, 425–432.
    5. Carceroni, R. L., and Kutulakos, K. N. 2002. Multi-view scene capture by surfel sampling: From video streams to nonrigid 3D motion, shape and reflectance. International Journal of Computer Vision 49, 2/3, 175–214.
    6. Chaurasia, G., Duchene, S., Sorkine-Hornung, O., and Drettakis, G. 2013. Depth synthesis and local warps for plausible image-based navigation. ACM Transactions on Graphics 32, 3, Article no. 30.
    7. Chen, S., and Williams, L. 1993. View interpolation for image synthesis. Proc. SIGGRAPH ’93, 279–288.
    8. Criminisi, A., Kang, S. B., Swaminathan, R., Szeliski, R., and Anandan, P. 2005. Extracting layers and analyzing their specular properties using epipolar-plane-image analysis. Computer Vision and Image Understanding 97, 1, 51–85.
    9. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. Proc. SIGGRAPH ’96, 11–20.
    10. Diamant, Y., and Schechner, Y. Y. 2008. Overcoming visual reverberations. Proc. Computer Vision and Pattern Recognition (CVPR’08), 1–8.
    11. Eisemann, M., De Decker, B., Magnor, M., Bekaert, P., de Aguiar, E., Ahmed, N., Theobalt, C., and Sellent, A. 2008. Floating textures. Computer Graphics Forum (Proc. Eurographics 2008) 27, 2, 409–418.
    12. Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., and Szeliski, R. 2010. Ambient point clouds for view interpolation. ACM Transactions on Graphics (Proc. SIGGRAPH 2010) 29, 4, Article no. 95.
    13. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. Proc. SIGGRAPH ’96, 43–54.
    14. Hirschmüller, H. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2, 328–341.
    15. Irani, M., Rousso, B., and Peleg, S. 1994. Computing occluding and transparent motions. International Journal of Computer Vision 12, 1, 5–16.
    16. Ju, S. X., Black, M. J., and Jepson, A. D. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. Proc. Computer Vision and Pattern Recognition (CVPR ’96), 307–314.
    17. Kolmogorov, V., and Zabih, R. 2002. Multi-camera scene reconstruction via graph cuts. Proc. European Conference on Computer Vision 2002 (ECCV 2002), 82–96.
    18. Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. Proc. Computer Vision and Pattern Recognition 2004 (CVPR 2004) 1, 306–313.
    19. Levoy, M., and Hanrahan, P. 1996. Light field rendering. Proc. SIGGRAPH ’96, 31–42.
    20. Linz, C., Lipski, C., and Magnor, M. 2010. Multi-image interpolation based on graph-cuts and symmetric optical flow. Proc. Vision, Modeling and Visualization 2010 (VMV 2010), 115–122.
    21. Loop, C., and Zhang, Z. 1999. Computing rectifying homographies for stereo vision. Proc. Computer Vision and Pattern Recognition (CVPR ’99), 125–131.
    22. Mahajan, D., Huang, F.-C., Matusik, W., Ramamoorthi, R., and Belhumeur, P. 2009. Moving gradients: a path-based method for plausible image interpolation. ACM Transactions on Graphics (Proc. SIGGRAPH 2009) 28, 3, Article no. 42.
    23. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics (Proc. SIGGRAPH 2003) 22, 3, 313–318.
    24. Popescu, V., Mei, C., Dauble, J., and Sacks, E. 2006. Reflected-scene impostors for realistic reflections at interactive rates. Computer Graphics Forum (Proc. Eurographics 2006) 25, 3, 313–322.
    25. Schechner, Y. Y., Shamir, J., and Kiryati, N. 1999. Polarization-based decorrelation of transparent layers: The inclination angle of an invisible surface. Proc. International Conference on Computer Vision (ICCV ’99), 814–819.
    26. Schechner, Y. Y., Kiryati, N., and Shamir, J. 2000. Blind recovery of transparent and semireflected scenes. Proc. Computer Vision and Pattern Recognition (CVPR 2000), 38–43.
    27. Shade, J., Gortler, S., He, L., and Szeliski, R. 1998. Layered depth images. Proc. SIGGRAPH ’98, 231–242.
    28. Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., Carnegie Mellon University.
    29. Shizawa, M., and Mase, K. 1991. A unified computational theory of motion transparency and motion boundaries based on eigenenergy analysis. Proc. Computer Vision and Pattern Recognition (CVPR ’91), 289–295.
    30. Shum, H.-Y., Chan, S.-C., and Kang, S. B. 2007. Image-Based Rendering. Springer, New York, NY.
    31. Sinha, S. N., Steedly, D., and Szeliski, R. 2009. Piecewise planar stereo for image-based rendering. Proc. International Conference on Computer Vision (ICCV 2009), 1881–1888.
    32. Sinha, S. N., Kopf, J., Goesele, M., Scharstein, D., and Szeliski, R. 2012. Image-based rendering for scenes with reflections. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4, Article no. 100.
    33. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (Proc. SIGGRAPH 2006) 25, 3, 835–846.
    34. Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. Proc. Computer Vision and Pattern Recognition (CVPR 2000), 246–253.
    35. Tsin, Y., Kang, S. B., and Szeliski, R. 2006. Stereo matching with linear superposition of layers. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2, 290–301.
    36. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics (Proc. SIGGRAPH 2004) 23, 3, 600–608.

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org