“Ambient point clouds for view interpolation” by Goesele, Ackermann, Fuhrmann, Haubold, Klowsky, et al. …

  • ©Michael Goesele, Jens Ackermann, Simon Fuhrmann, Carsten Haubold, Ronny Klowsky, Drew Steedly, and Richard Szeliski




    Ambient point clouds for view interpolation



    View interpolation and image-based rendering algorithms often produce visual artifacts in regions where the 3D scene geometry is erroneous, uncertain, or incomplete. We introduce ambient point clouds constructed from colored pixels with uncertain depth, which help reduce these artifacts while providing non-photorealistic background coloring and emphasizing reconstructed 3D geometry. Ambient point clouds are created by randomly sampling colored points along the viewing rays associated with uncertain pixels. Our real-time rendering system combines these with more traditional rigid 3D point clouds and colored surface meshes obtained using multiview stereo. Our resulting system can handle larger-range view transitions with fewer visible artifacts than previous approaches.


    1. Buehler, C., Bosse, M., McMillan, L., Gortler, S. J., and Cohen, M. F. 2001. Unstructured lumigraph rendering. Proc. SIGGRAPH, 425–432. Google ScholarDigital Library
    2. Chen, S. E., and Williams, L. 1993. View interpolation for image synthesis. In Proc. SIGGRAPH, 279–288. Google ScholarDigital Library
    3. Evers-Senne, J.-F., and Koch, R. 2003. Image based interactive rendering with view dependent geometry. In Proc. EG, 573–582.Google Scholar
    4. Fitzgibbon, A. W., Wexler, Y., and Zisserman, A. 2005. Image-based rendering using image-based priors. IJCV 63, 2, 141–151. Google ScholarDigital Library
    5. Furukawa, Y., Curless, B., Seitz, S. M., and Szeliski, R. 2009. Reconstructing building interiors from images. In Proc. ICCV.Google Scholar
    6. Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S. M. 2007. Multi-view stereo for community photo collections. In Proc. ICCV.Google Scholar
    7. Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In Proc. SIGGRAPH, 43–54. Google ScholarDigital Library
    8. Hammer, P. L., Hansen, P., and Simeone, B. 1984. Roof duality, complementation and persistency in quadratic 0–1 optimization. Mathematical Programming 28, 121–155.Google ScholarCross Ref
    9. Heigl, B., Koch, R., Pollefeys, M., Denzler, J., and Van Gool, L. J. 1999. Plenoptic modeling and rendering from image sequences taken by hand-held camera. In Proc. DAGM, 94–101. Google ScholarDigital Library
    10. Hofsetz, C., Ng, K., Chen, G., McGuinness, P., Max, N., and Liu, Y. 2004. Image-based rendering of range data with estimated depth uncertainty. CG&A 24, 4, 34–41. Google ScholarDigital Library
    11. Hofsetz, C., Chen, G., Max, N., Ng, K. C., Liu, Y., Hong, L., and McGuinness, P. 2004. Light-field rendering using colored point clouds—a dual-space approach. Presence: Teleoperators & Virtual Environments 13, 6, 726–741. Google ScholarDigital Library
    12. Hornung, A., and Kobbelt, L. 2009. Interactive pixelaccurate free viewpoint rendering from images with silhouette aware sampling. Computer Graphics Forum 28, 8, 2090–2103.Google ScholarCross Ref
    13. Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proc. SIGGRAPH, 31–42. Google ScholarDigital Library
    14. Lhuillier, M., and Quan, L. 2003. Image-based rendering by joint view triangulation. TCSVT 13, 11, 1051–1063. Google ScholarDigital Library
    15. McMillan, L., and Bishop, G. 1995. Plenoptic modeling: an image-based rendering system. In Proc. SIGGRAPH, 39–46. Google ScholarDigital Library
    16. Narayanan, P., Rander, P., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. ICCV, 3–10. Google ScholarDigital Library
    17. Ng, K. C., Trivedi, M. M., and Ishiguro, H. 2002. Generalized multiple baseline stereo and direct virtual view synthesis using range-space search, match, and render. IJCV 47, 1–3, 131–147. Google ScholarDigital Library
    18. Pulli, K., Cohen, M., Duchamp, T., Hoppe, H., Shapiro, L., and Stuetzle, W. 1997. View-based rendering: Visualizing real objects from scanned range and color data. In Proc. EGWR, 23–34. Google ScholarDigital Library
    19. Seitz, S. M., and Dyer, C. R. 1996. View morphing. In Proc. SIGGRAPH, 21–30. Google ScholarDigital Library
    20. Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. In Proc. SIGGRAPH, 231–242. Google ScholarDigital Library
    21. Shahrokni, A., Mei, C., Torr, P. H. S., and Reid, I. D. 2008. From visual query to visual portrayal. In Proc. BMVC.Google Scholar
    22. Sinha, S. N., Steedly, D., and Szeliski, R. 2009. Piecewise planar stereo for image-based rendering. In Proc. ICCV.Google Scholar
    23. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: Exploring photo collections in 3D. ACM TOG 25, 3, 835–846. Google ScholarDigital Library
    24. Snavely, N., Garg, R., Seitz, S. M., and Szeliski, R. 2008. Finding paths through the world’s photos. ACM TOG 27, 3, 11–21. Google ScholarDigital Library
    25. Snavely, N., Seitz, S. M., and Szeliski, R. 2008. Skeletal graphs for efficient structure from motion. In Proc. CVPR.Google Scholar
    26. Szeliski, R. 2006. Locally adapted hierarchical basis preconditioning. ACM TOG 25, 3, 1135–1143. Google ScholarDigital Library
    27. Xu, H., and Chen, B. 2004. Stylized rendering of 3D scanned real world environments. In Proc. NPAR, 25–34. Google ScholarDigital Library
    28. Xu, H., Gossett, N., and Chen, B. 2004. Pointworks: Abstraction and rendering of sparsely scanned outdoor environments. In Proc. EGSR, 45–52. Google ScholarDigital Library
    29. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM TOG 23, 3, 600–608. Google ScholarDigital Library
    30. Zomet, A., Feldman, D., Peleg, S., and Weinshall, D. 2003. Mosaicing new views: The crossed-slits projection. TPAMI, 741–754. Google ScholarDigital Library

ACM Digital Library Publication: