“Deep photo: model-based photograph enhancement and viewing”
Conference:
Type(s):
Title:
- Deep photo: model-based photograph enhancement and viewing
Session/Category Title: Fun with single images
Presenter(s)/Author(s):
Abstract:
In this paper, we introduce a novel system for browsing, enhancing, and manipulating casual outdoor photographs by combining them with already existing georeferenced digital terrain and urban models. A simple interactive registration process is used to align a photograph with such a model. Once the photograph and the model have been registered, an abundance of information, such as depth, texture, and GIS data, becomes immediately available to our system. This information, in turn, enables a variety of operations, ranging from dehazing and relighting the photograph, to novel view synthesis, and overlaying with geographic information. We describe the implementation of a number of these applications and discuss possible extensions. Our results show that augmenting photographs with already available 3D models of the world supports a wide variety of new ways for us to experience and interact with our everyday snapshots.
References:
1. Ashikhmin, M. 2001. Synthesizing natural textures. Proceedings of the 2001 symposium on Interactive 3D graphics (I3D), 217–226. Google Scholar
2. Chen, B., Ramos, G., Ofek, E., Cohen, M., Drucker, S., and Nister, D. 2008. Interactive techniques for registering images to digital terrain and building models. Microsoft Research Technical Report MSR-TR-2008-115.Google Scholar
3. Cho, P. L. 2007. 3D organization of 2D urban imagery. Proceedings of the 36th Applied Imagery Pattern Recognition Workshop, 3–8. Google Scholar
4. Criminisi, A., Reid, I. D., and Zisserman, A. 2000. Single view metrology. International Journal of Computer Vision 40, 2, 123–148. Google ScholarDigital Library
5. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. Proceedings of SIGGRAPH ’96, 11–20. Google Scholar
6. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. Proceedings of SIGGRAPH 2000, 145–156. Google Scholar
7. Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2003) 22, 3, 303–312. Google Scholar
8. E-on Software, 2008. Vue 6 Infinite. http://www.e-onsoftware.com/products/vue/vue_6_infinite.Google Scholar
9. Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. Proceedings of IEEE International Conference on Computer Vision (ICCV) ’99 2, 1033–1038. Google ScholarDigital Library
10. Fattal, R. 2008. Single image dehazing. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008) 27, 3, 73. Google Scholar
11. Früh, C., and Zakhor, A. 2003. Constructing 3D city models by merging aerial and ground views. IEEE Computer Graphics and Applications 23, 6, 52–61. Google ScholarDigital Library
12. Gruen, A., and Huang, T. S. 2001. Calibration and Orientation of Cameras in Computer Vision. Springer-Verlag, Secaucus, NJ, USA. Google Scholar
13. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 4. Google Scholar
14. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. Proceedings of SIGGRAPH 2001, 327–340. Google Scholar
15. Hoiem, D., Efros, A. A., and Hebert, M. 2005. Automatic photo pop-up. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005) 24, 3, 577–584. Google Scholar
16. Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. Proceedings of SIGGRAPH ’97, 225–232. Google Scholar
17. Kaftory, R., Schechner, Y. Y., and Zeevi, Y. Y. 2007. Variational distance-dependent image restoration. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2007, 1–8.Google Scholar
18. Kang, S. B. 1998. Depth painting for image-based rendering applications. Tech. rep., Compaq Cambridge Research Lab.Google Scholar
19. Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 3. Google Scholar
20. Loscos, C., Drettakis, G., and Robert, L. 2000. Interactive virtual relighting of real scenes. IEEE Transactions on Visualization and Computer Graphics 6, 4, 289–305. Google ScholarDigital Library
21. McCartney, E. J. 1976. Optics of the Atmosphere: Scattering by Molecules and Particles. John Wiley and Sons, New York, NY, USA.Google Scholar
22. Narasimhan, S. G., and Nayar, S. K. 2003. Contrast restoration of weather degraded images. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 6, 713–724. Google ScholarCross Ref
23. Narasimhan, S. G., and Nayar, S. K. 2003. Interactive (de)weathering of an image using physical models. IEEE Workshop on Color and Photometric Methods in Computer Vision.Google Scholar
24. NASA, 2008. The landsat program. http://landsat.gsfc.nasa.gov/.Google Scholar
25. NASA, 2008. Shuttle radar topography mission. http://www2.jpl.nasa.gov/srtm/.Google Scholar
26. Nayar, S. K., and Narasimhan, S. G. 1999. Vision in bad weather. Proceedings of IEEE International Conference on Computer Vision (ICCV) ’99, 820–827. Google ScholarDigital Library
27. Nister, D., and Stewenius, H. 2007. A minimal solution to the generalised 3-point pose problem. Journal of Mathematical Imaging and Vision 27, 1, 67–79. Google ScholarDigital Library
28. Oakley, J. P., and Satherley, B. L. 1998. Improving image quality in poor visibility conditions using a physical model for contrast degradation. IEEE Transactions on Image Processing 7, 2, 167–179. Google ScholarDigital Library
29. Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. Proceedings of ACM SIGGRAPH 2001, 433–442. Google Scholar
30. Reichmann, M., 2001. The art of photography. http://www.luminous-landscape.com/essays/theartof.shtml.Google Scholar
31. Schechner, Y. Y., and Averbuch, Y. 2007. Regularized image recovery in scattering media. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 9, 1655–1660. Google ScholarDigital Library
32. Schechner, Y. Y., Narasimhan, S. G., and Nayar, S. K. 2003. Polarization-based vision through haze. Applied Optics 42, 3, 511–525.Google ScholarCross Ref
33. Shade, J., Gortler, S., He, L.-W., and Szeliski, R. 1998. Layered depth images. Proceedings of SIGGRAPH ’98, 231–242. Google Scholar
34. Shum, H.-Y., Han, M., and Szeliski, R. 1998. Interactive construction of 3-d models from panoramic mosaics. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998, 427–433. Google ScholarDigital Library
35. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006) 25, 3, 835–846. Google Scholar
36. Stamos, I., and Allen, P. K. 2000. 3-D model construction using range and image data. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1998, 531–536.Google Scholar
37. Sunkavalli, K., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2007. Factored time-lapse video. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3, 101. Google Scholar
38. Tan, R. T. 2008. Visibility in bad weather from a single image. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2008, to appear.Google ScholarCross Ref
39. Toyama, K., Logan, R., and Roseway, A. 2003. Geographic location tags on digital images. Proceedings of the 11th ACM international conference on Multimedia, 156–166. Google Scholar
40. Wang, J., Agrawala, M., and Cohen, M. F. 2007. Soft scissors: an interactive tool for realtime high quality matting. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 3. Google Scholar
41. Yu, Y., and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. Proceedings of SIGGRAPH ’98, 207–217. Google Scholar
42. Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: recovering reflectance models of real scenes from photographs. Proceedings of SIGGRAPH ’99, 215–224. Google Scholar
43. Zhang, L., Dugas-Phocion, G., Samson, J.-S., and Seitz, S. M. 2002. Single-view modelling of free-form scenes. The Journal of Visualization and Computer Animation 13, 4, 225–235.Google ScholarCross Ref
