“Webcam clip art: appearance and illuminant transfer from time-lapse sequences”
Conference:
Type(s):
Title:
- Webcam clip art: appearance and illuminant transfer from time-lapse sequences
Session/Category Title: Lighting & materials
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Webcams placed all over the world observe and record the visual appearance of a variety of outdoor scenes over long periods of time. The recorded time-lapse image sequences cover a wide range of illumination and weather conditions — a vast untapped resource for creating visual realism. In this work, we propose to use a large repository of webcams as a “clip art” library from which users may transfer scene appearance (objects, scene backdrops, outdoor illumination) into their own time-lapse sequences or even single photographs. The goal is to combine the recent ideas from data-driven appearance transfer techniques with a general and theoretically-grounded physically-based illumination model. To accomplish this, the paper presents three main research contributions: 1) a new, high-quality outdoor webcam database that has been calibrated radiometrically and geometrically; 2) a novel approach for matching illuminations across different scenes based on the estimation of the properties of natural illuminants (sun, sky, weather and clouds), the camera geometry, and illumination-dependent scene features; 3) a new algorithm for generating physically plausible high dynamic range environment maps for each frame in a webcam sequence.
References:
1. Arnold, D., Chalmers, A., Niccolucci, F., Stumpfel, J., Tchou, C., Yun, N., Hawkins, T., Jones, A., Emerson, B., and Debevec, P. 2003. Digital reunification of the parthenon and its sculptures. In 4th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage. Google ScholarCross Ref
2. Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., and Nayar, S. K. 2008. Face swapping: automatically replacing faces in photographs. ACM Transactions on Graphics (SIGGRAPH) 27, 3, 39. Google ScholarDigital Library
3. Debevec, P., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In ACM SIGGRAPH. Google ScholarDigital Library
4. Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., and Hawkins, T. 2002. A lighting reproduction approach to live-action compositing. ACM Transactions on Graphics (SIGGRAPH). Google ScholarDigital Library
5. Debevec, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In ACM SIGGRAPH. Google ScholarDigital Library
6. Dror, R. O., Willsky, A. S., and Adelson, E. H. 2004. Statistical characterization of real-world illumination. Journal of Vision 4.Google ScholarCross Ref
7. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In ACM SIGGRAPH. Google ScholarDigital Library
8. Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S. M. 2007. Multi-view stereo for community photo collections. In International Conference on Computer Vision.Google Scholar
9. Haber, T., Fuchs, C., Bekaert, P., Seidel, H.-P., Goesele, M., and Lensch, H. P. A. 2009. Relighting objects from image collections. In Conference on Computer Vision and Pattern Recognition.Google Scholar
10. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Transactions on Graphics (SIGGRAPH) 26, 3, 4. Google ScholarDigital Library
11. Hoiem, D., Efros, A. A., and Hebert, M. 2005. Geometric context from a single image. In International Conference on Computer Vision. Google ScholarDigital Library
12. Horry, Y., Anjyo, K.-I., and Arai, K. 1997. Tour into the picture: using a spidery mesh interface to make animation from a single image. In ACM SIGGRAPH. Google ScholarDigital Library
13. Jacobs, N., Roman, N., and Pless, R. 2007. Consistent temporal variations in many outdoor scenes. In Conference on Computer Vision and Pattern Recognition.Google Scholar
14. Jacobs, N., Satkin, S., Roman, N., Speyer, R., and Pless, R. 2007. Geolocating static cameras. In International Conference on Computer Vision.Google Scholar
15. Jacobs, N., Roman, N., and Pless, R. 2008. Toward fully automatic geo-location and geo-orientation of static outdoor cameras. In Workshop on applications of computer vision. Google ScholarDigital Library
16. Kasten, F., and Young, A. T. 1989. Revised optical air mass tables and approximation formula. Applied optics 28.Google Scholar
17. Khan, E. A., Reinhard, E., Fleming, R., and Büelthoff, H. 2006. Image-based material editing. ACM Transactions on Graphics (SIGGRAPH). Google ScholarDigital Library
18. Kim, S. J., Frahm, J.-M., and Polleyfeys, M. 2008. Radiometric calibration with illumination change for outdoor scene analysis. In Conference on Computer Vision and Pattern Recognition.Google Scholar
19. Koppal, S. J., and Narasimhan, S. G. 2006. Clustering appearance for scene analysis. In International Conference on Computer Vision. Google ScholarDigital Library
20. Kuthirummal, S., Agarwala, A., Glodman, D. B., and Nayar, S. K. 2008. Priors for large photo collections and what they reveal about cameras. In European Conference on Computer Vision. Google ScholarDigital Library
21. Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Transactions on Graphics (SIGGRAPH) 26, 3, 3. Google ScholarDigital Library
22. Lalonde, J.-F., Narasimhan, S. G., and Efros, A. A. 2009. What do the sun and the sky tell us about the camera? International Journal on Computer Vision (to appear). Google ScholarDigital Library
23. Lin, S., Gu, J., Yamazaki, S., and Shum, H.-Y. 2004. Radiometric calibration from a single image. In Conference on Computer Vision and Pattern Recognition. Google ScholarDigital Library
24. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2. Google ScholarDigital Library
25. Opperman, I., Ferndriger, S., and Eugster, J., 2009. Touristic webcams worldwide. Accessed: 09/03/09, http://www.webcams.travel.Google Scholar
26. Perez, R., Seals, R., and Michalsky, J. 1993. All-weather model for sky luminance distribution — preliminary configuration and validation. Solar Energy 50, 3 (March), 235–245.Google ScholarCross Ref
27. Preetham, A. J., Shirley, P., and Smits, B. 1999. A practical analytic model for daylight. In ACM SIGGRAPH. Google ScholarDigital Library
28. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications, special issue on Applied Perception 21, 5 (September–October), 34–41. Google ScholarDigital Library
29. Schöld, A., Szeliski, R., Salesin, D., and Essa, I. 2000. Video textures. In ACM SIGGRAPH.Google Scholar
30. Snavely, N., Garg, R., Seitz, S. M., and Szeliski, R. 2008. Finding paths through the world’s photos. ACM Transactions on Graphics (SIGGRAPH) 27, 3, 15. Google ScholarDigital Library
31. Stumpfel, J., Jones, A., Wenger, A., Tchou, C., Hawkins, T., and Debevec, P. 2004. Direct HDR capture of the sun and sky. In AFRIGRAPH. Google ScholarDigital Library
32. Sunkavalli, K., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2007. Factored time-lapse video. ACM Transactions on Graphics (SIGGRAPH) 26, 3, 101. Google ScholarDigital Library
33. Sunkavalli, K., Romeiro, F., Matusik, W., Zickler, T., and Pfister, H. 2008. What do color changes reveal about an outdoor scene? In Conference on Computer Vision and Pattern Recognition.Google Scholar
34. Tao, L., Yuan, L., and Sun, J. 2009. SkyFinder: attribute-based sky image search. ACM Transactions on Graphics (SIGGRAPH) 28, 3, 68. Google ScholarDigital Library
35. Weiss, Y. 2001. Deriving intrinsic images from image sequences. In International Conference on Computer Vision.Google ScholarCross Ref
