“Transient attributes for high-level understanding and editing of outdoor scenes” by Laffont, Ren, Tao, Qian and Hays
Conference:
Type:
Title:
- Transient attributes for high-level understanding and editing of outdoor scenes
Session/Category Title: Changing Your Perception
Presenter(s)/Author(s):
Moderator(s):
Abstract:
We live in a dynamic visual world where the appearance of scenes changes dramatically from hour to hour or season to season. In this work we study “transient scene attributes” — high level properties which affect scene appearance, such as “snow”, “autumn”, “dusk”, “fog”. We define 40 transient attributes and use crowdsourcing to annotate thousands of images from 101 webcams. We use this “transient attribute database” to train regressors that can predict the presence of attributes in novel images. We demonstrate a photo organization method based on predicted attributes. Finally we propose a high-level image editing method which allows a user to adjust the attributes of a scene, e.g. change a scene to be “snowy” or “sunset”. To support attribute manipulation we introduce a novel appearance transfer technique which is simple and fast yet competitive with the state-of-the-art. We show that we can convincingly modify many transient attributes in outdoor scenes.
References:
1. An, X., and Pellacini, F. 2010. User-controllable color transfer. Comput. Graph. Forum 29, 2. Google ScholarDigital Library
2. Bell, S., Upchurch, P., Snavely, N., and Bala, K. 2013. Opensurfaces: A richly annotated catalog of surface appearance. ACM Trans. Graph. (proc. SIGGRAPH) 32, 4. Google ScholarDigital Library
3. Berthouzoz, F., Li, W., Dontcheva, M., and Agrawala, M. 2011. A framework for content-adaptive photo manipulation macros. ACM Trans. Graph. 30, 5. Google ScholarDigital Library
4. Bonneel, N., Sunkavalli, K., Paris, S., and Pfister, H. 2013. Example-based video color grading. ACM Trans. Graph. (proc. SIGGRAPH) 32. Google ScholarDigital Library
5. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input / output image pairs. In CVPR. Google ScholarDigital Library
6. Caicedo, J. C., Kapoor, A., and Kang, S. B. 2011. Collaborative personalization of image enhancement. In CVPR. Google ScholarDigital Library
7. Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. (proc. SIGGRAPH) 26, 3. Google ScholarDigital Library
8. Cheng, M.-M., Zheng, S., Lin, W.-Y., Vineet, V., Sturgess, P., Crook, N., Mitra, N., and Torr, P. 2014. Imagespirit: Verbal guided image parsing. ACM Trans. Graph..Google ScholarDigital Library
9. Comaniciu, D., and Meer, P. 2002. Mean shift: a robust approach toward feature space analysis. IEEE Trans. PAMI 24. Google ScholarDigital Library
10. Cusano, C., Gasparini, F., and Schettini, R. 2012. Color transfer using semantic image annotation. In SPIE, vol. 8299. Google ScholarDigital Library
11. Dale, K., Johnson, M. K., Sunkavalli, K., Matusik, W., and Pfister, H. 2009. Image restoration using online photo collections. In ICCV.Google Scholar
12. Dhar, S., Ordonez, V., and Berg, T. L. 2011. High level describable attributes for predicting aesthetics and interestingness. In CVPR. Google ScholarDigital Library
13. Eitz, M., Hays, J., and Alexa, M. 2012. How do humans sketch objects? ACM Trans. Graph. (proc. SIGGRAPH) 31, 4. Google ScholarDigital Library
14. Farhadi, A., Endres, I., Hoiem, D., and Forsyth, D. 2009. Describing objects by their attributes. In CVPR. Google ScholarDigital Library
15. Fattal, R. 2008. Single image dehazing. ACM Trans. Graph. (proc. SIGGRAPH) 27, 3. Google ScholarDigital Library
16. Ferrari, V., and Zisserman, A. 2007. Learning visual attributes. In NIPS.Google Scholar
17. Garg, R., Du, H., Seitz, S. M., and Snavely, N. 2009. The dimensionality of scene appearance. In ICCV.Google Scholar
18. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. In SIGGRAPH. Google ScholarDigital Library
19. Hoiem, D., Efros, A. A., and Hebert, M. 2007. Recovering surface layout from an image. Int. J. Comput. Vision 75, 1. Google ScholarDigital Library
20. Jacobs, N., Roman, N., and Pless, R. 2007. Consistent temporal variations in many outdoor scenes. In CVPR.Google Scholar
21. Johnson, M. K., Dale, K., Avidan, S., Pfister, H., Freeman, W. T., and Matusik, W. 2011. Cg2real: Improving the realism of computer generated images using a large collection of photographs. IEEE Trans. Vis. Comput. Graph. 17, 9. Google ScholarDigital Library
22. Kang, S. B., Kapoor, A., and Lischinski, D. 2010. Personalization of image enhancement. In CVPR.Google Scholar
23. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Trans. Graph. (proc. SIGGRAPH) 26, 3. Google ScholarDigital Library
24. Kovashka, A., Parikh, D., and Grauman, K. 2012. Whittle-search: Image search with relative attribute feedback. In CVPR. Google ScholarDigital Library
25. Kumar, N., Berg, A., Belhumeur, P., and Nayar, S. 2011. Describable visual attributes for face verification and image search. IEEE Trans. PAMI 33, 10. Google ScholarDigital Library
26. Laffont, P.-Y., Bousseau, A., Paris, S., Durand, F., and Drettakis, G. 2012. Coherent intrinsic images from photo collections. ACM Trans. Graph. (proc. SIGGRAPH Asia) 31, 6. Google ScholarDigital Library
27. Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Trans. Graph. (proc. SIGGRAPH) 26, 3. Google ScholarDigital Library
28. Lalonde, J.-F., Efros, A., and Narasimhan, S. 2009. Web-cam clip art: Appearance and illuminant transfer from time-lapse sequences. ACM Trans. Graph. (proc. SIGGRAPH Asia) 28, 5. Google ScholarDigital Library
29. Liu, Q., Ihler, A., and Steyvers, M. 2013. Scoring workers in crowdsourcing: how many control questions are enough? In NIPS.Google Scholar
30. Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Trans. Graph. (proc. SIGGRAPH) 22, 3. Google ScholarDigital Library
31. Murphy, K. P. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press. Google ScholarDigital Library
32. Narasimhan, S., Wang, C., and Nayar, S. 2002. All the images of an outdoor scene. In ECCV. Google ScholarDigital Library
33. Parikh, D., and Grauman, K. 2011. Relative attributes. In ICCV. Google ScholarDigital Library
34. Patterson, G., and Hays, J. 2012. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR. Google ScholarDigital Library
35. Perronnin, F., Sánchez, J., and Mensink, T. 2010. Improving the fisher kernel for large-scale image classification. In ECCV. Google ScholarDigital Library
36. Pitié, F., Kokaram, A., and Dahyot, R. 2005. N-Dimensional Probability Density Function Transfer and its Application to Colour Transfer. In ICCV. Google ScholarDigital Library
37. Pouli, T., and Reinhard, E. 2011. Progressive color transfer for images of arbitrary dynamic range. Computers & Graphics 35. Google ScholarDigital Library
38. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5. Google ScholarDigital Library
39. Scholkopf, B., Smola, A., Williamson, R., and Bartlett, P. 2000. New support vector algorithms. Neural Computation 12. Google ScholarDigital Library
40. Shih, Y., Paris, S., Durand, F., and Freeman, W. T. 2013. Data-driven hallucination of different times of day from a single outdoor photo. ACM Trans. Graph. (proc. SIGGRAPH Asia) 32, 6. Google ScholarDigital Library
41. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. (proc. SIGGRAPH) 25, 3. Google ScholarDigital Library
42. Sunkavalli, K., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2007. Factored time-lapse video. ACM Trans. Graph. (proc. SIGGRAPH) 26, 3. Google ScholarDigital Library
43. Tao, L., Yuan, L., and Sun, J. 2009. Skyfinder: Attribute-based sky image search. ACM Trans. Graph. (proc. SIGGRAPH) 28, 3. Google ScholarDigital Library
44. Wu, F., Dong, W., Kong, Y., Mei, X., Paul, J.-C., and Zhang, X. 2013. Content-Based Colour Transfer. Comput. Graph. Forum 32, 1.Google ScholarDigital Library
45. Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR.Google Scholar
46. Yu, Y., and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. In SIGGRAPH. Google ScholarDigital Library
