“OpenSurfaces: a richly annotated catalog of surface appearance” by Bell, Upchurch, Snavely and Bala
Conference:
Type:
Title:
- OpenSurfaces: a richly annotated catalog of surface appearance
Session/Category Title: Materials
Presenter(s)/Author(s):
Moderator(s):
Abstract:
The appearance of surfaces in real-world scenes is determined by the materials, textures, and context in which the surfaces appear. However, the datasets we have for visualizing and modeling rich surface appearance in context, in applications such as home remodeling, are quite limited. To help address this need, we present OpenSurfaces, a rich, labeled database consisting of thousands of examples of surfaces segmented from consumer photographs of interiors, and annotated with material parameters (reflectance, material names), texture information (surface normals, rectified textures), and contextual information (scene category, and object names).Retrieving usable surface information from uncalibrated Internet photo collections is challenging. We use human annotations and present a new methodology for segmenting and annotating materials in Internet photo collections suitable for crowdsourcing (e.g., through Amazon’s Mechanical Turk). Because of the noise and variability inherent in Internet photos and novice annotators, designing this annotation engine was a key challenge; we present a multi-stage set of annotation tasks with quality checks and validation. We demonstrate the use of this database in proof-of-concept applications including surface retexturing and material and image browsing, and discuss future uses. OpenSurfaces is a public resource available at http://opensurfaces.cs.cornell.edu/.
References:
1. Adelson, E. H. 2001. On seeing stuff: the perception of materials by humans and machines. Proc. SPIE Human Vision and Electronic Imaging 4299.Google Scholar
2. Ben-Artzi, A., Overbeck, R., and Ramamoorthi, R. 2006. Real-time BRDF editing in complex lighting. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
3. Brainard, D. H., Brunt, W., and Speigle, J. 1997. Color constancy in the nearly natural image. J. of the Optical Society of America 14, 9.Google Scholar
4. Cgal, Computational Geometry Algorithms Library. http://www.cgal.org/.Google Scholar
5. Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
6. Cole, F., Sanik, K., DeCarlo, D., Finkelstein, A., Funkhouser, T., Rusinkiewicz, S., and Singh, M. 2009. How well do line drawings depict shape? In SIGGRAPH Conf. Proc. Google ScholarDigital Library
7. Dana, K., Van-Ginneken, B., Nayar, S., and Koenderink, J. 1999. Reflectance and texture of real world surfaces. ACM Transactions on Graphics 18, 1. Google ScholarDigital Library
8. Debevec, P. 1998. Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
9. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and FeiFei, L. 2009. ImageNet: A large-scale hierarchical image database. In Proc. Comp. Vision and Pattern Recognition.Google ScholarCross Ref
10. Dror, R., Adelson, E. H., and Willsky, A. 2001. Estimating surface reflectance properties from images under unknown illumination. In Proc. SPIE Human Vision and Electronic Imaging.Google Scholar
11. Endres, I., Farhadi, A., Hoiem, D., and Forsyth, D. 2010. The benefits and challenges of collecting richer object annotations. In Workshop on Advancing Computer Vision with Humans in the Loop.Google Scholar
12. Feng, C., Deng, F., and Kamat, V. R. 2010. Semi-automatic 3D reconstruction of piecewise planar building models from single image. In Int. Conf. on Construction Appl. of Virtual Reality.Google Scholar
13. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. J. of Vision 3, 5.Google ScholarCross Ref
14. Fleming, R. W., Torralba, A., and Adelson, E. H. 2004. Specular reflections and the perception of shape. J. of Vision 4, 9.Google ScholarCross Ref
15. Geisler-Moroder, D., and Dür, A. 2010. A new Ward BRDF model with bounded albedo. In Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
16. Gingold, Y., Shamir, A., and Cohen-Or, D. 2012. Micro perceptual human computation. ACM Transactions on Graphics 31, 5. Google ScholarDigital Library
17. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. In SIGGRAPH Conf. Proc., 4:1–4:7. Google ScholarDigital Library
18. Hu, D., Bo, L., and Ren, X. 2011. Toward robust material recognition for everyday objects. In Proc. British Machine Vision Conf.Google Scholar
19. Juracek, J. 1996. Surfaces: Visual Research for Artists and Designers. Norton.Google Scholar
20. Karsch, K., Hedau, V., Forsyth, D., and Hoiem, D. 2011. Rendering synthetic objects into legacy photographs. In SIGGRAPH Asia Conf. Proc. Google ScholarDigital Library
21. Kerr, W. B., and Pellacini, F. 2010. Toward evaluating material design interface paradigms for novice users. ACM Transactions on Graphics 29, 4. Google ScholarDigital Library
22. Koenderink, J. J., Doorn, A. J. V., and Kappers, A. M. L. 1992. Surface perception in pictures. Perception & Psychophysics.Google Scholar
23. Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
24. Liu, Y., Lin, W.-C., and Hays, J. 2004. Near regular texture analysis and manipulation. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
25. Liu, C., Sharan, L., Adelson, E., and Rosenholtz, R. 2010. Exploring features in a Bayesian framework for material recognition. In Proc. Comp. Vision and Pattern Recognition.Google Scholar
26. Marge, M., Banerjee, S., and Rudnicky, A. I. 2010. Using the Amazon Mechanical Turk for transcription of spoken language. In Int. Conf. on Acoustics, Speech, and Signal Processing.Google Scholar
27. Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Transactions on Graphics 22, 3. Google ScholarDigital Library
28. Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of BRDF models. In Proc. Eurographics Symp. on Rendering. Google ScholarDigital Library
29. Pellacini, F., Ferwerda, J. A., and Greenberg, D. P. 2000. Toward a psychophysically-based light reflection model for image synthesis. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
30. Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: Towards a new standard for image fidelity. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
31. Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
32. Ren, P., Wang, J., Snyder, J., Tong, X., and Guo, B. 2011. Pocket reflectometry. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
33. Romeiro, F., and Zickler, T. 2010. Blind reflectometry. In Proc. European Conf. on Comp. Vision. Google ScholarDigital Library
34. Rubinstein, M., Gutierrez, D., Sorkine, O., and Shamir, A. 2010. A comparative study of image retargeting. In SIGGRAPH Asia Conf. Proc. Google ScholarDigital Library
35. Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2008. LabelMe: A database and web-based tool for image annotation. Int. J. of Computer Vision 77, 1–3. Google ScholarDigital Library
36. Sharan, L., Rosenholtz, R., and Adelson, E. H. 2009. Material perception: What can you see in a brief glance? J. of Vision 9, 8.Google Scholar
37. Tardif, J.-P. 2009. Non-iterative approach for fast and accurate vanishing point detection. In Proc. Int. Conf. on Comp. Vision.Google ScholarCross Ref
38. Toldo, R., and Fusiello, A. 2008. Robust multiple structures estimation with J-linkage. In Proc. European Conf. on Comp. Vision. Google ScholarDigital Library
39. Torralba, A., Fergus, R., and Freeman, W. T. 2008. 80 million tiny images: A large data set for nonparametric object and scene recognition. Trans. on Pattern Analysis and Machine Intelligence 30, 11. Google ScholarDigital Library
40. Vangorp, P., Laurijssen, J., and Dutré, P. 2007. The influence of shape on the perception of material reflectance. ACM Transactions on Graphics 26, 3. Google ScholarDigital Library
41. von Gioi, R. G., Jakubowicz, J., Morel, J.-M., and Randall, G. 2010. LSD: A fast line segment detector with a false detection control. Trans. on Pattern Analysis and Machine Intelligence 32, 4. Google ScholarDigital Library
42. Walter, B., Khungurn, P., and Bala, K. 2012. Bidirectional lightcuts. In SIGGRAPH Conf. Proc.Google Scholar
43. Ward, G. 1992. Measuring and modeling anisotropic reflection. In SIGGRAPH Conf. Proc. Google ScholarDigital Library
44. Welinder, P., Branson, S., Belongie, S., and Perona, P. 2010. The multidimensional wisdom of crowds. In Proc. Neural Information Processing Systems.Google Scholar
45. Weyrich, T., Lawrence, J., Lensch, H. P. A., Rusinkiewicz, S., and Zickler, T. 2009. Principles of appearance acquisition and representation. Foundations and Trends in Computer Graphics and Vision 4, 2. Google ScholarDigital Library
46. Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. 2010. SUN database: Large-scale scene recognition from abbey to zoo. In Proc. Comp. Vision and Pattern Recognition.Google Scholar
