“A perceptually validated model for surface depth hallucination” by Glencross, Melendez, Jay, Liu, Hubbold, et al. …

Capturing detailed surface geometry currently requires specialized equipment such as laser range scanners, which despite their high accuracy, leave gaps in the surfaces that must be reconciled with photographic capture for relighting applications. Using only a standard digital camera and a single view, we present a method for recovering models of predominantly diffuse textured surfaces that can be plausibly relit and viewed from any angle under any illumination. Our multiscale shape-from-shading technique uses diffuse-lit/flash-lit image pairs to produce an albedo map and textured height field. Using two lighting conditions enables us to subtract one from the other to estimate albedo. In the absence of a flash-lit image of a surface for which we already have a similar exemplar pair, we approximate both albedo and diffuse shading images using histogram matching. Our depth estimation is based on local visibility. Unlike other depth-from-shading approaches, all operations are performed on the diffuse shading image in image space, and we impose no constant albedo restrictions. An experimental validation shows our method works for a broad range of textured surfaces, and viewers are frequently unable to identify our results as synthetic in a randomized presentation. Furthermore, in side-by-side comparisons, subjects found a rendering of our depth map equally plausible to one generated from a laser range scan. We see this method as a significant advance in acquiring surface detail for texturing using a standard digital camera, with applications in architecture, archaeological reconstruction, games and special effects.

1. Belhumeur, P. N., Kriegman, D. J., and Yuille, A. L. 1999. The bas-relief ambiguity. International Journal of Computer Vision 35, 1, 33–44. Google ScholarDigital Library
2. Burt, P., and Adelson, E. 1983. The Laplacian pyramid as a compact image code. IEEE Transactions on Communications 31, 4, 532–540.Google ScholarCross Ref
3. Dana, K. J., van Ginneken, B., Nayar, S. K., and Koenderink, J. J. 1999. Reflectance and texture of real-world surfaces. ACM Transactions on Graphics (TOG) 18, 1, 1–34. Google ScholarDigital Library
4. Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: a hybrid geometry and image-based approach. In SIGGRAPH, ACM, 11–20. Google ScholarDigital Library
5. Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. In SIGGRAPH, ACM, 341–346. Google ScholarDigital Library
6. Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. In SIGGRAPH, ACM, 673–678. Google ScholarDigital Library
7. Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. Journal of Vision 3, 5, 347–368.Google ScholarCross Ref
8. Haddon, J. A., and Forsyth, D. A. 1998. Shading primitives: Finding folds and shallow grooves. In ICCV, 236–241. Google ScholarDigital Library
9. Haddon, J. A., and Forsyth, D. A. 1998. Shape representations from shading primitives. In ECCV ’98: Proceedings of the 5th European Conference on Computer Vision-Volume II, Springer-Verlag, London, UK, 415–431. Google ScholarDigital Library
10. Heeger, D. J., and Bergen, J. R. 1995. Pyramid-based texture analysis/synthesis. In SIGGRAPH, ACM, 229–238. Google ScholarDigital Library
11. Hershberger, W., 2008. Taming those annoying highlights: cross-polarization flash macro photography. Online article. http://www.naturescapes.net/042004/wh0404.htm.Google Scholar
12. Horn, B. K. P. 1989. Obtaining shape from shading information. In Series of Artificial Intelligence: Shape from Shading. Mit Press, Cambridge, MA, 123–171. Google ScholarDigital Library
13. Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. In SIGGRAPH, ACM, 654–663. Google ScholarDigital Library
14. Koenderink, J., and van Doorn, A. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. J. Opt. Soc. Am. 73, 6 (June), 843–850.Google ScholarCross Ref
15. Langer, M. S., and Bülthoff, H. H. 2000. Depth discrimination from shading under diffuse lighting. Perception 29, 6, 649–660.Google ScholarCross Ref
16. Langer, M. S., and Zucker, S. W. 1994. Shape-from-shading on a cloudy day. Journal of the Optical Society of America 11, 2, 467–478.Google ScholarCross Ref
17. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Transactions on Graphics (TOG) 22, 2, 234–257. Google ScholarDigital Library
18. Li, H., Foo, S.-C., Torrance, K. E., and Westin, S. H. 2006. Automated three-axis gonioreflectometer for computer graphics applications. Optical Engineering 45, 4, 1–11.Google ScholarCross Ref
19. Malik, J., and Maydan, D. 1989. Recovering threedimensional shape from a single image of curved objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 11, 6. Google ScholarDigital Library
20. Narasimhan, S. G., Visvanathan, R., and Nayar, S. K. 2003. A class of photometric invariants: separating material from shape and illumination. In ICCV, IEEE, 1387–1394. Google ScholarDigital Library
21. Ngan, A., and Durand, F. 2006. Statistical acquisition of texture appearance. In Eurographics Symposium on Rendering, T. Akenine-Möller and W. Heidrich, Eds. The Eurographics Association, 31–40. Google ScholarCross Ref
22. Ostrovsky, Y., Cavanagh, P., and Sinha, P. 2005. Perceiving illumination inconsistencies in scenes. Perception 34, 11, 1301–1314.Google ScholarCross Ref
23. Paterson, J. A., Claus, D., and Fitzgibbon, A. W. 2005. BRDF and geometry capture from extended inhomogeneous samples using flash photography. In Computer Graphics Forum, vol. 24, Eurographics, 383–391.Google Scholar
24. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. In SIGGRAPH, ACM, 664–672. Google ScholarDigital Library
25. Prados, E., and Faugeras, O. 2005. Shape from shading: a well-posed problem? In Computer Vision and Pattern Recognition (CVPR), IEEE, 870–877. Google ScholarDigital Library
26. Ramachandran, V. S. 1988. Perception of shape from shading. Nature, 331, 163–166.Google ScholarCross Ref
27. Rushmeier, H., and Bernardini, F. 1999. Computing consistent normals and colors from photometric data. In Second Conference on 3-D Imaging and Modeling 3DIM, IEEE, 99–108. Google ScholarDigital Library
28. Ward, G. J. 1994. The RADIANCE lighting simulation and rendering system. In SIGGRAPH, ACM, 459–72. Google ScholarDigital Library
29. Yu, Y., Debevec, P., Malik, J., and Hawkins, T. 1999. Inverse global illumination: recovering reflectance models of real scenes from photographs. In SIGGRAPH, ACM, 215–224. Google ScholarDigital Library
30. Zhang, R., Tsai, P.-S., Cryer, J. E., and Shah, M. 1999. Shape from shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 8, 690–706. Google ScholarDigital Library