“Efficiently combining positions and normals for precise 3D geometry” by Nehab, Rusinkiewicz, Davis and Ramamoorthi

Range scanning, manual 3D editing, and other modeling approaches can provide information about the geometry of surfaces in the form of either 3D positions (e.g., triangle meshes or range images) or orientations (normal maps or bump maps). We present an algorithm that combines these two kinds of estimates to produce a new surface that approximates both. Our formulation is linear, allowing it to operate efficiently on complex meshes commonly used in graphics. It also treats high-and low-frequency components separately, allowing it to optimally combine outputs from data sources such as stereo triangulation and photometric stereo, which have different error-vs.-frequency characteristics. We demonstrate the ability of our technique to both recover high-frequency details and avoid low-frequency bias, producing surfaces that are more widely applicable than position or orientation data alone.

1. Banerjee, S., Sastry, P., and Venkatesh, Y. 1992. Surface reconstruction from disparate shading: An integration of shape-from-shading and stereopsis. In IAPR, volume 1, pages 141–144.Google Scholar
2. Bernardini, F., Martin, I., Mittleman, J., Rushmeier, H., and Taubin, G. 2002. Building a digital model of michelangelo’s florentine pietà. IEEE Computer Graphics and Applications, 22(1):59–67. Google ScholarDigital Library
3. Besl, P. J. 1989. Active Optical Range Imaging Sensors, pages 1–63. Google ScholarDigital Library
4. Besl, P. J. and Mckay, N. D. 1992. A method for registration of 3-d shapes. PAMI, 14(2):239–256. Google ScholarDigital Library
5. Bouguet, J.-Y. 2004. Camera Calibration Toolbox for Matlab. URL http://www.vision.caltech.edu/bouguetj/calib_doc.Google Scholar
6. Chen, C.-Y., Klette, R., and Chen, C.-F. 2003. Shape from photometric-stereo and contours. In N. Petkov and M. Westenberg, editors, CAIP, Springer-Verlag, pages 377–384.Google Scholar
7. Cryer, J. E., Tsai, P.-S., and Shah, M. 1995. Integration of shape from shading and stereo. Pattern Recognition, 28(7): 1033–1043.Google ScholarCross Ref
8. Curless, B. 1997. New Methods For Surface Reconstruction From Range Images. PhD thesis, Stanford University. Google ScholarDigital Library
9. Curless, B. and Levoy, M. 1996. A volumetric method for building complex models from range images. In SIGGRAPH, ACM Press, pages 303–312. Google ScholarDigital Library
10. Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. PAMI, 27(2):296–302. Google ScholarDigital Library
11. Epstein, R., Yuille, A. L., and Belhumeur, P. N. 1996. Learning object representations from lighting variations. In ECCV, pages 179–199. Google ScholarDigital Library
12. Fua, P. and Leclerc, Y. G. 1994. Using 3-dimensional meshes to combine image-based and geometry-based constraints. In J.-O. Eklundh, editor, ECCV, volume 2, Springer-Verlag, pages 281–291. Google ScholarDigital Library
13. Georghiades. A. 2003. Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo. In ICCV, volume 2, pages 816–823. Google ScholarDigital Library
14. Horn, B. K. P. 1970. Shape from Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View. PhD thesis, Massachusetts Institute of Technology.Google Scholar
15. Ikeuchi, K. 1983. Constructing a depth map from images. A. I Memo AIM-744, Artificial Intelligence Laboratory, MIT.Google Scholar
16. Ikeuchi, K. 1987. Determining a depth map using a dual photometric stereo. IJRR, 6(1):15–31. Google ScholarDigital Library
17. Lange, H. 1999. Advances in the cooperation of shape from shading and stereo vision. In 3DIM, pages 46–58. Google ScholarDigital Library
18. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. 2003. Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph., 22(2): 234–257. Google ScholarDigital Library
19. Miller, G. 1994. Efficient algorithms for local and global accessibility shading. In SIGGRAPH, pages 319–326. Google ScholarDigital Library
20. Mostafa, M., Yamany, S., and Farag, A. 1999. Integrating shape from shading and range data using neural networks. In CVPR, volume 2, IEEE Computer Society, pages 15–20.Google Scholar
21. Paige, C. C. and Saunders, M. A. 1982. LSQR: An algorithm for sparse linear equations and sparse least squares. TOMS, 8(1): 43–71. Google ScholarDigital Library
22. Poussart, D. and Laurendeau, D. 1989. 3-D Sensing for Industrial Computer Vision, pages 122–159. Google ScholarDigital Library
23. Pulli, K. 1999. Multiview registration for large datasets. In 3DIM, pages 160–168. Google ScholarDigital Library
24. Rushmeier, H. and Bernardini, F. 1999. Computing consistent normals and colors from photometric data. In 3DIM, pages 99–108. Google ScholarDigital Library
25. Tagare, H. D. and Defigueiredo, R. J. P. 1991. A theory of photometric stereo for a class of diffuse non-lambertian surfaces. PAMI, 13(2):133–152. Google ScholarDigital Library
26. Terzopoulos, D. 1988. The computation of visible-surface representations. PAMI, 10(4):417–438. Google ScholarDigital Library
27. Woodham, R. J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19 (1):139–144.Google ScholarCross Ref
28. Zhang, L., Curless, B., and Seitz, S. M. 2003. Spacetime stereo: Shape recovery for dynamic scenes. In CVPR, pages 367–374.Google Scholar