“Image-based 3D photography using opacity hulls”

We have built a system for acquiring and displaying high quality graphical models of objects that are impossible to scan with traditional scanners. Our system can acquire highly specular and fuzzy materials, such as fur and feathers. The hardware set-up consists of a turntable, two plasma displays, an array of cameras, and a rotating array of directional lights. We use multi-background matting techniques to acquire alpha mattes of the object from multiple viewpoints. The alpha mattes are used to construct an opacity hull. The opacity hull is a new shape representation, defined as the visual hull of the object with view-dependent opacity. It enables visualization of complex object silhouettes and seamless blending of objects into new environments. Our system also supports relighting of objects with arbitrary appearance using surface reflectance fields, a purely image-based appearance representation. Our system is the first to acquire and render surface reflectance fields under varying illumination from arbitrary viewpoints. We have built three generations of digitizers with increasing sophistication. In this paper, we present our results from digitizing hundreds of models.

1. BEARDSLEY, P. 2002. Calibration of stereo cameras for a turntable 3d scanner. Tech. rep., MERL. TR 2002/20.Google Scholar
2. BERNARDINI, F., MARTIN, I., AND RUSHMEIER, H. 2001. High-quality texture reconstruction from multiple scans. IEEE Trans. on Vis. and Comp. Graph. 7, 4 (Oct.-Dec.), 318-332. Google Scholar
3. BUEHLER, C., BOSSE, M., MCMILLAN, L., GORTLER, S., AND COHEN, M. 2001. Unstructured lumigraph rendering. In Computer Graphics, SIGGRAPH 2001 Proceedings, 425-432. Google Scholar
4. CHEN, S. E., AND WILLIAMS, L. 1993. View interpolation for image synthesis. In Computer Graphics, SIGGRAPH 93 Proceedings, 279-288. Google Scholar
5. CHEN, W.-C., GRZESZCZUK, R., AND BOUGUET, J.-Y. 2002. Light field mapping: Efficient representation and hardware rendering of surface light fields. In Computer Graphics, To appear in the SIGGRAPH 2002 Proceedings. Google Scholar
6. CURLESS, B., AND LEVOY, M. 1996. A volumetric method for building complex models from range images. In Computer Graphics, SIGGRAPH 96 Proceedings, 303-312. Google Scholar
7. DEBEVEC, P., AND MALIK, J. 1997. Recovering high dynamic range radiance maps from photographs. In Computer Graphics, SIGGRAPH 97 Proceedings, 369-378. Google Scholar
8. DEBEVEC, P., TAYLOR, C., AND MALIK, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Computer Graphics, SIGGRAPH 96 Proceedings, 11-20. Google Scholar
9. DEBEVEC, P., YU, Y., AND BORSHUKOV, G. 1998. Efficient view-dependent image-based rendering with projective texture-mapping. In Proceedings of the 9th Eurographics Workshop on Rendering, 105-116. Google Scholar
10. DEBEVEC, P., HAWKINS, T., TCHOU, C., DUIKER, H.-P., SAROKIN, W., AND SAGAR, M. 2000. Acquiring the reflectance field of a human face. In Computer Graphics, SIGGRAPH 2000 Proceedings, 145-156. Google Scholar
11. GEORGHIADES, A., BELHUMEUR, P., AND KRIEGMAN, D. 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47-54. Google Scholar
12. GORTLER, S., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. 1996. The lumigraph. In Computer Graphics, SIGGRAPH 96 Proceedings, 43-54. Google Scholar
13. HAWKINS, T., COHEN, J., AND DEBEVEC, P. 2001. A photometric approach to digitizing cultural artifacts. In 2nd International Symposium on Virtual Reality, Archaeology, and Cultural Heritage. Google Scholar
14. KOUDELKA, M., BELHUMEUR, P., MAGDA, S., AND KRIEGMAN, D. 2001. Image-based modeling and rendering of surfaces with arbitrary brdfs. In Proc. of Computer Vision and Pattern Recognition, in press.Google Scholar
15. LAURENTINI, A. 1994. The visual hull concept for silhouette-based image understanding. PAMI 16, 2 (February), 150-162. Google Scholar
16. LENGYEL, J., PRAUN, E., FINKELSTEIN, A., AND HOPPE, H. 2001. Real-time fur over arbitrary surfaces. In Symposium on Interactive 3D Graphics, 227-232. Google Scholar
17. LENSCH, H., KAUTZ, J., GOESELE, M., HEIDRICH, W., AND SEIDEL, H.-P. 2001. Image-based reconstruction of spatially varying materials. In Proceedings of the 12th Eurographics Workshop on Rendering. Google Scholar
18. LEVOY, M., AND HANRAHAN, P. 1996. Light field rendering. In Computer Graphics, SIGGRAPH 96 Proceedings, 31-42. Google Scholar
19. LEVOY, M., AND WHITTED, T. 1985. The use of points as display primitives. Tech. Rep. TR 85-022, The University of North Carolina at Chapel Hill, Department of Computer Science.Google Scholar
20. LEVOY, M., PULLI, K., CURLESS, B., RUSINKIEWICZ, S., KOLLER, D., PEREIRA, L., GINZTON, M., ANDERSON, S., DAVIS, J., GINSBERG, J., SHADE, J., AND FULK, D. 2000. The digital michelangelo project: 3d scanning of large statues. In Computer Graphics, SIGGRAPH 2000 Proceedings, 131-144. Google Scholar
21. MALZBENDER, T., GELB, D., AND WOLTERS, H. 2001. Polynomial texture maps. In Computer Graphics, SIGGRAPH 2001 Proceedings, 519-528. Google Scholar
22. MARSCHNER, S., WESTIN, S., LAFORTUNE, E., TORRANCE, K., AND GREENBERG, D. 1999. Image-based brdf measurement including human skin. In Proceedings of the 10th Eurographics Workshop on Rendering, 139-152. Google Scholar
23. MATUSIK, W., BUEHLER, C., RASKAR, R., GORTLER, S., AND MCMILLAN, L. 2000. Image-based visual hulls. In Computer Graphics, SIGGRAPH 2000 Proceedings, 369-374. Google Scholar
24. MCMILLAN, L., AND BISHOP, G. 1995. Plenoptic modeling: An image-based rendering system. In Computer Graphics, SIGGRAPH 95 Proceedings, 39-46. Google Scholar
25. MILLER, G., RUBIN, S., AND PONCELEON, D. 1998. Lazy decompression of surface light fields for precomputed global illumination. In Proceedings of the 9th Eurographics Workshop on Rendering, 281-292.Google Scholar
26. NISHINO, K., SATO, Y., AND IKEUCHI, K. 1999. Appearance compression and synthesis based on 3d model for mixed reality. In Proceedings of IEEE ICCV ’99, 38-45.Google Scholar
27. NISHINO, K., SATO, Y., AND IKEUCHI, K. 1999. Eigen-texture method: Appearance compression based on 3d model. In Proc. of Computer Vision and Pattern Recognition, 618-624.Google Scholar
28. PFISTER, H., ZWICKER, M., VAN BAAR, J., AND GROSS, M. 2000. Surfels: Surface elements as rendering primitives. In Computer Graphics, SIGGRAPH 2000 Proceedings, 335-342. Google Scholar
29. PULLI, K., COHEN, M., DUCHAMP, T., HOPPE, H., SHAPIRO, L., AND STUETZLE, W. 1997. View-based rendering: Visualizing real objects from scanned range and color data. In Eurographics Rendering Workshop 1997, 23-34. Google Scholar
30. RAMAMOORTHI, R., AND HANRAHAN, P. 2001. A signal-processing framework for inverse rendering. In Computer Graphics, SIGGRAPH 2001 Proceedings, 117-128. Google Scholar
31. RUSHMEIER, H., BERNARDINI, F., MITTLEMAN, J., AND TAUBIN, G. 1998. Acquiring input for rendering at appropriate levels of detail: Digitizing a pietà. In Proceedings of the 9th Eurographics Workshop on Rendering, 81-92.Google Scholar
32. RUSINKIEWICZ, S., AND LEVOY, M. 2000. Qsplat: A multiresolution point rendering system for large meshes. In Computer Graphics, SIGGRAPH 2000 Proceedings, 343-352. Google Scholar
33. SANDER, P., GU, X., GORTLER, S., HOPPE, H., AND SNYDER, J. 2000. Silhouette clipping. In Computer Graphics, SIGGRAPH 2000 Proceedings, 327-334. Google Scholar
34. SATO, Y., WHEELER, M. D., AND IKEUCHI, K. 1997. Object shape and reflectance modeling from observation. In Computer Graphics, SIGGRAPH 97 Proceedings, 379-387. Google Scholar
35. SMITH, A. R., AND BLINN, J. F. 1996. Blue screen matting. In Computer Graphics, vol. 30 of SIGGRAPH 96 Proceedings, 259-268. Google Scholar
36. TURK, G., AND LEVOY, M. 1994. Zippered polygon meshes from range images. In Computer Graphics, SIGGRAPH 94 Proceedings, 311-318. Google Scholar
37. UDUPA, J., AND ODHNER, D. 1993. Shell rendering. IEEE Computer Graphics & Applications 13, 6 (Nov.), 58-67. Google Scholar
38. WOOD, D., AZUMA, D., ALDINGER, K., CURLESS, B., DUCHAMP, T., SALESIN, D., AND STUETZLE, W. 2000. Surface light fields for 3d photography. In Computer Graphics, SIGGRAPH 2000 Proceedings, 287-296. Google Scholar
39. YU, Y., DEBEVEC, P., MALIK, J., AND HAWKINS, T. 1999. Inverse global illumination: Recovering reflectance models of real scenes from photographs. In Computer Graphics, SIGGRAPH 99 Proceedings, 215-224. Google Scholar
40. ZONGKER, D., WERNER, D., CURLESS, B., AND SALESIN, D. 1999. Environment matting and compositing. In Computer Graphics, SIGGRAPH 99 Proceedings, 205-214. Google Scholar
41. ZWICKER, M., PFISTER., H., BAAR, J. V., AND GROSS, M. 2001. Surface splatting. In Computer Graphics, SIGGRAPH 2001 Proceedings, 371-378. Google Scholar