“Continuous capture of skin deformation” by Sand, McMillan and Popović

We describe a method for the acquisition of deformable human geometry from silhouettes. Our technique uses a commercial tracking system to determine the motion of the skeleton, then estimates geometry for each bone using constraints provided by the silhouettes from one or more cameras. These silhouettes do not give a complete characterization of the geometry for a particular point in time, but when the subject moves, many observations of the same local geometries allow the construction of a complete model. Our reconstruction algorithm provides a simple mechanism for solving the problems of view aggregation, occlusion handling, hole filling, noise removal, and deformation modeling. The resulting model is parameterized to synthesize geometry for new poses of the skeleton. We demonstrate this capability by rendering the geometry for motion sequences that were not included in the original datasets.

1. ALLEN, B., CURLESS, B., AND POPOVIĆ, Z. 2002. Articulated body deformation from range scan data. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 612–619. Google Scholar
2. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 187–194. Google Scholar
3. BRAND, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:456–463.Google Scholar
4. BREGLER, C., HERTZMANN, A., AND BIERMANN, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:690–696.Google ScholarCross Ref
5. BROOMHEAD, D., AND LOWE, D. 1988. Multivariable functional interpolation and adaptive networks. Complex Systems 2, 3, 321–355.Google Scholar
6. CARR, J. C., BEATSON, R. K., CHERRIE, J. B., MITCHELL, T. J., FRIGHT, W. R., MCCALLUM, B. C., AND EVANS, T. R. 2001. Reconstruction and representation of 3d objects with radial basis functions. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 67–76. Google Scholar
7. COOTES, T. F., EDWARDS, G. J., AND TAYLOR, C. J. 1998. Active appearance models. In Proceedings of the Fifth European Conference on Computer Vision (ECCV), 484–498. Google Scholar
8. GU, X., GORTLER, S. J., HOPPE, H., MCMILLAN, L., BROWN, B. J., AND STONE, A. D. 1999. Silhouette mapping. Tech. Rep. TR-1-99, Harvard.Google Scholar
9. GUENTER, B., GRIMM, C., WOOD, D., MALVAR, H., AND PIGHIN, F. 1998. Making faces. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 55–66. Google Scholar
10. KAKADIARIS, I. A., AND METAXAS, D. 1993. 3d human body model acquisition from multiple views. In Proceedings of the 5th IEEE International Conference on Computer Vision (ICCV), 618–623. Google Scholar
11. KAYDARA. 2001. FiLMBOX Reference Guide. Kaydara Inc., Montréal, Québec.Google Scholar
12. LEWIS, J. P., CORDNER, M., AND FONG, N. 2000. Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 165–172. Google ScholarDigital Library
13. MATUSIK, W., BUEHLER, C., RASKAR, R., GORTLER, S. J., AND MCMILLAN, L. 2000. Image-based visual hulls. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 369–374. Google ScholarDigital Library
14. MIKIĆ, I., TRIVEDI, M., HUNTER, E., AND COSMAN, P. Human body model acquisition and tracking using voxel data. International Journal of Computer Vision. In Press. Google Scholar
15. NEBEL, J.-C., RODRIGUEZ-MIGUEL, F. J., AND COCKSHOTT, W. P. 2001. Stroboscopic stereo rangefinder. In Proceedings of the Third International Conference on 3D Imaging and Modeling, 59–64.Google Scholar
16. NELDER, J. A., AND MEAD, R. 1965. A simplex method for function minimization. Computer Journal 7, 4, 308–313.Google ScholarCross Ref
17. NEVATIA, R., AND BINFORD, T. O. 1977. Description and recognition of curved objects. Artificial Intelligence 8, 1, 77–98.Google ScholarDigital Library
18. PLÄNKERS, R., AND FUA, P. 2001. Articulated soft objects for video-based body modeling. In Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV), I:394–401.Google Scholar
19. RUSINKIEWICZ, S., HALL-HOLT, O., AND LEVOY, M. 2002. Real-time 3d model acquisition. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 438–446. Google Scholar
20. SLOAN, P.-P. J., CHARLES F. ROSE, I., AND COHEN, M. F. 2001. Shape by example. In Proceedings of the 2001 symposium on Interactive 3D Graphics, 135–143. Google Scholar
21. STOKDYK, S., HAHN, K., NOFZ, P., AND ANDERSON, G., 2002. Spiderman: Behind the mask. Special Session of SIGGRAPH 2002.Google Scholar
22. SZELISKI, R., AND TONNESEN, D. 1992. Surface modeling with oriented particle systems. In Proceedings of the 19th annual conference on Computer graphics and interactive techniques (SIGGRAPH), 185–194. Google Scholar
23. THEOBALT, C., MAGNOR, M., SCHUELER, P., AND SEIDEL, H.-P. 2002. Combining 2d feature tracking and volume reconstruction for online video-based human motion capture. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, 96–103. Google Scholar
24. TORRESANI, L., AND BREGLER, C. 2002. Space-time tracking. In Proceedings of the 7th European Conference on Computer Vision (ECCV), 801–812. Google Scholar
25. VEDULA, S., BAKER, S., AND KANADE, T. 2002. Spatio-temporal view interpolation. In Proceedings of the 13th ACM Eurographics Workshop on Rendering, 65–76. Google Scholar
26. VICON. 2003. Vicon iQ Reference Manual. Vicon Motion Systems Inc., Lake Forest, CA.Google Scholar
27. WANG, X. C., AND PHILLIPS, C. 2002. Multi-weight enveloping: least-squares approximation techniques for skin animation. In Proceedings of the ACM SIGGRAPH symposium on Computer animation, 129–138. Google Scholar
28. WÜRMLIN, S., LAMBORAY, E., STAADT, O. G., AND GROSS., M. H. 2002. 3d video recorder. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, 325–334. Google Scholar