“Dynamic shape capture using multi-view photometric stereo” – ACM SIGGRAPH HISTORY ARCHIVES

“Dynamic shape capture using multi-view photometric stereo”

  • ©

Conference:


Type(s):


Title:

    Dynamic shape capture using multi-view photometric stereo

Session/Category Title:   Reconstruction & modeling

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    We describe a system for high-resolution capture of moving 3D geometry, beginning with dynamic normal maps from multiple views. The normal maps are captured using active shape-from-shading (photometric stereo), with a large lighting dome providing a series of novel spherical lighting configurations. To compensate for low-frequency deformation, we perform multi-view matching and thin-plate spline deformation on the initial surfaces obtained by integrating the normal maps. Next, the corrected meshes are merged into a single mesh using a volumetric method. The final output is a set of meshes, which were impossible to produce with previous methods. The meshes exhibit details on the order of a few millimeters, and represent the performance over human-size working volumes at a temporal resolution of 60Hz.

References:


    1. Ahmed, N., Theobalt, C., Dobre, P., Seidel, H.-P., and Thrun, S. 2008. Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. In Computer Vision and Pattern Recognition, 1–8.Google Scholar
    2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: Shape Completion and Animation of People. ACM Transactions on Graphics 24, 3 (Aug.), 408–416. Google ScholarDigital Library
    3. Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In Computer Vision and Pattern Recognition.Google Scholar
    4. Bernardini, F., Rushmeier, H., Martin, I. M., Mittleman, J., and Taubin, G. 2002. Building a digital model of michelangelo’s florentine pietà. IEEE Computer Graphics&Applications 22, 1 (Jan./Feb.), 59–67. Google ScholarDigital Library
    5. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM Transactions on Graphics 27, 3 (Aug.), 99. Google ScholarDigital Library
    6. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proceedings of the 8th European Conference on Computer Vision, 25–36.Google Scholar
    7. Campbell, N., Vogiatzis, G., Hernandez, C., and Cipolla, R. 2007. Automatic 3d object segmentation in multiple views using volumetric graph-cuts. In British Machine Vision Conference.Google Scholar
    8. Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. ACM Transactions on Graphics 22, 3 (July), 569–577. Google ScholarDigital Library
    9. Chang, W., and Zwicker, M. 2008. Automatic registration for articulated shapes. Computer Graphics Forum (Proceedings of SGP 2008) 27, 5, 1459–1468. Google ScholarDigital Library
    10. Corazza, S., Mündermann, L., Chaudhari, A., Demattio, T., Cobelli, C., and Andriacchi, T. P. 2006. A markerless motion capture system to study musculoskeletal biomechanics: Visual hull and simulated annealing approach. Annals of Biomedical Engineering 34, 6 (July), 1019–1029.Google ScholarCross Ref
    11. Criminisi, A., Blake, A., Rother, C., Shotton, J., and Torr, P. H. 2007. Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming. Int. Journal of Computer Vision 71, 1, 89–110. Google ScholarDigital Library
    12. Curless, B., and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 303–312. Google ScholarDigital Library
    13. Davis, J., Marschner, S. R., Garr, M., and Levoy, M. 2002. Filling holes in complex surfaces using volumetric diffusion. In Symposium on 3D Data Processing, Visualization, and Transmission, 428–438.Google Scholar
    14. Davis, J., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 196–302. Google ScholarDigital Library
    15. de Aguiar, E., Theobalt, C., Stoll, C., and Seidel, H.-P. 2007. Marker-less deformable mesh tracking for human shape and motion capture. In Computer Vision and Pattern Recognition.Google Scholar
    16. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Transactions on Graphics 27, 3 (Aug.), 98. Google ScholarDigital Library
    17. Einarsson, P., Chabert, C.-F., Jones, A., Ma, W.-C., Lamond, B., Hawkins, T., Bolas, M., Sylwan, S., and Debevec, P. 2006. Relighting human locomotion with flowed reflectance fields. In Proc. of Eurographics Symposium on Rendering, 183–194. Google ScholarCross Ref
    18. Furukawa, Y., and Ponce, J. 2006. Carved visual hulls for image-based modeling. In European Conference on Computer Vision, 564–577. Google ScholarDigital Library
    19. Hernandez, C., and Schmitt, F. 2004. Silhouette and stereo fusion for 3D object modeling. Computer Vision and Image Understanding 96, 3 (Dec.), 367–392. Google ScholarDigital Library
    20. Hernandez, C., Vogiatzis, G., and Cipolla, R. 2008. Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30, 3, 548–554. Google ScholarDigital Library
    21. Hertzmann, A., and Seitz, S. M. 2005. Example-based photometric stereo: Shape reconstruction with general, varying brdfs. IEEE Trans. Pattern Anal. Mach. Intell. 27, 8, 1254–1264. Google ScholarDigital Library
    22. Hornung, A., and Kobbelt, L. 2006. Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces based on dual graph embedding. In Computer Vision and Pattern Recognition, 503–510. Google ScholarDigital Library
    23. Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-rigid registration under isometric deformations. Computer Graphics Forum (Proc. SGP’08) 27, 5, 1449–1457. Google ScholarDigital Library
    24. Joshi, N., and Kriegman, D. 2007. Shape from varying illumination and viewpoint. In International Conference on Computer Vision.Google Scholar
    25. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In Symposium on Geometry Processing. Google ScholarDigital Library
    26. Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum 27, 5, 1421–1430.Google ScholarDigital Library
    27. Lim, J., Ho, J., Yang, M.-H., and Kriegman, D. 2005. Passive photometric stereo from motion. In International Conference on Computer Vision. Google ScholarDigital Library
    28. Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Rendering Techniques, 183–194. Google ScholarCross Ref
    29. Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proc. Symposium on Geometry Processing, 173–182. Google ScholarDigital Library
    30. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3d geometry. ACM Transactions on Graphics 24, 3 (Aug.), 536–543. Google ScholarDigital Library
    31. Okutomi, M., and Kanade, T. 1993. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 15, 4, 353–363. Google ScholarDigital Library
    32. Pekelny, Y., and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Computer Graphics Forum 27, 2 (Apr.), 399–408.Google ScholarCross Ref
    33. Rander, P. W., Narayanan, P., and Kanade, T. 1997. Virtualized reality: Constructing time-varying virtual worlds from real world events. In IEEE Visualization, 277–284. Google ScholarDigital Library
    34. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM Transactions on Graphics 21, 3 (July), 438–446. Google ScholarDigital Library
    35. Sagawa, R., Osawa, N., and Yagi, Y. 2007. Deformable registration of textured range images by using texture and shape features. In 3DIM ’07: Proceedings of the Sixth International Conference on 3-D Digital Imaging and Modeling, 65–72. Google ScholarDigital Library
    36. Seitz, S. M., and Dyer, C. R. 1999. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision 35, 2, 151–173. Google ScholarDigital Library
    37. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multiview stereo reconstruction algorithms. In Computer Vision and Pattern Recognition, 519–528. Google ScholarDigital Library
    38. Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-time surface reconstruction using incompressible flow. ACM Trans. Graph. 27, 5, 1–10. Google ScholarDigital Library
    39. Starck, J., and Hilton, A. 2003. Model-based multiple view reconstruction of people. In International Conference on Computer Vision, 915–922. Google ScholarDigital Library
    40. Starck, J., and Hilton, A. 2007. Surface capture for performance based animation. IEEE Computer Graphics and Applications 27(3), 21–31. Google ScholarDigital Library
    41. Svoboda, T., Martinec, D., and Pajdla, T. 2005. A convenient multi-camera self-calibration for virtual environments. PRESENCE: Teleoperators and Virtual Environments 14, 4 (August), 407–422. Google ScholarDigital Library
    42. Theobalt, C., Ahmed, N., Lensch, H., Magnor, M., and Seidel, H.-P. 2007. Seeing people in different light-joint shape, motion, and reflectance capture. IEEE Transactions on Visualization and Computer Graphics 13, 4 (July/Aug.), 663–674. Google ScholarDigital Library
    43. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Transactions on Graphics 27, 3 (Aug.), 97. Google ScholarDigital Library
    44. Vogiatzis, G., Torr, P. H. S., and Cipolla, R. 2005. Multiview stereo via volumetric graph-cuts. In Computer Vision and Pattern Recognition, 391–398. Google ScholarDigital Library
    45. Vogiatzis, G., Hernandez, C., and Cipolla, R. 2006. Reconstruction in the round using photometric normals and silhouettes. In 2006 Conference on Computer Vision and Pattern Recognition (CVPR 2006), 1847–1854. Google ScholarDigital Library
    46. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proc. Symposium on Geometry Processing, 49–58. Google ScholarDigital Library
    47. Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of nonrigid shape and motion from real-time 3d scanner data. ACM Transactions on Graphics 28, 2 (Apr.), 15. Google ScholarDigital Library
    48. Woodham, R. J. 1978. Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In Proc. SPIE’s 22nd Annual Technical Symposium, vol. 155.Google Scholar
    49. Zhang, S., and Huang, P. 2006. High-resolution real-time three-dimensional shape measurement. Optical Engineering 45, 12.Google Scholar
    50. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3 (Aug.), 548–558. Google ScholarDigital Library
    51. Zhang, H., Sheffer, A., Cohen-Or, D., Zhou, Q., van Kaick, O., and Tagliasacchi, A. 2008. Deformation-driven shape correspondence. Proc. Symposium on Geometry Processing 27, 5 (July), 1431–1439. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org