“Markerless garment capture” by Bradley, Popa, Sheffer, Heidrich and Boubekeur

  • ©Derek Bradley, Tiberiu Popa, Alla Sheffer, Wolfgang Heidrich, and Tamy Boubekeur




    Markerless garment capture



    A lot of research has recently focused on the problem of capturing the geometry and motion of garments. Such work usually relies on special markers printed on the fabric to establish temporally coherent correspondences between points on the garment’s surface at different times. Unfortunately, this approach is tedious and prevents the capture of off-the-shelf clothing made from interesting fabrics.In this paper, we describe a marker-free approach to capturing garment motion that avoids these downsides. We establish temporally coherent parameterizations between incomplete geometries that we extract at each timestep with a multiview stereo algorithm. We then fill holes in the geometry using a template. This approach, for the first time, allows us to capture the geometry and motion of unpatterned, off-the-shelf garments made from a range of different fabrics.


    1. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graphics (Proc. SIGGRAPH), 587–594. Google ScholarDigital Library
    2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: shape completion and animation of people. ACM Trans. Graphics (Proc. SIGGRAPH), 408–416. Google ScholarDigital Library
    3. Bhat, K., Twigg, C., Hodgins, J., Khosla, P., Popovic, Z., and Seitz., S. 2003. Estimating cloth simulation parameters from video. In Proc. SCA, 37–51. Google ScholarDigital Library
    4. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graphics (Proc. SIGGRAPH), 33. Google ScholarDigital Library
    5. Boubekeur, T., Reuter, P., and Schlick, C. 2005. Visualization of point-based surfaces with locally reconstructed subdivision surfaces. In Proc. Shape Modeling International. Google ScholarDigital Library
    6. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.Google Scholar
    7. Cheung, K., Baker, S., and Kanade, T. 2005. Shape-from-silhouette across time part ii: Applications to human modeling and markerless motion tracking. IJCV 63, 3, 225–245. Google ScholarDigital Library
    8. de Aguiar, E., Theobalt, C., Stoll, C., and Seidel, H.-P. 2007. Marker-less deformable mesh tracking for human shape and motion capture. In Proc. CVPR, 1–8.Google Scholar
    9. Fiala, M., and Shu, C. 2005. Fully automatic camera calibration using self-identifying calibration targets. Tech. Rep. NRC-48306 ERB-1130, National Research Council of Canada.Google Scholar
    10. Fischler, M., and Bolles, R. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
    11. Floater, M. 2003. Mean value coordinates. Comput. Aided Geom. Des. 20, 1, 19–27. Google ScholarDigital Library
    12. Goldenthal, R., Harmon, D., Fattal, R., Bercovier, M., and Grinspun, E. 2007. Efficient simulation of inextensible cloth. ACM Trans. Graphics (Proc. SIGGRAPH), 49. Google ScholarDigital Library
    13. Guskov, I., Klibanov, S., and Bryant, B. 2003. Trackable surfaces. In Proc. SCA, 251–257. Google ScholarDigital Library
    14. Hasler, N., Asbach, M., Rosenhahn, B., Ohm, J.-R., and Seidel, H.-P. 2006. Physically based tracking of cloth. In Proc. Workshop on Vision, Modeling, and Visualization, 49–56.Google Scholar
    15. Hernandez, C., Vogiatzis, G., Brostow, G. J., Stenger, B., and Cipolla, R. 2007. Non-rigid photometric stereo with colored lights. In Proc. ICCV.Google Scholar
    16. Kraevoy, V., and Sheffer, A. 2004. Cross-parameterization and compatible remeshing of 3d models. ACM Trans. Graphics (Proc. SIGGRAPH), 861–869. Google ScholarDigital Library
    17. Kraevoy, V., and Sheffer, A. 2005. Template-based mesh completion. In Proc. SGP, 13. Google ScholarDigital Library
    18. Mitra, N., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proc. SGP, 173–182. Google ScholarDigital Library
    19. Mori, Y., and Igarashi, T. 2007. Plushie: an interactive design system for plush toys. ACM Trans. Graphics (Proc. SIGGRAPH), 45. Google ScholarDigital Library
    20. MVIEW. http://vision.middlebury.edu/mview/.Google Scholar
    21. Praun, E., Sweldens, W., and Schröder, P. 2001. Consistent mesh parameterizations. In ACM Trans. Graphics (Proc. SIGGRAPH), 179–184. Google ScholarDigital Library
    22. Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. In Proc. Eurographics, 263–271. Google ScholarDigital Library
    23. Rosenhahn, B., Kersting, U., Powell, K., Klette, R., Klette, G., and Seidel, H.-P. 2007. A system for articulated tracking incorporating a clothing model. Machine Vision and Applications 18, 1, 25–40. Google ScholarDigital Library
    24. Scholz, V., and Magnor, M. A. 2004. Cloth motion from optical flow. In Proc. Vision, Modeling and Visualization 2004, 117–124.Google Scholar
    25. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Proc. Eurographics, 439–448. Google ScholarDigital Library
    26. Schreiner, J., Asirvatham, A., Praun, E., and Hoppe, H. 2004. Inter-surface mapping. ACM Trans. Graphics (Proc. SIGGRAPH), 870–877. Google ScholarDigital Library
    27. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. CVPR, 519–528. Google ScholarDigital Library
    28. Sheffer, A., and Kraevoy, V. 2004. Pyramid Coordinates for morphing and deformation. In Proc. 3DPVT, 68–75. Google ScholarCross Ref
    29. Shewchuk, J. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. In Applied Computational Geometry: Towards Geometric Engineering, M. C. Lin and D. Manocha, Eds., vol. 1148 of Lecture Notes in Computer Science. Springer-Verlag, 203–222. Google ScholarDigital Library
    30. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839. Google ScholarDigital Library
    31. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proc. SGP, 49–58. Google ScholarDigital Library
    32. White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. ACM Trans. Graphics (Proc. SIGGRAPH), 34. Google ScholarDigital Library
    33. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graphics (Proc. SIGGRAPH), 548–558. Google ScholarDigital Library

ACM Digital Library Publication: