“Markerless garment capture” by Bradley, Popa, Sheffer, Heidrich and Boubekeur

A lot of research has recently focused on the problem of capturing the geometry and motion of garments. Such work usually relies on special markers printed on the fabric to establish temporally coherent correspondences between points on the garment’s surface at different times. Unfortunately, this approach is tedious and prevents the capture of off-the-shelf clothing made from interesting fabrics.In this paper, we describe a marker-free approach to capturing garment motion that avoids these downsides. We establish temporally coherent parameterizations between incomplete geometries that we extract at each timestep with a multiview stereo algorithm. We then fill holes in the geometry using a template. This approach, for the first time, allows us to capture the geometry and motion of unpatterned, off-the-shelf garments made from a range of different fabrics.

1. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graphics (Proc. SIGGRAPH), 587–594. Google ScholarDigital Library
2. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: shape completion and animation of people. ACM Trans. Graphics (Proc. SIGGRAPH), 408–416. Google ScholarDigital Library
3. Bhat, K., Twigg, C., Hodgins, J., Khosla, P., Popovic, Z., and Seitz., S. 2003. Estimating cloth simulation parameters from video. In Proc. SCA, 37–51. Google ScholarDigital Library
4. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graphics (Proc. SIGGRAPH), 33. Google ScholarDigital Library
5. Boubekeur, T., Reuter, P., and Schlick, C. 2005. Visualization of point-based surfaces with locally reconstructed subdivision surfaces. In Proc. Shape Modeling International. Google ScholarDigital Library
6. Bradley, D., Boubekeur, T., and Heidrich, W. 2008. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. In Proc. CVPR.Google Scholar
7. Cheung, K., Baker, S., and Kanade, T. 2005. Shape-from-silhouette across time part ii: Applications to human modeling and markerless motion tracking. IJCV 63, 3, 225–245. Google ScholarDigital Library
8. de Aguiar, E., Theobalt, C., Stoll, C., and Seidel, H.-P. 2007. Marker-less deformable mesh tracking for human shape and motion capture. In Proc. CVPR, 1–8.Google Scholar
9. Fiala, M., and Shu, C. 2005. Fully automatic camera calibration using self-identifying calibration targets. Tech. Rep. NRC-48306 ERB-1130, National Research Council of Canada.Google Scholar
10. Fischler, M., and Bolles, R. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381–395. Google ScholarDigital Library
11. Floater, M. 2003. Mean value coordinates. Comput. Aided Geom. Des. 20, 1, 19–27. Google ScholarDigital Library
12. Goldenthal, R., Harmon, D., Fattal, R., Bercovier, M., and Grinspun, E. 2007. Efficient simulation of inextensible cloth. ACM Trans. Graphics (Proc. SIGGRAPH), 49. Google ScholarDigital Library
13. Guskov, I., Klibanov, S., and Bryant, B. 2003. Trackable surfaces. In Proc. SCA, 251–257. Google ScholarDigital Library
14. Hasler, N., Asbach, M., Rosenhahn, B., Ohm, J.-R., and Seidel, H.-P. 2006. Physically based tracking of cloth. In Proc. Workshop on Vision, Modeling, and Visualization, 49–56.Google Scholar
15. Hernandez, C., Vogiatzis, G., Brostow, G. J., Stenger, B., and Cipolla, R. 2007. Non-rigid photometric stereo with colored lights. In Proc. ICCV.Google Scholar
16. Kraevoy, V., and Sheffer, A. 2004. Cross-parameterization and compatible remeshing of 3d models. ACM Trans. Graphics (Proc. SIGGRAPH), 861–869. Google ScholarDigital Library
17. Kraevoy, V., and Sheffer, A. 2005. Template-based mesh completion. In Proc. SGP, 13. Google ScholarDigital Library
18. Mitra, N., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proc. SGP, 173–182. Google ScholarDigital Library
19. Mori, Y., and Igarashi, T. 2007. Plushie: an interactive design system for plush toys. ACM Trans. Graphics (Proc. SIGGRAPH), 45. Google ScholarDigital Library
20. MVIEW. http://vision.middlebury.edu/mview/.Google Scholar
21. Praun, E., Sweldens, W., and Schröder, P. 2001. Consistent mesh parameterizations. In ACM Trans. Graphics (Proc. SIGGRAPH), 179–184. Google ScholarDigital Library
22. Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. In Proc. Eurographics, 263–271. Google ScholarDigital Library
23. Rosenhahn, B., Kersting, U., Powell, K., Klette, R., Klette, G., and Seidel, H.-P. 2007. A system for articulated tracking incorporating a clothing model. Machine Vision and Applications 18, 1, 25–40. Google ScholarDigital Library
24. Scholz, V., and Magnor, M. A. 2004. Cloth motion from optical flow. In Proc. Vision, Modeling and Visualization 2004, 117–124.Google Scholar
25. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Proc. Eurographics, 439–448. Google ScholarDigital Library
26. Schreiner, J., Asirvatham, A., Praun, E., and Hoppe, H. 2004. Inter-surface mapping. ACM Trans. Graphics (Proc. SIGGRAPH), 870–877. Google ScholarDigital Library
27. Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. CVPR, 519–528. Google ScholarDigital Library
28. Sheffer, A., and Kraevoy, V. 2004. Pyramid Coordinates for morphing and deformation. In Proc. 3DPVT, 68–75. Google ScholarCross Ref
29. Shewchuk, J. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. In Applied Computational Geometry: Towards Geometric Engineering, M. C. Lin and D. Manocha, Eds., vol. 1148 of Lecture Notes in Computer Science. Springer-Verlag, 203–222. Google ScholarDigital Library
30. Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839. Google ScholarDigital Library
31. Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proc. SGP, 49–58. Google ScholarDigital Library
32. White, R., Crane, K., and Forsyth, D. 2007. Capturing and animating occluded cloth. ACM Trans. Graphics (Proc. SIGGRAPH), 34. Google ScholarDigital Library
33. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graphics (Proc. SIGGRAPH), 548–558. Google ScholarDigital Library