“Capturing and animating occluded cloth” by White, Crane and Forsyth

We capture the shape of moving cloth using a custom set of color markers printed on the surface of the cloth. The output is a sequence of triangle meshes with static connectivity and with detail at the scale of individual markers in both smooth and folded regions. We compute markers’ coordinates in space using correspondence across multiple synchronized video cameras. Correspondence is determined from color information in small neighborhoods and refined using a novel strain pruning process. Final correspondence does not require neighborhood information. We use a novel data driven hole-filling technique to fill occluded regions. Our results include several challenging examples: a wrinkled shirt sleeve, a dancing pair of pants, and a rag tossed onto a cup. Finally, we demonstrate that cloth capture is reusable by animating a pair of pants using human motion capture data.

1. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. In SIGGRAPH. Google ScholarDigital Library
2. Baraff, D., Witkin, A., and Kass, M. 2003. Untangling cloth. In SIGGRAPH. Google ScholarDigital Library
3. Bhat, K., Twigg, C., Hodgins, J. K., Khosla, P., Popovic, Z., and Seitz, S. 2003. Estimating cloth simulation parameters from video. In SCA. Google ScholarDigital Library
4. Bouguet, J.-Y., 2005. Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/.Google Scholar
5. Bradley, D., Roth, G., and Bose, P. 2005. Augmented clothing. In Graphics Interface.Google Scholar
6. Bridson, R., Marino, S., and Fedkiw, R. 2003. Simulation of clothing with folds and wrinkles. In SCA. Google ScholarDigital Library
7. Choi, K.-J., and Ko, H.-S. 2002. Stable but responsive cloth. In SIGGRAPH. Google ScholarDigital Library
8. Cover, T. M., and Thomas, J. A. 1991. Information Theory. John Wiley and Sons.Google Scholar
9. Forsyth, D., and Ponce, J. 2002. Computer Vision: a modern approach. Prentice-Hall. Google ScholarDigital Library
10. Guskov, I., and Zhukov, L. 2002. Direct pattern tracking on flexible geometry. In WSCG.Google Scholar
11. Guskov, I., Klibanov, S., and Bryant, B. 2003. Trackable surfaces. In SCA. Google ScholarDigital Library
12. Hartley, R., and Zisserman, A. 2000. Multiple View Geometry. Cambridge University Press. Google ScholarDigital Library
13. Hasler, N., Asbach, M., Rosenhahn, B., Ohm, J.-R., and Seidel, H.-P. 2006. Physically based tracking of cloth. In VMV.Google Scholar
14. House, D., and Breen, D., Eds. 2000. Cloth Modelling and Animation. A. K. Peters. Google ScholarDigital Library
15. Kircher, S., and Garland, M. 2006. Editing arbitrarily deforming surface animations. In SIGGRAPH. Google ScholarDigital Library
16. Lin, W.-C., and Liu, Y. 2006. Tracking dynamic near-regular textures under occlusion and rapid movements. In ECCV. Google ScholarDigital Library
17. Lipman, Y., Sorkine, O., Levin, D., and Cohen-Or, D. 2005. Linear rotation-invariant coordinates for meshes. In SIGGRAPH. Google ScholarDigital Library
18. Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV. Google ScholarDigital Library
19. Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. In SIGGRAPH. Google ScholarDigital Library
20. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. PAMI. Google ScholarDigital Library
21. Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. Eurographics.Google Scholar
22. Provot, X. 1995. Deformation constraints in a mass-spring model to describe rigid cloth behavior. In Graphics Interface.Google Scholar
23. Scholz, V., and Magnor, M. A. 2004. Cloth motion from optical flow. In VMV.Google Scholar
24. Scholz, V., and Magnor, M. 2006. Texture replacement of garments in monocular video sequences. In Rendering Techniques. Google ScholarCross Ref
25. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Eurographics.Google Scholar
26. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Symposium of Geometry Processing. Google ScholarDigital Library
27. Sumner, R. W., and Popovic, J. 2004. Deformation transfer for triangle meshes. In SIGGRAPH. Google ScholarDigital Library
28. Sumner, R. W., Zwicker, M., Gotsman, C., and Popovic, J. 2005. Mesh-based inverse kinematics. In SIGGRAPH. Google ScholarDigital Library
29. Tanie, H., Yamane, K., and Nakamura, Y. 2005. High marker density motion capture by retroreflective mesh suit. In ICRA.Google Scholar
30. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In SIGGRAPH. Google ScholarDigital Library
31. White, R., and Forsyth, D., 2005. Deforming objects provide better camera calibration. UC Berkeley Technical Report.Google Scholar
32. White, R., and Forsyth, D. 2006. Retexturing single views using texture and shading. In ECCV. Google ScholarDigital Library
33. White, R., Lobay, A., and Forsyth, D., 2005. Cloth capture. UC Berkeley Technical Report.Google Scholar
34. White, R., Forsyth, D., and Vasanth, J., 2006. Capturing real folds in cloth. UC Berkeley Technical Report.Google Scholar
35. Zhang, Z. 2002. A flexible new technique for camera calibration. PAMI. Google ScholarDigital Library