“Capturing and animating occluded cloth” by White, Crane and Forsyth

  • ©Ryan White, Keenan Crane, and David A. Forsyth




    Capturing and animating occluded cloth



    We capture the shape of moving cloth using a custom set of color markers printed on the surface of the cloth. The output is a sequence of triangle meshes with static connectivity and with detail at the scale of individual markers in both smooth and folded regions. We compute markers’ coordinates in space using correspondence across multiple synchronized video cameras. Correspondence is determined from color information in small neighborhoods and refined using a novel strain pruning process. Final correspondence does not require neighborhood information. We use a novel data driven hole-filling technique to fill occluded regions. Our results include several challenging examples: a wrinkled shirt sleeve, a dancing pair of pants, and a rag tossed onto a cup. Finally, we demonstrate that cloth capture is reusable by animating a pair of pants using human motion capture data.


    1. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. In SIGGRAPH. Google ScholarDigital Library
    2. Baraff, D., Witkin, A., and Kass, M. 2003. Untangling cloth. In SIGGRAPH. Google ScholarDigital Library
    3. Bhat, K., Twigg, C., Hodgins, J. K., Khosla, P., Popovic, Z., and Seitz, S. 2003. Estimating cloth simulation parameters from video. In SCA. Google ScholarDigital Library
    4. Bouguet, J.-Y., 2005. Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/.Google Scholar
    5. Bradley, D., Roth, G., and Bose, P. 2005. Augmented clothing. In Graphics Interface.Google Scholar
    6. Bridson, R., Marino, S., and Fedkiw, R. 2003. Simulation of clothing with folds and wrinkles. In SCA. Google ScholarDigital Library
    7. Choi, K.-J., and Ko, H.-S. 2002. Stable but responsive cloth. In SIGGRAPH. Google ScholarDigital Library
    8. Cover, T. M., and Thomas, J. A. 1991. Information Theory. John Wiley and Sons.Google Scholar
    9. Forsyth, D., and Ponce, J. 2002. Computer Vision: a modern approach. Prentice-Hall. Google ScholarDigital Library
    10. Guskov, I., and Zhukov, L. 2002. Direct pattern tracking on flexible geometry. In WSCG.Google Scholar
    11. Guskov, I., Klibanov, S., and Bryant, B. 2003. Trackable surfaces. In SCA. Google ScholarDigital Library
    12. Hartley, R., and Zisserman, A. 2000. Multiple View Geometry. Cambridge University Press. Google ScholarDigital Library
    13. Hasler, N., Asbach, M., Rosenhahn, B., Ohm, J.-R., and Seidel, H.-P. 2006. Physically based tracking of cloth. In VMV.Google Scholar
    14. House, D., and Breen, D., Eds. 2000. Cloth Modelling and Animation. A. K. Peters. Google ScholarDigital Library
    15. Kircher, S., and Garland, M. 2006. Editing arbitrarily deforming surface animations. In SIGGRAPH. Google ScholarDigital Library
    16. Lin, W.-C., and Liu, Y. 2006. Tracking dynamic near-regular textures under occlusion and rapid movements. In ECCV. Google ScholarDigital Library
    17. Lipman, Y., Sorkine, O., Levin, D., and Cohen-Or, D. 2005. Linear rotation-invariant coordinates for meshes. In SIGGRAPH. Google ScholarDigital Library
    18. Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV. Google ScholarDigital Library
    19. Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. In SIGGRAPH. Google ScholarDigital Library
    20. Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. PAMI. Google ScholarDigital Library
    21. Pritchard, D., and Heidrich, W. 2003. Cloth motion capture. Eurographics.Google Scholar
    22. Provot, X. 1995. Deformation constraints in a mass-spring model to describe rigid cloth behavior. In Graphics Interface.Google Scholar
    23. Scholz, V., and Magnor, M. A. 2004. Cloth motion from optical flow. In VMV.Google Scholar
    24. Scholz, V., and Magnor, M. 2006. Texture replacement of garments in monocular video sequences. In Rendering Techniques. Google ScholarCross Ref
    25. Scholz, V., Stich, T., Keckeisen, M., Wacker, M., and Magnor, M. 2005. Garment motion capture using color-coded patterns. In Eurographics.Google Scholar
    26. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Symposium of Geometry Processing. Google ScholarDigital Library
    27. Sumner, R. W., and Popovic, J. 2004. Deformation transfer for triangle meshes. In SIGGRAPH. Google ScholarDigital Library
    28. Sumner, R. W., Zwicker, M., Gotsman, C., and Popovic, J. 2005. Mesh-based inverse kinematics. In SIGGRAPH. Google ScholarDigital Library
    29. Tanie, H., Yamane, K., and Nakamura, Y. 2005. High marker density motion capture by retroreflective mesh suit. In ICRA.Google Scholar
    30. Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In SIGGRAPH. Google ScholarDigital Library
    31. White, R., and Forsyth, D., 2005. Deforming objects provide better camera calibration. UC Berkeley Technical Report.Google Scholar
    32. White, R., and Forsyth, D. 2006. Retexturing single views using texture and shading. In ECCV. Google ScholarDigital Library
    33. White, R., Lobay, A., and Forsyth, D., 2005. Cloth capture. UC Berkeley Technical Report.Google Scholar
    34. White, R., Forsyth, D., and Vasanth, J., 2006. Capturing real folds in cloth. UC Berkeley Technical Report.Google Scholar
    35. Zhang, Z. 2002. A flexible new technique for camera calibration. PAMI. Google ScholarDigital Library

ACM Digital Library Publication:

Overview Page: