“Global Registration of Dynamic Range Scans for Articulated Model Reconstruction” by Chang and Zwicker

We present the articulated global registration algorithm to reconstruct articulated 3D models from dynamic range scan sequences. This new algorithm aligns multiple range scans simultaneously to reconstruct a full 3D model from the geometry of these scans. Unlike other methods, we express the surface motion in terms of a reduced deformable model and solve for joints and skinning weights. This allows a user to interactively manipulate the reconstructed 3D model to create new animations.
We express the global registration as an optimization of both the alignment of the range scans and the articulated structure of the model. We employ a graph-based representation for the skinning weights that successfully handles difficult topological cases well. Joints between parts are estimated automatically and are used in the optimization to preserve the connectivity between parts. The algorithm also robustly handles difficult cases where parts suddenly disappear or reappear in the range scans. The global registration produces a more accurate registration compared to a sequential registration approach, because it estimates the articulated structure based on the motion observed in all input frames. We show that we can automatically reconstruct a variety of articulated models without the use of markers, user-placed correspondences, segmentation, or template model.

1. Ahmed, N., Theobalt, C., Dobrev, P., Seidel, H.-P., and Thrun, S. 2008. Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
2. Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM SIGGRAPH 21, 3, 612–619.
3. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 587–594.
4. Anguelov, D., Koller, D., Pang, H., Srinivasan, P., and Thrun, S. 2004a. Recovering articulated object models from 3d range data. In Uncertainty in Artificial Intelligence Conference (UAI).
5. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 408–416.
6. Anguelov, D., Srinivasan, P., Pang, H.-C., Koller, D., Thrun, S., and Davis, J. 2004b. The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In Conference on Advances in Neural Information Processing Systems (NIPS).
7. Besl, P. J. and McKay, H. 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 239–256.
8. Boykov, Y. and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9, 1124–1137.
9. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11, 1222–1239.
10. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.
11. Chang, W. and Zwicker, M. 2008. Automatic registration for articulated shapes. Comput. Graph. Forum. 27, 5, 1459–1468.
12. Chang, W. and Zwicker, M. 2009. Range scan registration using reduced deformable models. Comput. Graph. Forum. 28, 2, 447–456.
13. Cheung, G. K. M., Baker, S., and Kanade, T. 2003. Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 77–84.
14. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008a. Performance capture from sparse multi-view video. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.
15. de Aguiar, E., Theobalt, C., Thrun, S., and Seidel, H.-P. 2008b. Automatic conversion of mesh animations into skeleton-based animations. Comput. Graph. Forum. 27, 2, 389–397.
16. Gall, J., Stoll, C., de Aguiar, E., Theobalt, C., Rosenhahn, B., and Seidel, H.-P. 2009. Motion capture using joint skeleton tracking and surface estimation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
17. Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-rigid registration under isometric deformations. Comput. Graph. Forum. 27, 5, 1449–1457.
18. James, D. L. and Twigg, C. D. 2005. Skinning mesh animations. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques.
19. Kavan, L., Collins, S., Zára, J., and O’Sullivan, C. 2008. Geometric skinning with approximate dual quaternion blending. ACM Trans. Graph. 27, 4.
20. Knoop, S., Vacek, S., and Dillmann, R. 2005. Modeling joint constraints for an articulated 3d human body model with artificial correspondences in icp. In Proceedings of the IEEE-RAS Conference on Humanoid Robots.
21. Kolmogorov, V. and Zabih, R. 2004. What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26, 2, 147–159.
22. Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single view geometry and motion reconstruction. In Proceedings of the ACM SIGGRAPH ASIA Conference.
23. Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Comput. Graph. Forum. 27, 5, 1421–1430.
24. Manson, J., Petrova, G., and Schaefer, S. 2008. Streaming surface reconstruction using wavelets. Comput. Graph. Forum 27, 5, 1411–1420.
25. Mitchell, D. P. 1991. Spectrally optimal sampling for distribution ray tracing. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
26. Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L. J., and Pottmann, H. 2007. Dynamic geometry registration. In Proceedings of the Symposium on Geometry Processing. 173–182.
27. Neugebauer, P. J. 1997. Reconstruction of real-world objects via simultaneous registration and robust combination of multiple range images. Int. J. Shape Model. 3, 1/2, 71–90.
28. Nishino, K. and Ikeuchi, K. 2002. Robust simultaneous registration of multiple range images. In Proceedings of the Asian Conference on Computer Vision (ACCV).
29. Park, S. I. and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
30. Park, S. I. and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
31. Pauly, M., Mitra, N. J., Giesen, J., Gross, M. H., and Guibas, L. J. 2005. Example-based 3d scan completion. In Proceedings of the Symposium on Geometry Processing. 23–32.
32. Pekelny, Y. and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Comput. Graph. Forum. 27, 2, 399–408.
33. Popa, T., South-Dickinson, I., Bradley, D., Sheffer, A., and Heidrich, W. 2010. Globally consistent space-time reconstruction. Comput. Graph. Forum. 29, 5, 1633–1642.
34. Schaefer, S. and Yuksel, C. 2007. Example-Based skeleton extraction. In Proceedings of the Symposium on Geometry Processing. 153–162.
35. Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-Time surface reconstruction using incompressible flow. In Proceedings of the ACM SIGGRAPH ASIA Conference.
36. Sumner, R. W., Schmid, J., and Pauly, M. 2007. Embedded deformation for shape manipulation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
37. Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-based inverse kinematics. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
38. Süssmuth, J., Winter, M., and Greiner, G. 2008. Reconstructing animated meshes from time-varying point clouds. Comput. Graph. Forum. 27, 5, 1469–1476.
39. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.
40. Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of non-rigid shape and motion from real-time 3d scanner data. ACM Trans, Graph. 28.
41. Weise, T., Leibe, B., and Gool, L. J. V. 2007. Fast 3d scanning with automatic motion compensation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
42. Weise, T., Li, H., Gool, L. J. V., and Pauly, M. 2009. Face/off: live facial puppetry. In Proceedings of the Symposium on Computer Animation. 7–16.
43. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics Interactive Techniques.