“Dynamic hair capture using spacetime optimization” by Xu, Wu, Wang, Zheng, Tong, et al. …
Conference:
Type(s):
Title:
- Dynamic hair capture using spacetime optimization
Session/Category Title: Capturing Everything
Presenter(s)/Author(s):
Abstract:
Dynamic hair strands have complex structures and experience intricate collisions and occlusion, posing significant challenges for high-quality reconstruction of their motions. We present a comprehensive dynamic hair capture system for reconstructing realistic hair motions from multiple synchronized video sequences. To recover hair strands’ temporal correspondence, we propose a motion-path analysis algorithm that can robustly track local hair motions in input videos. To ensure the spatial and temporal coherence of the dynamic capture, we formulate the global hair reconstruction as a spacetime optimization problem solved iteratively. Demonstrated using a range of real-world hairstyles driven by different wind conditions and head motions, our approach is able to reconstruct complex hair dynamics matching closely with video recordings both in terms of geometry and motion details.
References:
1. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30 (Aug.), 75:1–75:10.
2. Beeler, T., Bickel, B., Noris, G., Beardsley, P., Marschner, S., Sumner, R. W., and Gross, M. 2012. Coupled 3D reconstruction of sparse facial hair and skin. ACM Trans. Graph. 31, 4, 117:1–10.
3. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July).
4. Bolles, R. C., Baker, H. H., and Marimont, D. H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision 1, 7–55.Cross Ref
5. Bonneel, N., Paris, S., Panne, M. V. D., Durand, F., and Drettakis, G. 2009. Single photo estimation of hair appearance. Computer Graphics Forum 28, 1171–1180.
6. Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boubekeur, T. 2008. Markerless garment capture. ACM Trans. Graph. 27, 3 (Aug.), 99:1–99:9.
7. Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. on Graphics (Proc. SIGGRAPH) 29, 3.
8. Bradski, G. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools.
9. Cai, M., Zheng, C., and Zhou, K. 2014. A reduced model for interactive hairs. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014) 33, 4 (Aug.).
10. Chai, M., Wang, L., Yu, Y., Weng, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1–8.
11. Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4.
12. Chen, Y., Davis, T. A., Hager, W. W., and Rajamanickam, S. 2009. Algorithm 887: Cholmod, supernodal sparse cholesky factorization and update/downdate. ACM Trans. Math. Softw..
13. Chen, Q., Li, D., and Tang, C.-K. 2012. KNN matting. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, 869–876.
14. Davis, T. A., and Hager, W. W. 2009. Dynamic supernodes in sparse cholesky update/downdate and triangular solves. ACM Trans. Math. Softw. 35, 4.
15. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1–98:10.
16. Fu, H., Wei, Y., Tai, C.-L., and Quan, L. 2007. Sketching hairstyles. In Proc. EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, 31–36.
17. Furukawa, Y., and Ponce, J. 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans. PAMI 32, 8, 1362–1376.
18. Grabli, S., Sillion, F., Marschner, S., and Lengyel, J. E. 2002. Image-based hair capture by inverse lighting. In Proc. Graphics Interface.
19. Gregson, J., Krimerman, M., Hullin, M. B., and Heidrich, W. 2012. Stochastic tomography and its applications in 3D imaging of mixing fluids. ACM Trans. Graph. 31, 4 (July), 52:1–52:10.
20. Herrera, T. L., Zinke, A., and Weber, A. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Trans. Graph. 31, 6, 146:1–9.
21. Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph..
22. Ihrke, I., and Magnor, M. 2004. Image-based tomographic reconstruction of flames. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA ’04, 365–373.
23. Ishikawa, T., Kazama, Y., Sugisaki, E., and Morishima, S. 2007. Hair motion reconstruction using motion capture system. ACM SIGGRAPH 2007 posters.
24. Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1–9.
25. Kim, C., Zimme R, H., Pritch, Y., Sorkine-Hornung, A., and Gross, M. 2013. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 4, 73:1–12.
26. Kong, W., Takahashi, H., and Nakajima, M. 1997. Generation of 3D hair model from multiple pictures. In Proc. Multimedia Modeling.
27. Li, Y., Fan, X., Mitra, N. J., Chamovitz, D., Cohen-Or, D., and Chen, B. 2013. Analyzing growing plants from 4D point cloud data. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2013) 32, 6.
28. Luo, L., Li, H., Weise, T., Paris, S., Pauly, M., and Rusinkiewicz, S. 2011. Dynamic hair capture. Tech. rep., Princeton University.
29. Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4.
30. Luo, L., Zhang, C., Zhang, Z., and Rusinkiewicz, S. 2013. Wide-baseline hair capture using strand-based refinement. Computer Vision and Pattern Recognition.
31. McLachlan, G. J., and Krishnan, T. 1997. The EM algorithm and extensions. In Wiley series in probability and statistics. John Wiley & Sons.
32. Müller, M., Heidelberger, B., Hennix, M., and Ratcliff, J. 2007. Position based dynamics. Journal of Visual Communication and Image Representation 18, 2, 109–118.
33. Paris, S., Briceño, H., and Sillion, F. 2004. Capture of hair geometry from multiple images. ACM Trans. Graph. 23, 3, 712–719.
34. Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1–9.
35. Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Roessl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Symposium on Geometry Processing.
36. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3 (Aug.), 97:1–97:9.
37. Wang, H., Liao, M., Zhang, Q., Yang, R., and Turk, G. 2009. Physically guided liquid surface modeling from videos. ACM Trans. Graph. 28, 3 (July), 90:1–90:11.
38. Wang, L., Yu, Y., Zhou, K., and Guo, B. 2009. Example-based hair geometry synthesis. ACM Trans. Graph. 28, 3, 56:1–9.
39. Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: styling, simulation, and rendering. IEEE Transactions on Visualization and Computer Graphics 13, 2, 213–234.
40. Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. ACM Trans. Graph. 24, 3, 816–820.
41. Wu, C., Agarwal, S., Curless, B., and Seitz, S. M. 2011. Multicore bundle adjustment. In Proc. CVPR.
42. Yamaguchi, T., Wilburn, B., and Ofek, E. 2009. Video-based modeling of dynamic hair. In the 3rd Pacific Rim Symposium on Advances in Image and Video Technology, 585–596.
43. Zhang, Q., Tong, J., Wang, H., Pan, Z., and Yang, R. 2012. Simulation guided hair dynamics modeling from video. Computer Graphics Forum 31, 7, 2003–2010.
