“3D self-portraits” by Li, Vouga, Gudym, Luo, Barron, et al. …
Conference:
Type(s):
Title:
- 3D self-portraits
Session/Category Title: Avatars
Presenter(s)/Author(s):
Abstract:
We develop an automatic pipeline that allows ordinary users to capture complete and fully textured 3D models of themselves in minutes, using only a single Kinect sensor, in the uncontrolled lighting environment of their own home. Our method requires neither a turntable nor a second operator, and is robust to the small deformations and changes of pose that inevitably arise during scanning. After the users rotate themselves with the same pose for a few scans from different views, our system stitches together the captured scans using multi-view non-rigid registration, and produces watertight final models. To ensure consistent texturing, we recover the underlying albedo from each scanned texture and generate seamless global textures using Poisson blending. Despite the minimal requirements we place on the hardware and users, our method is suitable for full body capture of challenging scenes that cannot be handled well using previous methods, such as those involving loose clothing, complex poses, and props.
References:
1. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. Scape: shape completion and animation of people. In ACM SIGGRAPH 2005 Papers, ACM, New York, NY, USA.
2. Barron, J. T., and Malik, J. 2012. Shape, albedo, and illumination from a single image of an unknown object. CVPR.
3. Barron, J. T., and Malik, J. 2013. Intrinsic scene properties from a single rgb-d image. CVPR.
4. Bojsen-Hansen, M., Li, H., and Wojtan, C. 2011. Tracking surfaces with evolving topology. ACM Transactions on Graphics (Proceedings SIGGRAPH 2012) 31, 4.
5. Brown, B., and Rusinkiewicz, S. 2007. Global non-rigid alignment of 3-D scans. ACM Transactions on Graphics (Proc. SIGGRAPH) 26, 3 (Aug.).
6. Chang, W., and Zwicker, M. 2008. Automatic registration for articulated shapes. Computer Graphics Forum (Proceedings of SGP 2008) 27, 5.
7. Chang, W., and Zwicker, M. 2009. Range scan registration using reduced deformable models. EG.
8. Chang, W., and Zwicker, M. 2011. Global registration of dynamic range scans for articulated model reconstruction. ACM Transactions on Graphics, to appear 30, 3.
9. Chen, Y., Davis, T. A., Hager, W. W., and Rajamanickam, S. 2008. Algorithm 887: Cholmod, supernodal sparse cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35, 3 (Oct.), 22:1–22:14.
10. Chuang, M., Luo, L., Brown, B. J., Rusinkiewicz, S., and Kazhdan, M. 2009. Estimating the Laplace-Beltrami operator by restricting 3D functions. SGP 2009 (July).
11. Cui, Y., Chang, W., Noll, T., and Stricker, D. 2012. Kinectavatar: Fully automatic body capture using a single kinect. In Asian Conference on Computer Vision (ACCV).
12. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. In Computer Graphics Forum (Proc. Eurographics 2008), P. Dutr’e and M. Stamminger, Eds., vol. 2.
13. Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-rigid registration under isometric deformations. In Proceedings of the Symposium on Geometry Processing, Eurographics Association, SGP ’08.
14. Kazhdan, M., Bolitho, M., and Hoppe, H. 2006. Poisson surface reconstruction. In Proceedings of the fourth Eurographics symposium on Geometry processing, EG Assoc.
15. Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum (Proc. SGP’08) 27, 5 (July).
16. Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Transactions on Graphics (SIGGRAPH Asia 2009) 28, 5 (December).
17. Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M., and Rusinkiewicz, S. 2012. Temporally coherent completion of dynamic shapes. ACM ToG 31, 1 (January).
18. Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proceedings of the fifth Eurographics symposium on Geometry processing, Eurographics Association, SGP ’07.
19. Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., and Fitzgibbon, A. 2011. Kinectfusion: Real-time dense surface mapping and tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, IEEE Computer Society, ISMAR ’11, 127–136.
20. Pulli, K. 1999. Multiview registration for large data sets. In Proceedings of the 2nd international conference on 3-D digital imaging and modeling.
21. Rusinkiewicz, S., and Levoy, M. 2001. Efficient variants of the ICP algorithm. In Third International Conference on 3D Digital Imaging and Modeling (3DIM).
22. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3d model acquisition. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA.
23. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R., 2006. A comparison and evaluation of multi-view stereo reconstruction algorithms.
24. Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-time surface reconstruction using incompressible flow. In ACM SIGGRAPH Asia 2008 papers, ACM, New York.
25. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from single depth images. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Washington, DC, USA.
26. Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., and Seidel, H.-P. 2012. Animation cartography–intrinsic reconstruction of shape and motion. ACM Trans. Graph. 31, 2 (Apr.).
27. Tong, J., Zhou, J., Liu, L., Pan, Z., and Yan, H. 2012. Scanning 3d full human bodies using kinects. IEEE Transactions on Visualization and Computer Graphics 18, 4.
28. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., and Matusik, W. 2009. Dynamic shape capture using multi-view photometric stereo. ACM Transactions on Graphics 28, 5, 174.
29. Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of nonrigid shape and motion from real-time 3d scanner data. ACM Trans. Graph. 28, 2 (May).
30. Wei, X., Zhang, P., and Chai, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (Nov.).
31. Weise, T., Wismer, T., Leibe, B., and Gool, L. V. 2011. Online loop closure for real-time interactive 3d scanning. Comput. Vis. Image Underst. 115, 5 (May).
32. Weiss, A., Hirshberg, D., and Black, M. 2011. Home 3D body scans from noisy image and range data. In Int. Conf. on Computer Vision (ICCV), IEEE, Barcelona.
