“Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling” by Gao, Lai, Liang, Chen and Xia

  • ©Lin Gao, Yu-Kun Lai, Dun Liang, Shu-Yu Chen, and Shihong Xia

Conference:


Type:


Title:

    Efficient and Flexible Deformation Representation for Data-Driven Surface Modeling

Session/Category Title: DEFORMABLE SURFACE DESIGN

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Effectively characterizing the behavior of deformable objects has wide applicability but remains challenging. We present a new rotation-invariant deformation representation and a novel reconstruction algorithm to accurately reconstruct the positions and local rotations simultaneously. Meshes can be very efficiently reconstructed from our representation by matrix pre-decomposition, while, at the same time, hard or soft constraints can be flexibly specified with only positions of handles needed. Our approach is thus particularly suitable for constrained deformations guided by examples, providing significant benefits over state-of-the-art methods. Based on this, we further propose novel data-driven approaches to mesh deformation and non-rigid registration of deformable objects. Both problems are formulated consistently as finding an optimized model in the shape space that satisfies boundary constraints, either specified by the user, or according to the scan. By effectively exploiting the knowledge in the shape space, our method produces realistic deformation results in real-time and produces high quality registrations from a template model to a single noisy scan captured using a low-quality depth camera, outperforming state-of-the-art methods.

References:


    1. Marc Alexa. 2002. Linear combination of transformations. ACM Trans. Graph. 21, 3 (2002), 380–387. 
    2. Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape completion and animation of people. ACM Trans. Graph. 24, 3 (2005), 408–416. 
    3. O. K.-C. Au, C. L. Tai, L. Liu, and H. Fu. 2006. Dual Laplacian editing for meshes. IEEE Trans. Vis. Comp. Graph. 12, 3 (2006), 386–395. 
    4. Ilya Baran, Daniel Vlasic, Eitan Grinspun, and Jovan Popović. 2009. Semantic deformation transfer. ACM Trans. Graph. 28, 3 (2009), 36:1–36:6. 
    5. P. J. Besl and Neil D. McKay. 1992. A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2 (1992), 239–256. 
    6. Charles Bloom, Jonathan Blow, and Casey Muratori. 2004. Errors and Omissions in Marc Alexa’s “Linear Combination of Transformations.” http://www.cbloom.com/3d/techdocs/lcot_errors.pdf. (2004).
    7. Mario Botsch and Olga Sorkine. 2008. On linear variational surface deformation methods. IEEE Trans. Vis. Comp. Graph. 14, 1 (2008), 213–230. 
    8. Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective dynamics: Fusing constraint projections for fast simulation. ACM Trans. Graph. 33, 4, Article 154 (2014), 11 pages. 
    9. Sofien Bouaziz and Mark Pauly. 2013. Dynamic 2d/3d registration for the kinect. In ACM SIGGRAPH 2013 Courses. 21:1–21:14. 
    10. Isaac Chao, Ulrich Pinkall, Patrick Sanan, and Peter Schröder. 2010. A simple geometric model for elastic deformations. ACM Trans. Graph. 29, 4 (2010), 38:1–38:6. 
    11. Wei-Wen Feng, Byung-Uck Kim, and Yizhou Yu. 2008. Real-time data driven deformation using kernel canonical correlation analysis. ACM Trans. Graph. 27, 3 (2008), 91:1–91:9. 
    12. Oren Freifeld and Michael J. Black. 2012. Lie bodies: A manifold representation of 3d human shape. In ECCV (1). 1–14. 
    13. Stefan Fröhlich and Mario Botsch. 2011. Example-driven deformations based on discrete shells. Comput. Graph. Forum 30, 8 (2011), 2246–2257.
    14. James Gain and Dominique Bechmann. 2008. A survey of spatial deformation from a user-centered perspective. ACM Trans. Graph. 27, 4 (2008), 107:1–107:21. 
    15. Lin Gao, Yu-Kun Lai, Qi-Xing Huang, and Shi-Min Hu. 2013. A data-driven approach to realistic shape morphing. Comput. Graph. Forum 32, 2 (2013), 449–457.
    16. Michael Garland and Paul S. Heckbert. 1997. Surface simplification using quadric error metrics. In Proc. ACM SIGGRAPH. 209–216. 
    17. Nils Hasler, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and Hans-Peter Seidel. 2009. A statistical model of human pose and body shape. Comput. Graph. Forum 28, 2 (2009), 337–346.
    18. Roger A. Horn and Charles R. Johnson (Eds.). 1986. Matrix Analysis. Cambridge University Press, New York, NY. 
    19. Jin Huang, Xiaohan Shi, Xinguo Liu, Kun Zhou, Li-Yi Wei, Shang-Hua Teng, Hujun Bao, Baining Guo, and Heung-Yeung Shum. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3 (2006), 1126–1134. 
    20. Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-rigid-as-possible shape manipulation. In Proc. ACM SIGGRAPH. 1134–1141. 
    21. Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard A. Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew J. Davison, and Andrew W. Fitzgibbon. 2011. KinectFusion: Real-time 3d reconstruction and interaction using a moving depth camera. In Proc. ACM Symposium on User Interface Software and Technology. 559–568. 
    22. Ben Jones, Jovan Popovic, James McCann, Wilmot Li, and Adam Bargteil. 2015. Dynamic sprites: Artistic authoring of interactive animations. Comput. Anim. Virt. Worlds 26 (2015), 97–108. 
    23. Scott Kircher and Michael Garland. 2008. Free-form motion processing. ACM Trans. Graph. 27, 2 (2008), 12:1–12:13. 
    24. Yuki Koyama, Kenshi Takayama, Nobuyuki Umetani, and Takeo Igarashi. 2012. Real-time example-based elastic deformation. In Symposium on Computer Animation. 19–24. 
    25. Zohar Levi and Craig Gotsman. 2015. Smooth rotation enhanced as-rigid-as-possible mesh animation. IEEE Trans. Vis. Comp. Graph. 21, 2 (2015), 264–277.
    26. Hao Li, Bart Adams, Leonidas J. Guibas, and Mark Pauly. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28, 5 (2009), 175:1–175:10. 
    27. Hao Li, Robert W. Sumner, and Mark Pauly. 2008. Global correspondence optimization for non-rigid registration of depth scans. Comput. Graph. Forum 27, 5 (2008), 1421–1430. 
    28. Hao Li, Etienne Vouga, Anton Gudym, Linjie Luo, Jonathan T. Barron, and Gleb Gusev. 2013. 3D self-portraits. ACM Trans. Graph. 32, 6 (2013), 187:1–187:9. 
    29. Yaron Lipman, Olga Sorkine, David Levin, and Daniel Cohen-Or. 2005. Linear rotation-invariant coordinates for meshes. ACM Trans. Graph. 24, 3 (2005), 479–487. 
    30. Matthew Loper, Naureen Mahmood, and Michael J. Black. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph. 33, 6 (2014), 220:1–220:13. 
    31. Sebastian Martin, Bernhard Thomaszewski, Eitan Grinspun, and Markus H. Gross. 2011. Example-based elastic materials. ACM Trans. Graph. 30, 4 (2011), 72:1–72:8. 
    32. Richard M. Murray, S. Shankar Sastry, and Li Zexiang. 1994. A Mathematical Introduction to Robotic Manipulation (1st ed.). CRC Press, Boca Raton, FL. 
    33. Helmut Pottmann, Qi-Xing Huang, Yong-Liang Yang, and Shi-Min Hu. 2006. Geometry and convergence analysis of algorithms for registration of 3d shapes. Int. J. Comput. Vision 67, 3 (2006), 277–296. 
    34. Raif M. Rustamov, Maks Ovsjanikov, Omri Azencot, Mirela Ben-Chen, Fr’ed’eric Chazal, and Leonidas Guibas. 2013. Map-based exploration of intrinsic shape differences and variability. ACM Trans. Graph. 32, 4 (2013), 72:1–72:12. 
    35. D. C. Schneider and P. Eisert. 2009. Fast nonrigid mesh registration with a data-driven deformation prior. In Proc. ICCV Workshops. 304–311.
    36. Christian Schumacher, Bernhard Thomaszewski, Stelian Coros, Sebastian Martin, Robert W. Sumner, and Markus H. Gross. 2012. Efficient simulation of example-based materials. In Symposium on Computer Animation. 1–8. 
    37. Xiaohan Shi, Kun Zhou, Yiying Tong, Mathieu Desbrun, Hujun Bao, and Baining Guo. 2008. Example-based dynamic skinning in real time. ACM Trans. Graph. 27, 3 (2008), 29:1–29:8. 
    38. Olga Sorkine and Marc Alexa. 2007. As-rigid-as-possible surface modeling. In Proc. Symposium on Geometry Processing. 109–116. 
    39. O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel. 2004. Laplacian surface editing. In Proc. Symposium on Geometry Processing. 175–184. 
    40. Robert W. Sumner and Jovan Popović. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3 (2004), 399–405. 
    41. Robert W. Sumner, Johannes Schmid, and Mark Pauly. 2007. Embedded deformation for shape manipulation. ACM Trans. Graph. 26, 3 (2007), 80:1–80:7. 
    42. Robert W. Sumner, Matthias Zwicker, Craig Gotsman, and Jovan Popović. 2005. Mesh-based inverse kinematics. ACM Trans. Graph. 24, 3 (2005), 488–495. 
    43. G. K. L. Tam, Zhi-Quan Cheng, Yu-Kun Lai, F. C. Langbein, Yonghuai Liu, D. Marshall, R. R. Martin, Xian-Fang Sun, and P. L. Rosin. 2013. Registration of 3d point clouds and meshes: A survey from rigid to nonrigid. IEEE Trans. Vis. Comp. Graph. 19, 7 (2013), 1199–1217. 
    44. Demetri Terzopoulos, John Platt, Alan Barr, and Kurt Fleischer. 1987. Elastically deformable models. In Proc. ACM SIGGRAPH. 205–214. 
    45. Daniel Vlasic, Ilya Baran, Wojciech Matusik, and Jovan Popović. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3 (2008), 97:1–9. 
    46. Christoph Von-Tycowicz, Christian Schulz, Hans-Peter Seidel, and Klaus Hildebrandt. 2015. Real-time nonlinear shape interpolation. ACM Trans. Graph. 34, 3 (2015), 34:1–34:10. 
    47. Xiaolin Wei, Peizhao Zhang, and Jinxiang Chai. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Trans. Graph. 31, 6 (2012), 188:1–188:12. 
    48. Thibaut Weise, Sofien Bouaziz, Hao Li, and Mark Pauly. 2011. Realtime performance-based facial animation. ACM Trans. Graph. 30, 4 (2011), 77:1–77:10. 
    49. Tim Winkler, Jens Drieseberg, Marc Alexa, and Kai Hormann. 2010. Multi-scale geometry interpolation. Comput. Graph. Forum 29, 2 (2010), 309–318.
    50. Yizhou Yu, Kun Zhou, Dong Xu, Xiaohan Shi, Hujun Bao, Baining Guo, and Heung-Yeung Shum. 2004. Mesh editing with poisson-based gradient field manipulation. ACM Trans. Graph. 23, 3 (2004), 644–651. 
    51. Li Zhang, Noah Snavely, Brian Curless, and Steven M. Seitz. 2004. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. 23, 3 (2004), 548–558. 
    52. Qing Zhang, Bo Fu, Mao Ye, and Ruigang Yang. 2014. Quality dynamic human body modeling using a single low-cost depth camera. In CVPR. 676–683. 
    53. Wenjing Zhang, Jianmin Zheng, and Nadia Magnenat Thalmann. 2015. Real-time subspace integration for example-based elastic material. Comput. Graph. Forum 34, 2 (2015), 395–404. 
    54. Kun Zhou, Jin Huang, John Snyder, Xinguo Liu, Hujun Bao, Baining Guo, and Heung-Yeung Shum. 2005. Large mesh deformation using the volumetric graph Laplacian. ACM Trans. Graph. 24, 3 (2005), 496–503. 
    55. Michael Zollhöfer, Matthias Niessner, Shahram Izadi, Christoph Rehmann, Christopher Zach, Matthew Fisher, Chenglei Wu, Andrew Fitzgibbon, Charles Loop, Christian Theobalt, and Marc Stamminger. 2014. Real-time non-rigid reconstruction using an RGB-D camera. ACM Trans. Graph. 33, 4 (2014), 156:1–156:12. 

ACM Digital Library Publication:


Overview Page: