“DAG amendment for inverse control of parametric shapes” by Michel and Boubekeur

Parametric shapes model objects as programs producing a geometry based on a few semantic degrees of freedom, called hyper-parameters. These shapes are the typical output of non-destructive modeling, CAD modeling or rigging. However they suffer from the core issue of being manipulated only indirectly, through a series of values rather than the geometry itself. In this paper, we introduce an amendment process of the underlying direct acyclic graph (DAG) of a parametric shape. This amendment enables a local differentiation of the shape w.r.t. its hyper-parameters that we leverage to provide interactive direct manipulation of the output. By acting on the shape synthesis process itself, our method is agnostic to the variations of the connectivity and topology that may occur in its output while changing the input hyper-parameters. Furthermore, our method is oblivious to the internal logic of the DAG nodes. We illustrate our approach on a collection of examples combining the typical nodes found in modern parametric modeling packages – such as deformation, booleans and surfacing operators – for which our method provides the user with inverse control over the hyper-parameters through a brush stroke metaphor.

1. Rinat Abdrashitov, Fanny Chevalier, and Karan Singh. 2020. Interactive Exploration and Refinement of Facial Expression Using Manifold Learning. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST ’20). Association for Computing Machinery, New York, NY, USA, 778–790. Google ScholarDigital Library
2. Dicko Ali-Hamadi, Tiantian Liu, Benjamin Gilles, Ladislav Kavan, François Faure, Olivier Palombi, and Marie-Paule Cani. 2013. Anatomy Transfer. ACM Trans. Graph. 32, 6, Article 188 (Nov. 2013). Google ScholarDigital Library
3. Daniel G. Aliaga, İlke Demir, Bedrich Benes, and Michael Wand. 2016. Inverse Procedural Modeling of 3D Models for Virtual Worlds. In ACM SIGGRAPH 2016 Courses (SIGGRAPH ’16). Association for Computing Machinery, New York, NY, USA, 1–316. Google ScholarDigital Library
4. Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape Completion and Animation of People. In ACM SIGGRAPH 2005 Papers (SIGGRAPH ’05). Association for Computing Machinery, New York, NY, USA, 408–416. Google ScholarDigital Library
5. A. Aristidou, J. Lasenby, Y. Chrysanthou, and A. Shamir. 2018. Inverse Kinematics Techniques in Computer Graphics: A Survey. Computer Graphics Forum 37, 6 (2018), 35–58. Google ScholarCross Ref
6. Grégoire Aujay, Franck Hétroy, Francis Lazarus, and Christine Depraz. 2007. Harmonic Skeleton for Realistic Character Animation. In SCA ’07 – ACM-SIGGRAPH/Eurographics Symposium on Computer Animation, Michael Gleicher and Daniel Thalmann (Eds.). Eurographics Association, San Diego, United States, 151–160. Google ScholarCross Ref
7. Quentin Avril, Donya Ghafourzadeh, Srinivasan Ramachandran, Sahel Fallahdoust, Sarah Ribet, Olivier Dionne, Martin de Lasa, and Eric Paquette. 2016. Animation Setup Transfer for 3D Characters. Computer Graphics Forum 35, 2 (2016), 115–126. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12816 Google ScholarCross Ref
8. P. Baerlocher and R. Boulic. 1998. Task-Priority Formulations for the Kinematic Control of Highly Redundant Articulated Structures. In Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190), Vol. 1. 323–329 vol.1. Google ScholarCross Ref
9. Paolo Baerlocher and Ronan Boulic. 2004. An Inverse Kinematics Architecture Enforcing an Arbitrary Number of Strict Priority Levels. The visual computer 20, 6 (2004), 402–417.Google Scholar
10. Ilya Baran and Jovan Popović. 2007. Automatic Rigging and Animation of 3D Characters. In ACM SIGGRAPH 2007 Papers (SIGGRAPH ’07). ACM, New York, NY, USA, Article 72. Google ScholarDigital Library
11. Santiago Barroso, Gonzalo Besuievsky, and Gustavo Patow. 2013. Visual Copy & Paste for Procedurally Modeled Buildings by Ruleset Rewriting. Computers & Graphics 37, 4 (2013), 238–246. Google ScholarDigital Library
12. Volker Blanz and Thomas Vetter. 1999. A Morphable Model for the Synthesis of 3D Faces. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’99). ACM Press/Addison-Wesley Publishing Co., USA, 187–194. Google ScholarDigital Library
13. Martin Bokeloh, Michael Wand, Hans-Peter Seidel, and Vladlen Koltun. 2012. An Algebraic Model for Parameterized Shape Editing. ACM Transactions on Graphics 31, 4 (July 2012), 78:1–78:10. Google ScholarDigital Library
14. Mario Botsch, Mark Pauly, Markus H Gross, and Leif Kobbelt. 2006. PriMo: Coupled Prisms for Intuitive Surface Modeling. In Symposium on Geometry Processing. 11–20.Google Scholar
15. M. Botsch and O. Sorkine. 2008. On Linear Variational Surface Deformation Methods. IEEE Transactions on Visualization and Computer Graphics 14, 1 (Jan. 2008), 213–230. Google ScholarDigital Library
16. Steve Capell, Matthew Burkhart, Brian Curless, Tom Duchamp, and Zoran Popović. 2005. Physically Based Rigging for Deformable Characters. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ’05). Association for Computing Machinery, New York, NY, USA, 301–310. Google ScholarDigital Library
17. Chia-Hsing Chiu, Yuki Koyama, Yu-Chi Lai, Takeo Igarashi, and Yonghao Yue. 2020. Human-in-the-Loop Differential Subspace Search in High-Dimensional Latent Space. ACM Trans. Graph. 39, 4, Article 85 (July 2020). Google ScholarDigital Library
18. Paul D. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A Hybrid Geometry-and Image-Based Approach. In Proceedings of the 23th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’96).Google ScholarDigital Library
19. A. S. Deo and I. D. Walker. 1992. Robot Subtask Performance with Singularity Robustness Using Optimal Damped Least-Squares. In Proceedings 1992 IEEE International Conference on Robotics and Automation. 434–441 vol.1. Google ScholarCross Ref
20. Matheus Gadelha, Giorgio Gori, Duygu Ceylan, Radomir Mech, Nathan Carr, Tamy Boubekeur, Rui Wang, and Subhransu Maji. 2020. Learning Generative Models of Shape Handles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 402–411.Google ScholarCross Ref
21. Ran Gal, Olga Sorkine, Niloy J. Mitra, and Daniel Cohen-Or. 2009. IWIRES: An Analyze-and-Edit Approach to Shape Manipulation. In ACM SIGGRAPH 2009 Papers (SIGGRAPH ’09). Association for Computing Machinery, New York, NY, USA, Article 33. Google ScholarDigital Library
22. Yaroslav Ganin, Tejas Kulkarni, Igor Babuschkin, S. M. Ali Eslami, and Oriol Vinyals. 2018. Synthesizing Programs for Images Using Reinforced Adversarial Learning. (2018). arXiv:1804.01118 [cs.CV]Google Scholar
23. Fabian Hahn, Sebastian Martin, Bernhard Thomaszewski, Robert Sumner, Stelian Coros, and Markus Gross. 2012. Rig-Space Physics. ACM Trans. Graph. 31, 4, Article 72 (July 2012). Google ScholarDigital Library
24. M. Hecher, P. Guerrero, P. Wonka, and M. Wimmer. 2018. How Do Users Map Points Between Dissimilar Shapes? IEEE Transactions on Visualization and Computer Graphics 24, 8 (Aug. 2018), 2327–2338. Google ScholarCross Ref
25. Masaki Hilaga, Yoshihisa Shinagawa, Taku Kohmura, and Tosiyasu L. Kunii. 2001. Topology Matching for Fully Automatic Similarity Estimation of 3D Shapes. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’01). Association for Computing Machinery, New York, NY, USA, 203–212. Google ScholarDigital Library
26. Daniel Holden, Jun Saito, and Taku Komura. 2015. Learning an Inverse Rig Mapping for Character Animation. In Proceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation. ACM, Los Angeles California, 165–173. Google ScholarDigital Library
27. Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-Rigid-as-Possible Shape Manipulation. ACM Trans. Graph. 24, 3 (July 2005), 1134–1141. Google ScholarDigital Library
28. Alec Jacobson, Ilya Baran, Jovan Popovic, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Trans. Graph. 30, 4 (2011), 78.Google ScholarDigital Library
29. R. Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy J. Mitra, and Daniel Ritchie. 2020. ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis. ACM Trans. Graph. 39, 6, Article 234 (Nov. 2020). Google ScholarDigital Library
30. Javor Kalojanov, Michael Wand, and Philipp Slusallek. 2016. Building Construction Sets by Tiling Grammar Simplification. Computer Graphics Forum 35, 2 (2016), 13–25. Google ScholarCross Ref
31. T. Kelly, P. Wonka, and P. Mueller. 2015. Interactive Dimensioning of Parametric Models. Computer Graphics Forum 34, 2 (May 2015), 117–129. Google ScholarDigital Library
32. Martin Kilian, Niloy J. Mitra, and Helmut Pottmann. 2007. Geometric Modeling in Shape Space. In ACM SIGGRAPH 2007 Papers (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 64–es. Google ScholarDigital Library
33. Vladislav Kraevoy and Alla Sheffer. 2004. Cross-Parameterization and Compatible Remeshing of 3D Models. ACM Trans. Graph. 23, 3 (Aug. 2004), 861–869. Google ScholarDigital Library
34. V. Krs, R. Mech, M. Gaillard, N. Carr, and B. Benes. 2020. PICO: Procedural Iterative Constrained Optimizer for Geometric Modeling. IEEE Transactions on Visualization and Computer Graphics (2020), 1–1. Google ScholarDigital Library
35. C. Kurz, X. Wu, M. Wand, T. Thormählen, P. Kohli, and H.-P. Seidel. 2014. Symmetry-Aware Template Deformation and Fitting. Computer Graphics Forum 33, 6 (2014), 205–219. Google ScholarDigital Library
36. Francis Lazarus and Anne Verroust. 1999. Level Set Diagrams of Polyhedral Objects. In SMA’99 Proceedings of the Fifth ACM Symposium on Solid Modeling and Applications. ACM, Ann Arbor, United States. Google ScholarDigital Library
37. Kurt Leimer, Lukas Gersthofer, Michael Wimmer, and Przemyslaw Musialski. 2017. Relation-Based Parametrization and Exploration of Shape Collections. Computers & Graphics 67 (Oct. 2017), 127–137. Google ScholarDigital Library
38. Zohar Levi and Craig Gotsman. 2015. Smooth Rotation Enhanced As-Rigid-as-Possible Mesh Animation. IEEE Transactions on Visualization and Computer Graphics 21, 2 (Feb. 2015), 264–277. Google ScholarDigital Library
39. J. P. Lewis and K. Anjyo. 2010. Direct Manipulation Blendshapes. IEEE Computer Graphics and Applications 30, 4 (July 2010), 42–50. Google ScholarDigital Library
40. Hao Li, Thibaut Weise, and Mark Pauly. 2010. Example-Based Facial Rigging. ACM Trans. Graph. 29, 4, Article 32 (July 2010). Google ScholarDigital Library
41. Tianye Li, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. 2017. Learning a Model of Facial Shape and Expression from 4D Scans. ACM Trans. Graph. 36, 6, Article 194 (Nov. 2017). Google ScholarDigital Library
42. Stefan Lienhard, Cheryl Lau, Pascal Müller, Peter Wonka, and Mark Pauly. 2017. Design Transformations for Rule-Based Procedural Modeling. Computer Graphics Forum 36, 2 (May 2017), 39–48. Google ScholarDigital Library
43. M. Lipp, M. Specht, C. Lau, P. Wonka, and P. Müller. 2019. Local Editing of Procedural Models. Computer Graphics Forum 38, 2 (2019), 13–25. Google ScholarCross Ref
44. Han Liu, Ulysse Vimont, Michael Wand, Marie-Paule Cani, Stefanie Hahmann, Damien Rohmer, and Niloy J. Mitra. 2015. Replaceable Substructures for Efficient Part-Based Modeling. Computer Graphics Forum 34, 2 (2015), 503–513. Google ScholarDigital Library
45. Lijuan Liu, Youyi Zheng, Di Tang, Yi Yuan, Changjie Fan, and Kun Zhou. 2019. NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks. ACM Trans. Graph. 38, 4, Article 114 (July 2019). Google ScholarDigital Library
46. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015). Google ScholarDigital Library
47. Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. 2019. AMASS: Archive of Motion Capture as Surface Shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).Google Scholar
48. Aman Mathur, Marcus Pirron, and Damien Zufferey. 2020. Interactive Programming for Parametric CAD. Computer Graphics Forum 39, 6 (Sept. 2020), 408–425. Google ScholarCross Ref
49. Bruce Merry, Patrick Marais, and James Gain. 2006. Animation Space: A Truly Linear Framework for Character Animation. ACM Trans. Graph. 25, 4 (Oct. 2006), 1400–1423. Google ScholarDigital Library
50. Christian Miller, Okan Arikan, and Don Fussell. 2010. Frankenrigs: Building Character Rigs from Multiple Sources (I3D ’10). Association for Computing Machinery, New York, NY, USA, 31–38. Google ScholarDigital Library
51. Niloy J. Mitra, Mark Pauly, Michael Wand, and Duygu Ceylan. 2012. Symmetry in 3D Geometry: Extraction and Applications. In EUROGRAPHICS State-of-the-Art Report. Google ScholarDigital Library
52. Niloy J. Mitra, Michael Wand, Hao Zhang, Daniel Cohen-Or, Vladimir Kim, and Qi-Xing Huang. 2014. Structure-Aware Shape Processing. In ACM SIGGRAPH 2014 Courses (SIGGRAPH ’14). Association for Computing Machinery, New York, NY, USA, 1–21. Google ScholarDigital Library
53. Ahmed A A Osman, Timo Bolkart, and Michael J. Black. 2020. STAR: A Spare Trained Articulated Human Body Regressor. In European Conference on Computer Vision (ECCV).Google Scholar
54. Valerio Pascucci, Giorgio Scorzelli, Peer-Timo Bremer, and Ajith Mascarenhas. 2007. Robust On-Line Computation of Reeb Graphs: Simplicity and Speed. In ACM SIGGRAPH 2007 Papers (SIGGRAPH ’07). Association for Computing Machinery, New York, NY, USA, 58–es. Google ScholarDigital Library
55. G. Patow. 2012. User-Friendly Graph Editing for Procedural Modeling of Buildings. IEEE Computer Graphics and Applications 32, 2 (March 2012), 66–75. Google ScholarDigital Library
56. D. Raunhardt and R. Boulic. 2007. Progressive Clamping. In Proceedings 2007 IEEE International Conference on Robotics and Automation. 4414–4419. Google ScholarCross Ref
57. Nadine Abu Rumman and Marco Fratarcangeli. 2016. State of the Art in Skinning Techniques for Articulated Deformable Characters. In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications: Volume 1: GRAPP (GRAPP 2016). SCITEPRESS – Science and Technology Publications, Lda, Setubal, PRT, 200–212. Google ScholarDigital Library
58. L. Saab, O. E. Ramos, F. Keith, N. Mansard, P. Souères, and J. Fourquet. 2013. Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints. IEEE Transactions on Robotics 29, 2 (April 2013), 346–362. Google ScholarDigital Library
59. John Schreiner, Arul Asirvatham, Emil Praun, and Hugues Hoppe. 2004. Inter-Surface Mapping. In ACM SIGGRAPH 2004 Papers (SIGGRAPH ’04). Association for Computing Machinery, New York, NY, USA, 870–877. Google ScholarDigital Library
60. Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. 2018. CSGNet: Neural Shape Parser for Constructive Solid Geometry. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
61. Olga Sorkine and Marc Alexa. 2007. As-Rigid-as-Possible Surface Modeling. In Proceedings of the Fifth Eurographics Symposium on Geometry Processing (SGP ’07). Eurographics Association, Goslar, DEU, 109–116.Google ScholarDigital Library
62. Robert W. Sumner, Matthias Zwicker, Craig Gotsman, and Jovan Popović. 2005. Mesh-Based Inverse Kinematics. ACM Transactions on Graphics 24, 3 (July 2005), 488–495. Google ScholarDigital Library
63. Jerry O. Talton, Daniel Gibson, Lingfeng Yang, Pat Hanrahan, and Vladlen Koltun. 2009. Exploratory Modeling with Collaborative Design Spaces. 28, 5 (Dec. 2009), 1–10. Google ScholarDigital Library
64. Jerry O. Talton, Yu Lou, Steve Lesser, Jared Duke, Radomír Měch, and Vladlen Koltun. 2011. Metropolis Procedural Modeling. ACM Trans. Graph. 30, 2, Article 11 (April 2011), 11:1–11:14 pages. Google ScholarDigital Library
65. Julien Tierny, Jean-Philippe Vandeborre, and Mohamed Daoudi. 2006. 3D Mesh Skeleton Extraction Using Topological and Geometrical Analyses. In 14th Pacific Conference on Computer Graphics and Applications (Pacific Graphics 2006). Tapei, Taiwan, s1poster.Google Scholar
66. Yea-Fu Tsao and King-Sun Fu. 1984. Stochastic Skeleton Modeling of Objects. Computer Vision, Graphics, and Image Processing 25, 3 (1984), 348–370. Google ScholarCross Ref
67. Nobuyuki Umetani. 2017. Exploring Generative 3D Shapes Using Autoencoder Networks. In SIGGRAPH Asia 2017 Technical Briefs (SA ’17). Association for Computing Machinery, New York, NY, USA, Article 24. Google ScholarDigital Library
68. Oliver van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. 2011. A Survey on Shape Correspondence. Computer Graphics Forum 30, 6 (2011), 1681–1707. Google ScholarCross Ref
69. Noranart Vesdapunt, Mitch Rundle, HsiangTao Wu, and Baoyuan Wang. 2020. JNR: Joint-Based Neural Rig Representation for Compact 3D Face Modeling. (2020). arXiv:2007.06755 [cs.CV]Google Scholar
70. L. Wade and R. E. Parent. 2000. Fast, Fully-Automated Generation of Control Skeletons for Use in Animation. In Computer Animation. IEEE Computer Society, Los Alamitos, CA, USA, 164. Google ScholarCross Ref
71. Kevin Wampler. 2016. Fast and Reliable Example-Based Mesh IK for Stylized Deformations. ACM Trans. Graph. 35, 6, Article 235 (Nov. 2016). Google ScholarDigital Library
72. Emily Whiting, John Ochsendorf, and Frédo Durand. 2009. Procedural Modeling of Structurally-Sound Masonry Buildings. (2009).Google Scholar
73. Xiaokun Wu, Michael Wand, Klaus Hildebrandt, Pushmeet Kohli, and Hans-Peter Seidel. 2014. Real-Time Symmetry-Preserving Deformation. Computer Graphics Forum 33, 7 (2014), 229–238. Google ScholarDigital Library
74. Z. Xu, Y. Zhou, E. Kalogerakis, and K. Singh. 2019. Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets. In 2019 International Conference on 3D Vision (3DV). 298–307. Google ScholarCross Ref
75. Yong-Liang Yang, Yi-Jun Yang, Helmut Pottmann, and Niloy J. Mitra. 2011. Shape Space Exploration of Constrained Meshes. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA ’11). Association for Computing Machinery, New York, NY, USA, Article 124. Google ScholarDigital Library
76. Jianfeng Zhang, Xuecheng Nie, and Jiashi Feng. 2020. Inference Stage Optimization for Cross-Scenario 3d Human Pose Estimation. Advances in Neural Information Processing Systems 33 (2020).Google Scholar
77. Xin Zhao, Cheng-Cheng Tang, Yong-Liang Yang, Helmut Pottmann, and Niloy J. Mitra. 2013. Intuitive Design Exploration of Constrained Meshes. In Advances in Architectural Geometry 2012, Lars Hesselgren, Shrikant Sharma, Johannes Wallner, Niccolo Baldassini, Philippe Bompas, and Jacques Raynaud (Eds.). Springer Vienna, Vienna, 305–318.Google Scholar