“Autocomplete 3D sculpting” by Peng, Xing and Wei

  • ©Mengqi Peng, Jun Xing, and Li-Yi Wei



Entry Number: 132

Session Title:



    Autocomplete 3D sculpting




    Digital sculpting is a popular means to create 3D models but remains a challenging task. We propose a 3D sculpting system that assists users, especially novices, in freely creating models with reduced input labor and enhanced output quality. With an interactive sculpting interface, our system silently records and analyzes users’ workflows including brush strokes and camera movements, and predicts what they might do in the future. Users can accept, partially accept, or ignore the suggestions and thus retain full control and individual style. They can also explicitly select and clone past workflows over output model regions. Our key idea is to consider how a model is authored via dynamic workflows in addition to what is shaped in static geometry. This allows our method for more accurate analysis of user intentions and more general synthesis of shape structures than prior workflow or geometry methods, such as large overlapping deformations. We evaluate our method via user feedbacks and authored models.


    1. Autodesk. 2014. Mudbox. (2014). http://www.autodesk.com/products/mudbox/.Google Scholar
    2. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: As-natural-as-possible Sketching System for Creating 3D Curve Models. In UIST ’08. 151–160. Google ScholarDigital Library
    3. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2009. EverybodyLovesSketch: 3D Sketching for a Broader Audience. In UIST ’09. 59–68. Google ScholarDigital Library
    4. Blender Foundation. 2016. Blender. (2016).Google Scholar
    5. Martin Bokeloh, Michael Wand, Vladlen Koltun, and Hans-Peter Seidel. 2011. Pattern-aware Shape Deformation Using Sliding Dockers. ACM Trans. Graph. 30, 6, Article 123 (2011), 10 pages. Google ScholarDigital Library
    6. Claudio Calabrese, Marco Fratarcangeli, and Fabio Pellacini. 2017. sLayer: a System for Multi-Layered Material Sculpting.Google Scholar
    7. Claudio Calabrese, Gabriele Salvati, Marco Tarini, and Fabio Pellacini. 2016. cSculpt: A System for Collaborative Sculpting. ACM Trans. Graph. 35, 4, Article 91 (2016), 8 pages. Google ScholarDigital Library
    8. Siddhartha Chaudhuri and Vladlen Koltun. 2010. Data-driven Suggestions for Creativity Support in 3D Modeling. ACM Trans. Graph. 29, 6, Article 183 (2010), 10 pages. Google ScholarDigital Library
    9. Hsiang-Ting Chen, Tovi Grossman, Li-Yi Wei, Ryan M. Schmidt, Björn Hartmann, George Fitzmaurice, and Maneesh Agrawala. 2014. History Assisted View Authoring for 3D Models. In CHI ’14. 2027–2036. Google ScholarDigital Library
    10. Hsiang-Ting Chen, Li-Yi Wei, and Chun-Fa Chang. 2011. Nonlinear Revision Control for Images. ACM Trans. Graph. 30, 4, Article 105 (2011), 10 pages. Google ScholarDigital Library
    11. Hsiang-Ting Chen, Li-Yi Wei, Björn Hartmann, and Maneesh Agrawala. 2016. Data-driven Adaptive History for Image Editing. In I3D ’16. 103–111. Google ScholarDigital Library
    12. Daniel Cohen-Or and Hao Zhang. 2016. From Inspired Modeling to Creative Modeling. Vis. Comput. 32, 1 (2016), 7–14. Google ScholarDigital Library
    13. Fernando De Goes and Doug L.James. 2017. Regularized Kelvinlets: Sculpting Brushes Based on Fundamental Solutions of Elasticity. ACM Trans. Graph. 36, 4, Article 40 (2017), 11 pages. Google ScholarDigital Library
    14. Jonathan D. Denning, William B. Kerr, and Fabio Pellacini. 2011. MeshFlow: Interactive Visualization of Mesh Construction Sequences. ACM Trans. Graph. 30, 4, Article 66 (2011), 8 pages. Google ScholarDigital Library
    15. Jonathan D. Denning and Fabio Pellacini. 2013. MeshGit: Diffing and Merging Meshes for Polygonal Modeling. ACM Trans. Graph. 32, 4, Article 35 (2013), 10 pages. Google ScholarDigital Library
    16. Jonathan D. Denning, Valentina Tibaldo, and Fabio Pellacini. 2015. 3DFlow: Continuous Summarization of Mesh Editing Workflows. ACM Trans. Graph. 34, 4, Article 140 (2015), 9 pages. Google ScholarDigital Library
    17. David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley. 2002. Texturing and Modeling: A Procedural Approach (3rd ed.). Morgan Kaufmann Publishers Inc. Google ScholarDigital Library
    18. Arnaud Emilien, Ulysse Vimont, Marie-Paule Cani, Pierre Poulin, and Bedrich Benes. 2015. WorldBrush: Interactive Example-based Synthesis of Procedural Virtual Worlds. ACM Trans. Graph. 34, 4, Article 106 (2015), 11 pages. Google ScholarDigital Library
    19. Lubin Fan, Ruimin Wang, Linlin Xu, Jiansong Deng, and Ligang Liu. 2013. Modeling by Drawing with Shadow Guidance. Computer Graphics Forum (Proc. Pacific Graphics) 23, 7 (2013), 157–166.Google ScholarCross Ref
    20. Chi-Wing Fu, Jiazhi Xia, and Ying He. 2010. LayerPaint: A Multi-layer Interactive 3D Painting Interface. In CHI ’10. 811–820. Google ScholarDigital Library
    21. Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J. Mitra. 2011. Animated Construction of Line Drawings. ACM Trans. Graph. 30, 6, Article 133 (2011), 10 pages. Google ScholarDigital Library
    22. Thomas Funkhouser, Michael Kazhdan, Philip Shilane, Patrick Min, William Kiefer, Ayellet Tal, Szymon Rusinkiewicz, and David Dobkin. 2004. Modeling by Example. ACM Trans. Graph. 23, 3 (2004), 652–663. Google ScholarDigital Library
    23. Raul Fernandez Hernandez. 2011. Dynamic Subdivision Sculpting. (2011).Google Scholar
    24. Ruizhen Hu, Oliver van Kaick, Bojian Wu, Hui Huang, Ariel Shamir, and Hao Zhang. 2016. Learning How Objects Function via Co-analysis of Interactions. ACM Trans. Graph. 35, 4, Article 47 (2016), 13 pages. Google ScholarDigital Library
    25. Shi-Min Hu, Kun Xu, Li-Qian Ma, Bin Liu, Bi-Ye Jiang, and Jue Wang. 2013. Inverse Image Editing: Recovering a Semantic Editing History from a Before-and-after Image Pair. ACM Trans. Graph. 32, 6, Article 194 (2013), 11 pages. Google ScholarDigital Library
    26. Takeo Igarashi and John F. Hughes. 2001. A Suggestive Interface for 3D Drawing. In UIST ’01. 173–181. Google ScholarDigital Library
    27. Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: A Sketching Interface for 3D Freeform Design. In SIGGRAPH ’99. 409–416. Google ScholarDigital Library
    28. Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, Shengdong Zhao, and George Fitzmaurice. 2014. Draco: Bringing Life to Illustrations with Kinetic Textures. In CHI ’14. 351–360. Google ScholarDigital Library
    29. Rubaiat Habib Kazi, Takeo Igarashi, Shengdong Zhao, and Richard Davis. 2012. Vignette: Interactive Texture Design and Manipulation with Freeform Gestures for Pen-and-ink Illustration. In CHI ’12. 1727–1736. Google ScholarDigital Library
    30. Matt Kloskowski. 2010. Cloning With a Preview in Photoshop CS4. (2010). http://www.photoshop.com/tutorials/4305.Google Scholar
    31. David Kurlander and Eric A. Bier. 1988. Graphical Search and Replace. In SIGGRAPH ’88. ACM, 113–120. Google ScholarDigital Library
    32. David Kurlander and Steven Feiner. 1992a. A History-based Macro by Example System. In UIST ’92. ACM, 99–106. Google ScholarDigital Library
    33. David Kurlander and Steven Feiner. 1992b. Interactive Constraint-based Search and Replace. In CHI ’92. ACM, 609–618. Google ScholarDigital Library
    34. Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. 2005. Texture Optimization for Example-based Synthesis. ACM Trans. Graph. 24, 3 (2005), 795–802. Google ScholarDigital Library
    35. Jerry Liu, Fisher Yu, and Thomas A. Funkhouser. 2017. Interactive 3D Modeling with a Generative Adversarial Network. CoRR abs/1706.05170 (2017).Google Scholar
    36. Zhaoliang Lun, Evangelos Kalogerakis, Rui Wang, and Alla Sheffer. 2016. Functionality Preserving Shape Style Transfer. ACM Trans. Graph. 35, 6, Article 209 (2016), 14 pages. Google ScholarDigital Library
    37. Chongyang Ma, Li-Yi Wei, Sylvain Lefebvre, and Xin Tong. 2013. Dynamic Element Textures. ACM Trans. Graph. 32, 4, Article 90 (2013), 10 pages. Google ScholarDigital Library
    38. André Maximo, Robert Patro, Amitabh Varshney, and R Farias. 2011. A robust and rotationally invariant local surface descriptor with applications to non-local mesh processing. Graphical Models 73, 5 (2011), 231–242. Google ScholarDigital Library
    39. Niloy J. Mitra, Leonidas J. Guibas, and Mark Pauly. 2006. Partial and Approximate Symmetry Detection for 3D Geometry. ACM Trans. Graph. 25, 3 (2006), 560–568. Google ScholarDigital Library
    40. Niloy J. Mitra, Michael Wand, Hao Zhang, Daniel Cohen-Or, and Martin Bokeloh. 2013. Structure-Aware Shape Processing. In Eurographics 2013 – State of the Art Reports.Google Scholar
    41. Mathieu Nancel and Andy Cockburn. 2014. Causality: A Conceptual Model of Interaction History. In CHI ’14. 1777–1786. Google ScholarDigital Library
    42. Gen Nishida, Ignacio Garcia-Dorado, Daniel G. Aliaga, Bedrich Benes, and Adrien Bousseau. 2016. Interactive Sketching of Urban Procedural Models. ACM Trans. Graph. 35, 4, Article 130 (2016), 11 pages. Google ScholarDigital Library
    43. Michaël Ortega and Thomas Vincent. 2014. Direct Drawing on 3D Shapes with Automated Camera Control. In CHI ’14. 2047–2050. Google ScholarDigital Library
    44. Mengqi Peng, Jun Xing, and Li-Yi Wei. 2017. Autocomplete 3D Sculpting. CoRR abs/1703.10405 (2017). arXiv:cs.GR/1703.10405Google Scholar
    45. Patrick Pérez, Michel Gangnet, and Andrew Blake. 2003. Poisson Image Editing. ACM Trans. Graph. 22, 3 (2003), 313–318. Google ScholarDigital Library
    46. Pixologic. 2011. Sculptris. (2011).Google Scholar
    47. Pixologic. 2015. ZBrush. (2015).Google Scholar
    48. Gabriele Salvati, Christian Santoni, Valentina Tibaldo, and Fabio Pellacini. 2015. Mesh-Histo: Collaborative Modeling by Sharing and Retargeting Editing Histories. ACM Trans. Graph. 34, 6, Article 205 (2015), 10 pages. Google ScholarDigital Library
    49. Christian Santoni, Claudio Calabrese, Francesco Di Renzo, and Fabio Pellacini. 2016. SculptStat: Statistical Analysis of Digital Sculpting Workflows. CoRR abs/1601.07765 (2016).Google Scholar
    50. Ryan Schmidt. 2013. Stroke Parameterization. Computer Graphics Forum 32, 2pt2 (2013), 255–263.Google Scholar
    51. Ryan Schmidt, Cindy Grimm, and Brian Wyvill. 2006. Interactive Decal Compositing with Discrete Exponential Maps. ACM Trans. Graph. 25, 3 (2006), 605–613. Google ScholarDigital Library
    52. Ryan Schmidt and Karan Singh. 2010. Meshmixer: An Interface for Rapid Mesh Composition. In SIGGRAPH ’10 Talks. Article 6, 1 pages. Google ScholarDigital Library
    53. Adriana Schulz, Ariel Shamir, David I. W. Levin, Pitchaya Sitthi-amorn, and Wojciech Matusik. 2014. Design and Fabrication by Example. ACM Trans. Graph. 33, 4, Article 62 (2014), 11 pages. Google ScholarDigital Library
    54. Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (2007), 20–32. Google ScholarDigital Library
    55. Qian Sun, Long Zhang, Minqi Zhang, Xiang Ying, Shi-Qing Xin, Jiazhi Xia, and Ying He. 2013. Texture Brush: An Interactive Surface Texturing Interface. In I3D ’13. 153–160. Google ScholarDigital Library
    56. Ryo Suzuki, Tom Yeh, Koji Yatani, and Mark D. Gross. 2017. Autocomplete Textures for 3D Printing. ArXiv e-prints (2017). arXiv:csHC/1703.05700Google Scholar
    57. Kenshi Takayama, Ryan Schmidt, Karan Singh, Takeo Igarashi, Tamy Boubekeur, and Olga Sorkine. 2011. Geobrush: Interactive mesh geometry cloning. In Computer Graphics Forum, Vol. 30. 613–622.Google ScholarCross Ref
    58. Jianchao Tan, Marek Dvorožňák, Daniel Sýkora, and Yotam Gingold. 2015. Decomposing Time-lapse Paintings into Layers. ACM Trans. Graph. 34, 4, Article 61 (2015), 10 pages. Google ScholarDigital Library
    59. Steve Tsang, Ravin Balakrishnan, Karan Singh, and Abhishek Ranjan. 2004. A Suggestive Interface for Image Guided 3D Sketching. In CHI ’04. 591–598. Google ScholarDigital Library
    60. Sidney W. Wang and Arie E. Kaufman. 1995. Volume Sculpting. In I3D ’95. 151–ff. Google ScholarDigital Library
    61. Jun Xing, Hsiang-Ting Chen, and Li-Yi Wei. 2014. Autocomplete Painting Repetitions. ACM Trans. Graph. 33, 6, Article 172 (2014), 11 pages. Google ScholarDigital Library
    62. Jun Xing, Li-Yi Wei, Takaaki Shiratori, and Koji Yatani. 2015. Autocomplete Hand-drawn Animations. ACM Trans. Graph. 34, 6, Article 169 (2015), 11 pages. Google ScholarDigital Library
    63. Ya-Ting Yue, Xiaolong Zhang, Yongliang Yang, Gang Ren, Yi-King Choi, and Wenping Wang. 2017. WireDraw: 3D Wire Sculpturing Guided with Mixed Reality. In CHI ’17. 3693–3704. Google ScholarDigital Library
    64. Steve Zelinka and Michael Garland. 2004. Similarity-based Surface Modelling Using Geodesic Fans. In SGP ’04. 204–213. Google ScholarDigital Library

ACM Digital Library Publication: