“Interactive liquid splash modeling by user sketches” by Yan, Chen, Yang and Wang
Conference:
Type(s):
Title:
- Interactive liquid splash modeling by user sketches
Session/Category Title: All About Sketches
Presenter(s)/Author(s):
Abstract:
Splashing is one of the most fascinating liquid phenomena in the real world and it is favored by artists to create stunning visual effects, both statically and dynamically. Unfortunately, the generation of complex and specialized liquid splashes is a challenging task and often requires considerable time and effort. In this paper, we present a novel system that synthesizes realistic liquid splashes from simple user sketch input. Our system adopts a conditional generative adversarial network (cGAN) trained with physics-based simulation data to produce raw liquid splash models from input sketches, and then applies model refinement processes to further improve their small-scale details. The system considers not only the trajectory of every user stroke, but also its speed, which makes the splash model simulation-ready with its underlying 3D flow. Compared with simulation-based modeling techniques through trials and errors, our system offers flexibility, convenience and intuition in liquid splash design and editing. We evaluate the usability and the efficiency of our system in an immersive virtual reality environment. Thanks to this system, an amateur user can now generate a variety of realistic liquid splashes in just a few minutes.
References:
1. Jeremiah Uhler Brackbill, Douglas B. Kothe, and Hans Max Ruppel. 1988. Flip: A Low-Dissipation, Particle-in-Cell Method for Fluid Flow. In the Workshop on Particle Methods in Fluid Dynamics and Plasma Physics.Google Scholar
2. José A. Canabal, David Miraut, Nils Thuerey, Theodore Kim, Javier Portilla, and Miguel A. Otaduy. 2016. Dispersion Kernels for Water Wave Simulation. ACM Trans. Graph. 35, 6, Article 202 (Nov. 2016), 10 pages.Google ScholarDigital Library
3. Mengyu Chu and Nils Thürey. 2017. Data-Driven Synthesis of Smoke Flows with CNN-Based Feature Descriptors. ACM Trans. Graph. (SIGGRAPH) 36, 4 (2017), 1–14.Google ScholarDigital Library
4. Jonathan M Cohen, Lee Markosian, Robert C Zeleznik, John F Hughes, and Ronen Barzel. 1999. An Interface for Sketching 3D Curves. In Proceedings of I3D. 17–21.Google ScholarDigital Library
5. Robert L. Cook. 1986. Stochastic Sampling in Computer Graphics. ACM Trans. Graph. 5, 1 (Jan. 1986), 51–72.Google ScholarDigital Library
6. Fang Da, David Hahn, Christopher Batty, Chris Wojtan, and Eitan Grinspun. 2016. Surface-Only Liquids. ACM Trans. Graph. (SIGGRAPH) 35, 4, Article 78 (July 2016), 12 pages.Google ScholarDigital Library
7. Chris De Paoli and Karan Singh. 2015. SecondSkin: Sketch-Based Construction of Layered 3D Models. ACM Trans. Graph. (SIGGRAPH) 34, 4 (2015), 1–10.Google ScholarDigital Library
8. Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, and Adrien Bousseau. 2018. 3D Sketching Using Multi-View Deep Volumetric Prediction. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 21 (July 2018), 22 pages.Google ScholarDigital Library
9. Nick Foster and Dimitri Metaxas. 1996. Realistic Animation of Liquids. Graphical models and image processing 58, 5 (1996), 471–483.Google Scholar
10. Ming Gao, Xinlei Wang, Kui Wu, Andre Pradhana, Eftychios Sifakis, Cem Yuksel, and Chenfanfu Jiang. 2018. GPU Optimization of Material Point Methods. ACM Trans. Graph. (SIGGRAPH) 37, 6, Article 254 (Dec. 2018), 12 pages.Google Scholar
11. Tony Generico. 2017. Splash: High-Speed Photography With Liquids. CreateSpace Independent Publishing.Google Scholar
12. Yotam Gingold, Takeo Igarashi, and Denis Zorin. 2009. Structured Annotations for 2D-to-3D Modeling. In ACM SIGGRAPH Asia 2009 Papers. Article 148, 9 pages.Google Scholar
13. Éric Guérin, Julie Digne, Éric Galin, Adrien Peytavie, Christian Wolf, Bedrich Benes, and Benoît Martinez. 2017. Interactive Example-Based Terrain Authoring with Conditional Generative Adversarial Networks. ACM Trans. Graph. 36, 6, Article 228 (Nov. 2017), 13 pages.Google ScholarDigital Library
14. Zhongyuan Hu, Haoran Xie, Tsukasa Fukusato, Takahiro Sato, and Takeo Igarashi. 2019. Sketch2VF: Sketch-Based Flow Design with Conditional Generative Adversarial Network. Computer Animation and Virtual Worlds 30, 3–4 (2019), 1889.Google ScholarCross Ref
15. Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2016. Shape Synthesis from Sketches via Procedural Models and Convolutional Networks. IEEE transactions on visualization and computer graphics 23, 8 (2016), 2003–2013.Google Scholar
16. Ruoguan Huang, Zeki Melek, and John Keyser. 2011. Preview-Based Sampling for Controlling Gaseous Simulations. In Proceedings of SCA. 177–186.Google ScholarDigital Library
17. Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. 409–416.Google ScholarDigital Library
18. Markus Ihmsen, Nadir Akinci, Gizem Akinci, and Matthias Teschner. 2012. Unified Spray, Foam and Air Bubbles for Particle-Based Fluids. The Visual Computer 28 (2012), 669–677.Google ScholarDigital Library
19. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. CoRR (2015). arXiv:1502.03167Google Scholar
20. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.Google ScholarCross Ref
21. Stefan Jeschke and Chris Wojtan. 2017. Water Wave Packets. ACM Trans. Graph. (SIGGRAPH) 36, 4, Article 103 (July 2017), 12 pages.Google ScholarDigital Library
22. Byungsoo Kim, Vinicius C Azevedo, Nils Thürey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep Fluids: A Generative Network for Parameterized Fluid Simulations. Computer Graphics Forum (Eurographics) 38, 2 (2019), 59–70.Google ScholarCross Ref
23. Theodore Kim, Jerry Tessendorf, and Nils Thürey. 2013. Closest Point Turbulence for Liquid Surfaces. ACM Trans. Graph. 32, 2, Article 15 (April 2013), 13 pages.Google ScholarDigital Library
24. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR.Google Scholar
25. LâĂŹubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-Driven Fluid Simulations Using Regression Forests. ACM Trans. Graph. 34, 6, Article 199 (Oct. 2015), 9 pages.Google Scholar
26. Jeehyung Lee and Thomas A Funkhouser. 2008. Sketch-Based Search and Composition of 3D models. In Proceedings of SBM. 97–104.Google Scholar
27. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017. BendSketch: Modeling Freeform Surfaces through 2D Sketching. ACM Trans. Graph. (SIGGRAPH) 36, 4, Article 125 (July 2017), 14 pages.Google ScholarDigital Library
28. Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling. ACM Trans. Graph. 37, 6, Article 238 (Dec. 2018), 12 pages.Google ScholarDigital Library
29. Tee Tai Lim and Alexander J. Smits. 2000. Flow Visualization: Techniques and examples. Imperial College Press.Google Scholar
30. Shengjun Liu, Xiaogang Jin, Charlie CL Wang, and Jim X Chen. 2006a. Water-Wave Animation on Mesh Surfaces. Computing in Science & Engineering 8, 5 (2006), 81–87.Google ScholarDigital Library
31. Zhanping Liu, Robert Moorhead, and Joe Groner. 2006b. An Advanced Evenly-Spaced Streamline Placement Algorithm. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 965–972.Google ScholarDigital Library
32. Miles Macklin and Matthias Müller. 2013. Position Based Fluids. ACM Trans. Graph. (SIGGRAPH) 32, 4, Article 104 (July 2013), 12 pages.Google ScholarDigital Library
33. Pierre-Luc Manteaux, Ulysse Vimont, Chris Wojtan, Damien Rohmer, and Marie-Paule Cani. 2016. Space-Time Sculpting of Liquid Animation. In Proceedings of the 9th International Conference on Motion in Games. 61–71.Google ScholarDigital Library
34. Antoine McNamara, Adrien Treuille, Zoran Popović, and Jos Stam. 2004. Fluid Control Using the Adjoint Method. In ACM SIGGRAPH 2004 Papers. 449–456.Google Scholar
35. Olivier Mercier, Cynthia Beauchemin, Nils Thuerey, Theodore Kim, and Derek Nowrouzezahrai. 2015. Surface Turbulence for Particle-Based Liquid Simulations. ACM Trans. Graph. 34, 6, Article 202 (Oct. 2015), 10 pages.Google ScholarDigital Library
36. Mehdi Mirza and Simon Osindero. 2014. Conditional Generative Adversarial Nets. CoRR abs/1411.1784 (2014). arXiv:1411.1784Google Scholar
37. Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: Designing Freeform Surfaces with 3D Curves. In ACM SIGGRAPH 2007 papers. 41–50.Google ScholarDigital Library
38. Andrew Nealen, Olga Sorkine, Marc Alexa, and Daniel Cohen-Or. 2005. A Sketch-Based Interface for Detail-Preserving Mesh Editing. In ACM SIGGRAPH 2005 Papers. 1142–1147.Google ScholarDigital Library
39. Michael B. Nielsen and Robert Bridson. 2011. Guide Shapes for High Resolution Naturalistic Liquid Simulation. In ACM SIGGRAPH 2011 Papers. Article 83, 8 pages.Google Scholar
40. Stanley Osher and Ronald Fedkiw. 2003. Level Set Methods and Dynamic Implicit Surfaces. Springer-Verlag.Google Scholar
41. Zherong Pan, Jin Huang, Yiying Tong, Changxi Zheng, and Hujun Bao. 2013. Interactive Localized Liquid Motion Editing. ACM Trans. Graph. 32, 6, Article 184 (Nov. 2013), 10 pages.Google ScholarDigital Library
42. Zherong Pan and Dinesh Manocha. 2017. Efficient Solver for Spacetime Control of Smoke. ACM Trans. Graph. (SIGGRAPH) 36, 5, Article 162 (July 2017), 13 pages.Google ScholarDigital Library
43. Nick Rasmussen, Douglas Enright, Duc Nguyen, Sebastian Marino, Nigel Sumner, Willi Geiger, Samir Hoon, and Ronald Fedkiw. 2004. Directable Photorealistic Liquids. In Proceedings of SCA. 193–202.Google ScholarDigital Library
44. Karthik Raveendran, Nils Thürey, Chris Wojtan, and Greg Turk. 2012. Controlling Liquids Using Meshes. In Proceedings of SCA. 255–264.Google Scholar
45. Karthik Raveendran, Chris Wojtan, Nils Thürey, and Greg Turk. 2014. Blending Liquids. ACM Trans. Graph. (SIGGRAPH) 33, 4, Article 137 (July 2014), 10 pages.Google ScholarDigital Library
46. Alec Rivers, Frédo Durand, and Takeo Igarashi. 2010. 3D Modeling with Silhouettes. ACM Trans. Graph. (SIGGRAPH) 29, 4, Article 109 (July 2010), 8 pages.Google ScholarDigital Library
47. Lior Rokach and Oded Maimon. 2005. Clustering Methods. In Data mining and knowledge discovery handbook. Springer, 321–352.Google Scholar
48. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), Vol. 9351. 234–241.Google Scholar
49. Bruno Roy, Eric Paquette, and Pierre Poulin. 2020. Particle Upsampling as a Flexible Post-Processing Approach to Increase Details in Animations of Splashing Liquids. Computer & Graphics 88 (2020), 57–69.Google ScholarCross Ref
50. Ryan Schmidt and Karan Singh. 2008. Sketch-Based Procedural Surface Modeling and Compositing Using Surface Trees. In Computer Graphics Forum, Vol. 27. 321–330.Google ScholarCross Ref
51. Lin Shi and Yizhou Yu. 2005. Controllable Smoke Animation with Guiding Objects. ACM Trans. Graph. 24, 1 (Jan. 2005), 140–164.Google ScholarDigital Library
52. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929–1958.Google ScholarDigital Library
53. Nils Thürey, Richard Keiser, Mark Pauly, and Ulrich Rüde. 2009. Detail-Preserving Fluid Control. Graphical Models 71, 6 (2009), 221–228.Google ScholarDigital Library
54. Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2017. Accelerating Eulerian Fluid Simulation with Convolutional Networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. 3424–3433.Google ScholarDigital Library
55. Adrien Treuille, Antoine McNamara, Zoran Popović, and Jos Stam. 2003. Keyframe Control of Smoke Simulations. ACM Trans. Graph. (SIGGRAPH) 22, 3 (July 2003), 716–723.Google ScholarDigital Library
56. Kiwon Um, Xiangyu Hu, and Nils Thürey. 2018. Liquid Splash Modeling with Neural Networks. Computer Graphics Forum 37, 8 (2018), 171–182.Google ScholarCross Ref
57. Nobuyuki Umetani and Bernd Bickel. 2018. Learning Three-Dimensional Flow for Interactive Aerodynamic Design. ACM Trans. Graph. (SIGGRAPH) 37, 4, Article 89 (July 2018), 10 pages.Google ScholarDigital Library
58. Steffen Wiewel, Moritz Becher, and Nils Thürey. 2019. Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow. In Computer Graphics Forum, Vol. 38. 71–82.Google ScholarCross Ref
59. Chris Wojtan, Nils Thürey, Markus Gross, and Greg Turk. 2010. Physics-Inspired Topology Changes for Thin Fluid Features. In ACM SIGGRAPH 2010 Papers. Article 50, 8 pages.Google Scholar
60. You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. TempoGAN: A Temporally Coherent, Volumetric GAN for Super-Resolution Fluid Flow. ACM Trans. Graph. (SIGGRAPH) 37, 4, Article 95 (July 2018), 15 pages.Google ScholarDigital Library
61. Kun Xu, Kang Chen, Hongbo Fu, Wei-Lun Sun, and Shi-Min Hu. 2013. Sketch2Scene: Sketch-Based Co-Retrieval and Co-Placement of 3D Models. ACM Trans. Graph. (SIGGRAPH) 32, 4, Article 123 (July 2013), 15 pages.Google ScholarDigital Library
62. Sheng Yang, Xiaowei He, Huamin Wang, Sheng Li, Guoping Wang, Enhua Wu, and Kun Zhou. 2016. Enriching SPH Simulation by Approximate Capillary Waves. In Proceedings of SCA. 29–36.Google Scholar
63. Bo Zhu, Michiaki Iwata, Ryo Haraguchi, Takashi Ashihara, Nobuyuki Umetani, Takeo Igarashi, and Kazuo Nakazawa. 2011. Sketch-Based Dynamic Illustration of Fluid Systems. ACM Trans. Graph. (SIGGRAPH Asia) 30, 6 (Dec. 2011), 1âĂŞ8.Google Scholar
64. Yongning Zhu and Robert Bridson. 2005. Animating Sand as a Fluid. ACM Trans. Graph. (SIGGRAPH) 24, 3 (July 2005), 965–972.Google ScholarDigital Library
