research-article

Interactive liquid splash modeling by user sketches

Authors:

Huamin WangAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 39, Issue 6

Article No.: 165, Pages 1 - 13

https://doi.org/10.1145/3414685.3417832

Published: 27 November 2020 Publication History

Abstract

Splashing is one of the most fascinating liquid phenomena in the real world and it is favored by artists to create stunning visual effects, both statically and dynamically. Unfortunately, the generation of complex and specialized liquid splashes is a challenging task and often requires considerable time and effort. In this paper, we present a novel system that synthesizes realistic liquid splashes from simple user sketch input. Our system adopts a conditional generative adversarial network (cGAN) trained with physics-based simulation data to produce raw liquid splash models from input sketches, and then applies model refinement processes to further improve their small-scale details. The system considers not only the trajectory of every user stroke, but also its speed, which makes the splash model simulation-ready with its underlying 3D flow. Compared with simulation-based modeling techniques through trials and errors, our system offers flexibility, convenience and intuition in liquid splash design and editing. We evaluate the usability and the efficiency of our system in an immersive virtual reality environment. Thanks to this system, an amateur user can now generate a variety of realistic liquid splashes in just a few minutes.

Supplementary Material

ZIP File (a165-yan.zip)

Supplemental material.

Download
3.68 MB

MP4 File (a165-yan.mp4)

Download
71.70 MB

MP4 File (3414685.3417832.mp4)

Presentation video

Download
409.12 MB

References

[1]

Jeremiah Uhler Brackbill, Douglas B. Kothe, and Hans Max Ruppel. 1988. Flip: A Low-Dissipation, Particle-in-Cell Method for Fluid Flow. In the Workshop on Particle Methods in Fluid Dynamics and Plasma Physics.

[2]

José A. Canabal, David Miraut, Nils Thuerey, Theodore Kim, Javier Portilla, and Miguel A. Otaduy. 2016. Dispersion Kernels for Water Wave Simulation. ACM Trans. Graph. 35, 6, Article 202 (Nov. 2016), 10 pages.

Digital Library

[3]

Mengyu Chu and Nils Thürey. 2017. Data-Driven Synthesis of Smoke Flows with CNN-Based Feature Descriptors. ACM Trans. Graph. (SIGGRAPH) 36, 4 (2017), 1--14.

Digital Library

[4]

Jonathan M Cohen, Lee Markosian, Robert C Zeleznik, John F Hughes, and Ronen Barzel. 1999. An Interface for Sketching 3D Curves. In Proceedings of I3D. 17--21.

Digital Library

[5]

Robert L. Cook. 1986. Stochastic Sampling in Computer Graphics. ACM Trans. Graph. 5, 1 (Jan. 1986), 51--72.

Digital Library

[6]

Fang Da, David Hahn, Christopher Batty, Chris Wojtan, and Eitan Grinspun. 2016. Surface-Only Liquids. ACM Trans. Graph. (SIGGRAPH) 35, 4, Article 78 (July 2016), 12 pages.

Digital Library

[7]

Chris De Paoli and Karan Singh. 2015. SecondSkin: Sketch-Based Construction of Layered 3D Models. ACM Trans. Graph. (SIGGRAPH) 34, 4 (2015), 1--10.

Digital Library

[8]

Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A. Efros, and Adrien Bousseau. 2018. 3D Sketching Using Multi-View Deep Volumetric Prediction. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 21 (July 2018), 22 pages.

Digital Library

[9]

Nick Foster and Dimitri Metaxas. 1996. Realistic Animation of Liquids. Graphical models and image processing 58, 5 (1996), 471--483.

[10]

Ming Gao, Xinlei Wang, Kui Wu, Andre Pradhana, Eftychios Sifakis, Cem Yuksel, and Chenfanfu Jiang. 2018. GPU Optimization of Material Point Methods. ACM Trans. Graph. (SIGGRAPH) 37, 6, Article 254 (Dec. 2018), 12 pages.

Digital Library

[11]

Tony Generico. 2017. Splash: High-Speed Photography With Liquids. CreateSpace Independent Publishing.

[12]

Yotam Gingold, Takeo Igarashi, and Denis Zorin. 2009. Structured Annotations for 2D-to-3D Modeling. In ACM SIGGRAPH Asia 2009 Papers. Article 148, 9 pages.

[13]

Éric Guérin, Julie Digne, Éric Galin, Adrien Peytavie, Christian Wolf, Bedrich Benes, and Benoît Martinez. 2017. Interactive Example-Based Terrain Authoring with Conditional Generative Adversarial Networks. ACM Trans. Graph. 36, 6, Article 228 (Nov. 2017), 13 pages.

Digital Library

[14]

Zhongyuan Hu, Haoran Xie, Tsukasa Fukusato, Takahiro Sato, and Takeo Igarashi. 2019. Sketch2VF: Sketch-Based Flow Design with Conditional Generative Adversarial Network. Computer Animation and Virtual Worlds 30, 3--4 (2019), 1889.

[15]

Haibin Huang, Evangelos Kalogerakis, Ersin Yumer, and Radomir Mech. 2016. Shape Synthesis from Sketches via Procedural Models and Convolutional Networks. IEEE transactions on visualization and computer graphics 23, 8 (2016), 2003--2013.

[16]

Ruoguan Huang, Zeki Melek, and John Keyser. 2011. Preview-Based Sampling for Controlling Gaseous Simulations. In Proceedings of SCA. 177--186.

Digital Library

[17]

Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. 409--416.

Digital Library

[18]

Markus Ihmsen, Nadir Akinci, Gizem Akinci, and Matthias Teschner. 2012. Unified Spray, Foam and Air Bubbles for Particle-Based Fluids. The Visual Computer 28 (2012), 669--677.

Digital Library

[19]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. CoRR (2015). arXiv:1502.03167

[20]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.

[21]

Stefan Jeschke and Chris Wojtan. 2017. Water Wave Packets. ACM Trans. Graph. (SIGGRAPH) 36, 4, Article 103 (July 2017), 12 pages.

Digital Library

[22]

Byungsoo Kim, Vinicius C Azevedo, Nils Thürey, Theodore Kim, Markus Gross, and Barbara Solenthaler. 2019. Deep Fluids: A Generative Network for Parameterized Fluid Simulations. Computer Graphics Forum (Eurographics) 38, 2 (2019), 59--70.

[23]

Theodore Kim, Jerry Tessendorf, and Nils Thürey. 2013. Closest Point Turbulence for Liquid Surfaces. ACM Trans. Graph. 32, 2, Article 15 (April 2013), 13 pages.

Digital Library

[24]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR.

[25]

LâĂ&Zacute;ubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. 2015. Data-Driven Fluid Simulations Using Regression Forests. ACM Trans. Graph. 34, 6, Article 199 (Oct. 2015), 9 pages.

[26]

Jeehyung Lee and Thomas A Funkhouser. 2008. Sketch-Based Search and Composition of 3D models. In Proceedings of SBM. 97--104.

[27]

Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017. BendSketch: Modeling Freeform Surfaces through 2D Sketching. ACM Trans. Graph. (SIGGRAPH) 36, 4, Article 125 (July 2017), 14 pages.

Digital Library

[28]

Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2018. Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface Modeling. ACM Trans. Graph. 37, 6, Article 238 (Dec. 2018), 12 pages.

Digital Library

[29]

Tee Tai Lim and Alexander J. Smits. 2000. Flow Visualization: Techniques and examples. Imperial College Press.

[30]

Shengjun Liu, Xiaogang Jin, Charlie CL Wang, and Jim X Chen. 2006a. Water-Wave Animation on Mesh Surfaces. Computing in Science & Engineering 8, 5 (2006), 81--87.

Digital Library

[31]

Zhanping Liu, Robert Moorhead, and Joe Groner. 2006b. An Advanced Evenly-Spaced Streamline Placement Algorithm. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 965--972.

Digital Library

[32]

Miles Macklin and Matthias Müller. 2013. Position Based Fluids. ACM Trans. Graph. (SIGGRAPH) 32, 4, Article 104 (July 2013), 12 pages.

Digital Library

[33]

Pierre-Luc Manteaux, Ulysse Vimont, Chris Wojtan, Damien Rohmer, and Marie-Paule Cani. 2016. Space-Time Sculpting of Liquid Animation. In Proceedings of the 9th International Conference on Motion in Games. 61--71.

Digital Library

[34]

Antoine McNamara, Adrien Treuille, Zoran Popović, and Jos Stam. 2004. Fluid Control Using the Adjoint Method. In ACM SIGGRAPH 2004 Papers. 449--456.

[35]

Olivier Mercier, Cynthia Beauchemin, Nils Thuerey, Theodore Kim, and Derek Nowrouzezahrai. 2015. Surface Turbulence for Particle-Based Liquid Simulations. ACM Trans. Graph. 34, 6, Article 202 (Oct. 2015), 10 pages.

Digital Library

[36]

Mehdi Mirza and Simon Osindero. 2014. Conditional Generative Adversarial Nets. CoRR abs/1411.1784 (2014). arXiv:1411.1784

[37]

Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: Designing Freeform Surfaces with 3D Curves. In ACM SIGGRAPH 2007 papers. 41--50.

Digital Library

[38]

Andrew Nealen, Olga Sorkine, Marc Alexa, and Daniel Cohen-Or. 2005. A Sketch-Based Interface for Detail-Preserving Mesh Editing. In ACM SIGGRAPH 2005 Papers. 1142--1147.

Digital Library

[39]

Michael B. Nielsen and Robert Bridson. 2011. Guide Shapes for High Resolution Naturalistic Liquid Simulation. In ACM SIGGRAPH 2011 Papers. Article 83, 8 pages.

[40]

Stanley Osher and Ronald Fedkiw. 2003. Level Set Methods and Dynamic Implicit Surfaces. Springer-Verlag.

[41]

Zherong Pan, Jin Huang, Yiying Tong, Changxi Zheng, and Hujun Bao. 2013. Interactive Localized Liquid Motion Editing. ACM Trans. Graph. 32, 6, Article 184 (Nov. 2013), 10 pages.

Digital Library

[42]

Zherong Pan and Dinesh Manocha. 2017. Efficient Solver for Spacetime Control of Smoke. ACM Trans. Graph. (SIGGRAPH) 36, 5, Article 162 (July 2017), 13 pages.

Digital Library

[43]

Nick Rasmussen, Douglas Enright, Duc Nguyen, Sebastian Marino, Nigel Sumner, Willi Geiger, Samir Hoon, and Ronald Fedkiw. 2004. Directable Photorealistic Liquids. In Proceedings of SCA. 193--202.

Digital Library

[44]

Karthik Raveendran, Nils Thürey, Chris Wojtan, and Greg Turk. 2012. Controlling Liquids Using Meshes. In Proceedings of SCA. 255--264.

[45]

Karthik Raveendran, Chris Wojtan, Nils Thürey, and Greg Turk. 2014. Blending Liquids. ACM Trans. Graph. (SIGGRAPH) 33, 4, Article 137 (July 2014), 10 pages.

Digital Library

[46]

Alec Rivers, Frédo Durand, and Takeo Igarashi. 2010. 3D Modeling with Silhouettes. ACM Trans. Graph. (SIGGRAPH) 29, 4, Article 109 (July 2010), 8 pages.

Digital Library

[47]

Lior Rokach and Oded Maimon. 2005. Clustering Methods. In Data mining and knowledge discovery handbook. Springer, 321--352.

[48]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), Vol. 9351. 234--241.

[49]

Bruno Roy, Eric Paquette, and Pierre Poulin. 2020. Particle Upsampling as a Flexible Post-Processing Approach to Increase Details in Animations of Splashing Liquids. Computer & Graphics 88 (2020), 57--69.

[50]

Ryan Schmidt and Karan Singh. 2008. Sketch-Based Procedural Surface Modeling and Compositing Using Surface Trees. In Computer Graphics Forum, Vol. 27. 321--330.

[51]

Lin Shi and Yizhou Yu. 2005. Controllable Smoke Animation with Guiding Objects. ACM Trans. Graph. 24, 1 (Jan. 2005), 140--164.

Digital Library

[52]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929--1958.

Digital Library

[53]

Nils Thürey, Richard Keiser, Mark Pauly, and Ulrich Rüde. 2009. Detail-Preserving Fluid Control. Graphical Models 71, 6 (2009), 221--228.

Digital Library

[54]

Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. 2017. Accelerating Eulerian Fluid Simulation with Convolutional Networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. 3424--3433.

Digital Library

[55]

Adrien Treuille, Antoine McNamara, Zoran Popović, and Jos Stam. 2003. Keyframe Control of Smoke Simulations. ACM Trans. Graph. (SIGGRAPH) 22, 3 (July 2003), 716--723.

Digital Library

[56]

Kiwon Um, Xiangyu Hu, and Nils Thürey. 2018. Liquid Splash Modeling with Neural Networks. Computer Graphics Forum 37, 8 (2018), 171--182.

[57]

Nobuyuki Umetani and Bernd Bickel. 2018. Learning Three-Dimensional Flow for Interactive Aerodynamic Design. ACM Trans. Graph. (SIGGRAPH) 37, 4, Article 89 (July 2018), 10 pages.

Digital Library

[58]

Steffen Wiewel, Moritz Becher, and Nils Thürey. 2019. Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow. In Computer Graphics Forum, Vol. 38. 71--82.

[59]

Chris Wojtan, Nils Thürey, Markus Gross, and Greg Turk. 2010. Physics-Inspired Topology Changes for Thin Fluid Features. In ACM SIGGRAPH 2010 Papers. Article 50, 8 pages.

[60]

You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. 2018. TempoGAN: A Temporally Coherent, Volumetric GAN for Super-Resolution Fluid Flow. ACM Trans. Graph. (SIGGRAPH) 37, 4, Article 95 (July 2018), 15 pages.

Digital Library

[61]

Kun Xu, Kang Chen, Hongbo Fu, Wei-Lun Sun, and Shi-Min Hu. 2013. Sketch2Scene: Sketch-Based Co-Retrieval and Co-Placement of 3D Models. ACM Trans. Graph. (SIGGRAPH) 32, 4, Article 123 (July 2013), 15 pages.

Digital Library

[62]

Sheng Yang, Xiaowei He, Huamin Wang, Sheng Li, Guoping Wang, Enhua Wu, and Kun Zhou. 2016. Enriching SPH Simulation by Approximate Capillary Waves. In Proceedings of SCA. 29--36.

[63]

Bo Zhu, Michiaki Iwata, Ryo Haraguchi, Takashi Ashihara, Nobuyuki Umetani, Takeo Igarashi, and Kazuo Nakazawa. 2011. Sketch-Based Dynamic Illustration of Fluid Systems. ACM Trans. Graph. (SIGGRAPH Asia) 30, 6 (Dec. 2011), 1âĂŞ8.

[64]

Yongning Zhu and Robert Bridson. 2005. Animating Sand as a Fluid. ACM Trans. Graph. (SIGGRAPH) 24, 3 (July 2005), 965--972.

Digital Library

Cited By

Wang YZhang YYang XWang HLiu DYang X(2024)Foveated Fluid Animation in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00074(535-545)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00074
Lei NLi ZXu ZLi YGu X(2024)What's the Situation With Intelligent Mesh Generation: A Survey and PerspectivesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328178130:8(4997-5017)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3281781
Bandyopadhyay HChowdhury PBhunia ASain AXiang TSong Y(2024)What Sketch Explainability Really Means for Downstream Tasks?2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01046(10997-11008)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01046
Show More Cited By

Index Terms

Interactive liquid splash modeling by user sketches
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling

Recommendations

RigMesh: automatic rigging for part-based shape modeling and deformation

The creation of a 3D model is only the first stage of the 3D character animation pipeline. Once a model has been created, and before it can be animated, it must be rigged. Manual rigging is laborious, and automatic rigging approaches are far from real-...
User Modeling and Recommendation Strategies for Tourism
An improved SPH method for modeling liquid sloshing dynamics

Smoothed particle hydrodynamics (SPH) is a popular meshfree, Lagrangian particle method with attractive features in modeling liquid sloshing dynamics, which is usually associated with changing and breakup of free surfaces, strong turbulence and vortex, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 39, Issue 6

December 2020

1605 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3414685

Editor:
Karol Myszkowski
MPI Informatik

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 November 2020

Published in TOG Volume 39, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Adobe
Nvidia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
706
Total Downloads

Downloads (Last 12 months)90
Downloads (Last 6 weeks)8

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YZhang YYang XWang HLiu DYang X(2024)Foveated Fluid Animation in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00074(535-545)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00074
Lei NLi ZXu ZLi YGu X(2024)What's the Situation With Intelligent Mesh Generation: A Survey and PerspectivesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328178130:8(4997-5017)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3281781
Bandyopadhyay HChowdhury PBhunia ASain AXiang TSong Y(2024)What Sketch Explainability Really Means for Downstream Tasks?2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01046(10997-11008)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01046
Wang XXu YLiu SRen BKosinka JTelea AWang JSong CChang JLi CZhang JBan X(2024)Physics-based fluid simulation in computer graphics: Survey, research trends, and challengesComputational Visual Media10.1007/s41095-023-0368-y10:5(803-858)Online publication date: 27-Apr-2024
https://doi.org/10.1007/s41095-023-0368-y
James DLevin D(2023)ViCMA: Visual Control of Multibody AnimationsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618223(1-11)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618223
Deng ZLiu YPan HJabi WZhang JDeng B(2023)Sketch2PQ: Freeform Planar Quadrilateral Mesh Design via a Single SketchIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317085329:9(3826-3839)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TVCG.2022.3170853
Bhunia AKoley SKumar ASain AChowdhury PXiang TSong Y(2023)Sketch2Saliency: Learning to Detect Salient Objects from Human Drawings2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00268(2733-2743)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00268
Aurand JOrtiz RNauer SAzevedo V(2022)Efficient Neural Style Transfer for Volumetric SimulationsACM Transactions on Graphics10.1145/3550454.355551741:6(1-10)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555517
Xiao CSu WLiao JLian ZSong YFu H(2022)DifferSketchingACM Transactions on Graphics10.1145/3550454.355549341:6(1-16)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555493
Goel PJames D(2022)Unified many-worlds browsing of arbitrary physics-based animationsACM Transactions on Graphics10.1145/3528223.353008241:4(1-15)Online publication date: 22-Jul-2022
https://dl.acm.org/doi/10.1145/3528223.3530082
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents