Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Animating Still Natural Images Using Warping

Published: 05 January 2023 Publication History

Abstract

From a single still image, a looping video could be generated by imparting subtle motion to objects in the image. The results are a hybrid of photography and video. They contain gentle motion in some objects, while the rest of the image remains still. Existing techniques are successful in animating such images. However, there are still some drawbacks that need to be investigated, such as too-large computation time necessary to retrieve the matched videos or the challenges of controlling the desired motion not only in terms of a single region but also in terms of consistency in regions. In this work, we address these issues by proposing an interactive system with a novel warping method. The key idea of our approach is to utilize user’s annotations to impart motion to certain objects. With two proposed phases in terms of preserve-curve-warping and cycle warping, a looping video is generated. We demonstrate the effectiveness of our method via various experimental challenging results and evaluations. We show that with a simple and lightweight method, our system is able to deal with animating a still image’s problems and results in realistic motion and appealing videos. In addition, using our proposed system, it is easy to create plausible animation using simple user annotations without referencing the video database or machine learning models and allows ordinary users with minimal expertise to produce compelling results.
Appendix

References

[1]
[2]
George B. Arfken and Hans J. Weber. 1999. Mathematical Methods for Physicists: A Comprehensive Guide. Academic Press.
[3]
Kiran S. Bhat, Steven M. Seitz, Jessica K. Hodgins, and Pradeep K. Khosla. 2004. Flow-based video synthesis and editing. In ACM SIGGRAPH 2004 Papers. 360–363.
[4]
Yung-Yu Chuang, Dan B. Goldman, Ke Colin Zheng, Brian Curless, David H. Salesin, and Richard Szeliski. 2005. Animating pictures with stochastic motion textures. In ACM SIGGRAPH 2005 Papers. 853–860.
[5]
Yuki Endo, Yoshihiro Kanamori, and Shigeru Kuriyama. 2019. Animating landscape: Self-supervised learning of decoupled motion and appearance for single-image video synthesis. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1–19.
[6]
Eirik Eng. 1996. Qt GUI Toolkit: Porting graphics to multiple platforms using a GUI toolkit. Linux J. 1996, 31es (1996), 2–es.
[7]
William T. Freeman, Edward H. Adelson, and David J. Heeger. 1991. Motion without movement. ACM Siggraph Comput. Graph. 25, 4 (1991), 27–30.
[8]
Ruohan Gao, Bo Xiong, and Kristen Grauman. 2018. Im2flow: Motion hallucination from static images for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5937–5947.
[9]
Eduardo S. L. Gastal and Manuel M. Oliveira. 2010. Shared sampling for real-time alpha matting. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 575–584.
[10]
Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. Retrieved from http://eigen.tuxfamily.org.
[11]
Yan Gui, Li-zhuang Ma, Chao Yin, and Zhi-hua Chen. 2012. Preserving global features of fluid animation from a single image using video examples. J. Zhejiang Univ. Sci. C 13, 7 (2012), 510–519.
[12]
Aleksander Holynski, Brian Curless, Steven M. Seitz, and Richard Szeliski. 2020. Animating pictures with eulerian motion fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5810–5819.
[13]
Kyung-Rae Kim, Whan Choi, Yeong Jun Koh, Seong-Gyun Jeong, and Chang-Su Kim. 2019. Instance-level future motion estimation in a single image based on ordinal regression. In Proceedings of the IEEE International Conference on Computer Vision. 273–282.
[14]
Kris M. Kitani, Brian D. Ziebart, James Andrew Bagnell, and Martial Hebert. 2012. Activity forecasting. In Proceedings of the European Conference on Computer Vision. Springer, 201–214.
[15]
Yu-Chi Lai, Bo-An Chen, Kuo-Wei Chen, Wei-Lin Si, Chih-Yuan Yao, and Eugene Zhang. 2016. Data-driven npr illustrations of natural flows in chinese painting. IEEE Trans. Vis. Comput. Graph. 23, 12 (2016), 2535–2549.
[16]
Zicheng Liao, Neel Joshi, and Hugues Hoppe. 2013. Automated video looping with progressive dynamism. ACM Trans Graph. 32, 4 (2013), 1–10.
[17]
Chih-Yang Lin, Yun-Wen Huang, and Timothy K. Shih. 2019. Creating waterfall animation on a single image. Multimedia Tools Appl. 78, 6 (2019), 6637–6653.
[18]
Elizaveta Logacheva, Roman Suvorov, Oleg Khomenko, Anton Mashikhin, and Victor Lempitsky. 2020. Deeplandscape: Adversarial modeling of landscape videos. In Proceedings of the European Conference on Computer Vision. Springer, 256–272.
[19]
Wei-Chiu Ma, De-An Huang, Namhoon Lee, and Kris M. Kitani. 2017. Forecasting interactive dynamics of pedestrians with fictitious play. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 774–782.
[20]
Eric N. Mortensen and William A. Barrett. 1995. Intelligent scissors for image composition. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. 191–198.
[21]
Roozbeh Mottaghi, Hessam Bagherinezhad, Mohammad Rastegari, and Ali Farhadi. 2016. Newtonian scene understanding: Unfolding the dynamics of objects in static images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3521–3529.
[22]
Makoto Okabe, Ken Anjyo, Takeo Igarashi, and Hans-Peter Seidel. 2009. Animating pictures of fluid using video examples. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 677–686.
[23]
Makoto Okabe, Ken Anjyor, and Rikio Onai. 2011. Creating fluid animation from a single image using video database. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 1973–1982.
[24]
Makoto Okabe, Yoshinori Dobashi, and Ken Anjyo. 2018. Animating pictures of water scenes using video retrieval. Vis. Comput. 34, 3 (2018), 347–358.
[25]
Pixaloop 2021. Retrieved from https://www.pixaloopapp.com/.
[26]
Plotagraph 2017. Retrieved from https://plotaverseapps.com/.
[27]
Ekta Prashnani, Maneli Noorkami, Daniel Vaquero, and Pradeep Sen. 2017. A phase-based approach for animating images using video examples. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 303–311.
[28]
William H. Press, William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling, Brian P. Flannery, and William T. Vetterling. 1989. Numerical Recipes in Pascal: The Art of Scientific Computing. Vol. 1. Cambridge University Press.
[29]
Arno Schödl, Richard Szeliski, David H. Salesin, and Irfan Essa. 2000. Video textures. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. 489–498.
[30]
Eric W. Weisstein. 2005. Moore neighborhood. In From MathWorld—A Wolfram Web Resource.
[31]
Wikipedia contributors. 2020. Linear interpolation. Retrieved January 8, 2021 from https://en.wikipedia.org/w/index.php?title=Linear_interpolation&oldid=986522475.
[32]
Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani, and Yoichi Sato. 2018. Future person localization in first-person videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7593–7602.
[33]
Jin Zhang. 2007. Visualization for Information Retrieval. Vol. 23. Springer Science & Business Media.

Cited By

View all
  • (2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 16-May-2024
  • (2023)Simulating Fluids in Real-World Still Images2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01459(15876-15885)Online publication date: 1-Oct-2023

Index Terms

  1. Animating Still Natural Images Using Warping

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 1
    January 2023
    505 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/3572858
    • Editor:
    • Abdulmotaleb El Saddik
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 January 2023
    Online AM: 18 February 2022
    Accepted: 17 January 2022
    Revised: 11 December 2021
    Received: 04 September 2021
    Published in TOMM Volume 19, Issue 1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Animation
    2. still images
    3. cycle warping
    4. preserve-curve-warping

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • Ministry of Science and Technology
    • Key Area Research Program of Universities in Guangdong Province (Nature science), China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)125
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 10 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 16-May-2024
    • (2023)Simulating Fluids in Real-World Still Images2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.01459(15876-15885)Online publication date: 1-Oct-2023

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media