article

Motion magnification

Authors:

Antonio Torralba,

William T. Freeman,

Edward H. AdelsonAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 24, Issue 3

Pages 519 - 526

https://doi.org/10.1145/1073204.1073223

Published: 01 July 2005 Publication History

Abstract

We present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, we need to accurately measure visual motions, and group the pixels to be modified. After an initial image registration step, we measure motion by a robust analysis of feature point trajectories, and segment pixels based on similarity of position, color, and motion. A novel measure of motion similarity groups even very small motions according to correlation over time, which often relates to physical cause. An outlier mask marks observations not explained by our layered motion model, and those pixels are simply reproduced on the output from the original registered observations.The motion of any selected layer may be magnified by a user-specified amount; texture synthesis fills-in unseen "holes" revealed by the amplified motions. The resulting motion-magnified images can reveal or emphasize small motions in the original sequence, as we demonstrate with deformations in load-bearing structures, subtle motions or balancing corrections of people, and "rigid" structures bending under hand pressure.

Supplementary Material

MP4 File (pps015.mp4)

Download
35.32 MB

References

[1]

Arikan, O., and Forsyth, D. A. 2002. Synthesizing constrained motions from examples. ACM Transactions on Graphics 21, 3 (July), 483--490.

Digital Library

[2]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Pat. Anal. Mach. Intell. 23, 11, 1222--1239.

Digital Library

[3]

Brand, M., and Hertzmann, A. 2000. Style machines. In Proceedings of ACM SIGGRAPH 2000, 183--192.

Digital Library

[4]

Brostow, G., and Essa, I. 1999. Motion-based video decompositing. In IEEE International Conference on Computer Vision (ICCV '99), 8--13.

[5]

Brostow, G., and Essa, I. 2001. Image-based motion blur for stop motion animation. In Proceedings of ACM SIGGRAPH 2001, 561--566.

Digital Library

[6]

Dempster, A. P., Laird, N. M., and Rubin. D. B. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Statist. Soc. B 39, 1--38.

[7]

Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In IEEE International Conference on Computer Vision, 1033--1038.

Digital Library

[8]

Fischler, M., and Bolles, R. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24, 6, 381--395.

Digital Library

[9]

Gleicher, M. 1998. Retargetting motion to new characters. In Proceedings of ACM SIGGRAPH 98, 33--42.

Digital Library

[10]

Harris, C., and Stephens, M. 1988. A combined corner and edge detector. In Proceedings of 4th Alvey Vision Conference, 147--151.

[11]

Jojic, N., and Frey, B. 2001. Learning flexible sprites in video layers. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'01), 199--206.

[12]

Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. ACM Transactions on Graphics 21, 3 (July), 491--500.

Digital Library

[13]

Li, Y., Wang, T., and Shum, H.-Y. 2002. Motion texture: A two-level statistical model for character motion synthesis. ACM Transactions on Graphics 21, 3 (July), 465--472.

Digital Library

[14]

Lucas, B., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Image Understanding Workshop, 121--130.

[15]

Nobel, A. 1989. Descriptions of Image Surfaces. PhD thesis, Oxford University, Oxford, UK.

[16]

Pullen, K., and Bregler, C. 2002. Motion capture assisted animation: Texture and synthesis. ACM Transactions on Graphics 21 (July), 501--508.

Digital Library

[17]

Rother, C., Kolmogorov, V., and Blake, A. 2004. Interactive foreground extraction using iterated graph cuts. In Proceedings of ACM SIGGRAPH 2004, 309--314.

Digital Library

[18]

Ruzon, M., and Tomasi. C. 2000. Alpha estimation in natural images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'00), 24--31.

[19]

Sand, P., and Teller, S. 2004. Video matching. In Proceedings of ACM SIGGRAPH 2004, 592--599.

Digital Library

[20]

Sawhney, H., Guo, Y., Hanna, K., Kumar, R., Adkins, S., and Zhou, S. 2001. Hybrid stereo camera: An ibr approach for synthesis of very high resolution stereoscopic image sequences. In Proceedings of ACM SIGGRAPH 2001, 451--460.

Digital Library

[21]

Schodl, A., Szeliski, R., Salesin, D., and Essa, I. 2000. Video textures. In Proceedings of ACM SIGGRAPH 2000, 489--498.

Digital Library

[22]

Shi, J., and Malik, J. 1998. Motion segmentation and tracking using normalized cuts. In Proceedings of International Conference on Computer Vision, 1154--1160.

Digital Library

[23]

Shi. J., and Tomasi, C. 1994. Good features to track. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94), 593--600.

[24]

Strang, G. 1986. Introduction to Applied Mathematics. Wellesley-Cambridge Press.

[25]

Torr, P. 1998. Philosophical Transactions of the Royal Society. Roy Soc, ch. Geometric Motion Segmentation and Model Selection, 1321--1340.

[26]

Unuma, M., Anjyo, K., and Takeuchi, R. 1995. Fourier principles for emotion-based human figure animation. In Proceedings of ACM SIGGRAPH 95, 91--96.

Digital Library

[27]

Verma, D., and Meila, M. 2003. A comparison of spectral methods. Tech. Rep. UW-CSE-03-05-01, ept. of Computer Science and Engineering, University of Washington.

[28]

Wang, J., and Adelson, E. 1994. Representing moving images with layers. IEEE Trans. Image Processing 3, 5, 625--638.

Digital Library

[29]

Weiss, Y., and Adelson, E. 1994. Perceptual organized EM: A framework for motion segmentation that combines information about form and motion. Tech. rep., MIT Media Laboratory Perceptual Computing Section Technical Report No. 315.

[30]

Weiss, Y. 1997. Smoothness in Layers: Motion segmentation using nonparametric mixture estimation. In Proceedings of Computer Vision and Pattern Recognition, 520--527.

Digital Library

[31]

Wills, J., Agarwal, S., and Belongie, S. 2003. What went where. In Proceedings of Computer Vision and Pattern Recognition, 37--44.

Digital Library

[32]

Zelnik-Manor, L., and Irani, M. 2003. Degeneracies, dependencies and their implications in multi-body and multi-sequence factorizations. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'03), 287--293.

Cited By

Lima JMiosso CFarias M(2025)SynFlowMap: A synchronized optical flow remapping for video motion magnificationSignal Processing: Image Communication10.1016/j.image.2024.117203130(117203)Online publication date: Jan-2025
https://doi.org/10.1016/j.image.2024.117203
Shen WYeh TLiao MLi M(2024)Ultrasound B-Mode Visualization of Imperceptible Subwavelength Vibration in Magnetomotive Ultrasound ImagingVibration10.3390/vibration70300407:3(764-775)Online publication date: 12-Jul-2024
https://doi.org/10.3390/vibration7030040
Solano MCerecedo-Nuñez HOvando MFaubert JMisaghian KLugo J(2024)A Method for Directly Observing Mechanical Oscillations in Photonic Structures Based on Porous Silicon NanostructuresMicro10.3390/micro40100064:1(80-96)Online publication date: 1-Feb-2024
https://doi.org/10.3390/micro4010006
Show More Cited By

Index Terms

Motion magnification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Interest point and salient region detections
      2. Image and video acquisition
        Motion capture
  2. Computer graphics
    1. Animation
      1. Motion capture
      2. Motion processing
    2. Image manipulation

Recommendations

Phase-based video motion processing

We introduce a technique to manipulate small movements in videos based on an analysis of motion in complex-valued image pyramids. Phase variations of the coefficients of a complex-valued steerable pyramid over time correspond to motion, and can be ...
Motion magnification
SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

We present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, ...
A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video

Recovering articulated shape and motion, especially human body motion, from video is a challenging problem with a wide range of applications in medical study, sport analysis and animation, etc. Previous work on articulated motion recovery generally ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 24, Issue 3

July 2005

826 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1073204

Issue’s Table of Contents

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2005

Published in TOG Volume 24, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

239
Total Citations
View Citations
3,186
Total Downloads

Downloads (Last 12 months)168
Downloads (Last 6 weeks)22

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lima JMiosso CFarias M(2025)SynFlowMap: A synchronized optical flow remapping for video motion magnificationSignal Processing: Image Communication10.1016/j.image.2024.117203130(117203)Online publication date: Jan-2025
https://doi.org/10.1016/j.image.2024.117203
Shen WYeh TLiao MLi M(2024)Ultrasound B-Mode Visualization of Imperceptible Subwavelength Vibration in Magnetomotive Ultrasound ImagingVibration10.3390/vibration70300407:3(764-775)Online publication date: 12-Jul-2024
https://doi.org/10.3390/vibration7030040
Solano MCerecedo-Nuñez HOvando MFaubert JMisaghian KLugo J(2024)A Method for Directly Observing Mechanical Oscillations in Photonic Structures Based on Porous Silicon NanostructuresMicro10.3390/micro40100064:1(80-96)Online publication date: 1-Feb-2024
https://doi.org/10.3390/micro4010006
Ding TTian LWang Q(2024)Improved video motion magnification method assisted by digital image correlationInternational Conference on Optical and Photonic Engineering (icOPEN 2023)10.1117/12.3023281(43)Online publication date: 15-Feb-2024
https://doi.org/10.1117/12.3023281
Agarwal PMishra NDeshmukh APatil S(2024)Contactless Heart Rate Detection Using Eulerian Video Magnification2024 International Conference on Emerging Smart Computing and Informatics (ESCI)10.1109/ESCI59607.2024.10497207(1-5)Online publication date: 5-Mar-2024
https://doi.org/10.1109/ESCI59607.2024.10497207
Liu DZhuang BMetaxas DChandraker M(2024)Instantaneous Perception of Moving Objects in 3D2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01851(19573-19583)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01851
Wang FGuo DLi KZhong ZWang M(2024)Frequency Decoupling for Motion Magnification Via Multi-Level Isomorphic Architecture2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01796(18984-18994)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01796
Yatim DFridman RBar-Tal OKasten YDekel T(2024)Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00809(8466-8476)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00809
Luo KKong XLi JHu JDeng L(2024)Motion magnification for video-based vibration measurement of civil structures: A reviewMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111681220(111681)Online publication date: Nov-2024
https://doi.org/10.1016/j.ymssp.2024.111681
Čufar KSlavič JBoltežar M(2024)Mode-shape magnification in high-speed camera measurementsMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111336213(111336)Online publication date: May-2024
https://doi.org/10.1016/j.ymssp.2024.111336
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents