Frame Rate Up-Conversion Algorithm Based on Infrared Image
Pages 225 - 230
Abstract
With the more and more common use of infrared video in life, people have higher and higher requirements for infrared imaging frame rates. Better quality infrared images provide a better basis for subsequent target recognition, video compression and decompression operations. Therefore, it is of great significance to effectively obtain infrared high frame rate images and improve the quality of infrared images. As an effective means of video conversion, frame rate enhancement technology has become a research hotspot in the direction of computer vision. However, the existing video interpolation methods based on infrared images are all implemented based on block matching, which cannot well approximate the complex moving real world. In order to solve these problems, an optical flow method based on pixel motion compensation is proposed for infrared video interpolation. The second interpolation method can make better use of the motion information in the video. In the end, an optical flow optimization network is used for optimization, which can better optimize the artifacts in the optical flow estimation and improve the final image quality. Experiments show that our method has a better effect than existing methods on models on various infrared video data sets, and has lower computational complexity.
References
[1]
Salton G, McGill M J. Introduction to modern information retrieval[J]. 1986.
[2]
Sullivan G J, Ohm J, Han W-J, Overview of the high efficiency video coding (HEVC) standard[J]. IEEE Transactions on circuits and systems for video technology, 2012, 22(12) : 1649 –1668.
[3]
Bauer D, Bonacker M, Cavonius C R . Frame repetition rate for flicker-free viewing of bright VDU screens[J]. Displays, Technology and Applications, 1983, 4(1):31-33.
[4]
Richard C, David D, Yadong L I. MOTION ADAPTIVE FRAME AVERAGING FOR ULTRASOUND DOPPLER COLOR FLOW IMAGING:, US 7153268 B2[P]. 2006.
[5]
Yan N, Liu D, Li H, A convolutional neural network approach for halfpel interpolation in video coding[C] // Circuits and Systems (ISCAS), 2017 IEEE International Symposium on. 2017 : 1–4.
[6]
Chen Z, He T, Jin X, Learning for Video Compression[J]. arXiv preprint arXiv:1804.09869, 2018]
[7]
Dosovitskiy A, Fischery P, Ilg E, FlowNet: Learning Optical Flow with Convolutional Networks[C]// IEEE International Conference on Computer Vision. 2015.
[8]
Ilg E, Mayer N, Saikia T, FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks[C]// IEEE Conference on Computer Vision & Pattern Recognition. 2017.
[9]
Zhou T, Tulsiani S, Sun W, View synthesis by appearance flow[C] // European conference on computer vision. 2016 : 286–301.
[10]
Liu Z, Yeh R, Tang X, Video frame synthesis using deep voxel flow[C]// International Conference on Computer Vision (ICCV) : Vol 2. 2017.
[11]
Niklaus S, Mai L, Liu F. Video frame interpolation via adaptive separable convolution[J]. arXiv preprint arXiv:1708.01692, 2017.
[12]
Niklaus S, Liu F. Context-aware Synthesis for Video Frame Interpolation[J]. 2018.
[13]
Jiang H, Sun D, Jampani V, Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation[J]. 2017.
[14]
Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv preprint arXiv:1502.03167, 2015.
[15]
Santurkar S, Tsipras D, Ilyas A, How Does Batch Normalization Help Optimization?(No, It Is Not About Internal Covariate Shift)[J]. arXiv preprint arXiv:1805.11604, 2018.
[16]
Horn B K P, Schunck B G. Determining optical flow[J]. Artificial Intelligence, 1980, 17(1–3):185-203.
[17]
Lucas B D, Kanade T. Optical navigation by the method of differences[C]// International Joint Conference on Artificial Intelligence. 1985.
[18]
Fleet DJ, Jepson A D. Computation of component image velocity from local phase information[J]. International Journal of Computer Vision, 1990, 5(1):77-104.
[19]
Mahajan D, Huang F C, Matusik W, Moving gradients: a path-based method for plausible image interpolation[C]// Acm Siggraph. 2009.
[20]
N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman. Phase-based video motion processing. ACM Trans. Graph., 32(4):80:1–80:10, 2013.
[21]
P. Didyk, P. Sitthi-amorn, W. T. Freeman, F. Durand, and W. Matusik. Joint view expansion and filtering for automultiscopic 3D displays. ACM Trans. Graph., 32(6):221:1–221:8, 2013.
[22]
Meister S, Hur J, Roth S. UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss[J]. 2017.
[23]
Liu P, Lyu M, King I, SelFlow: Self-Supervised Learning of Optical Flow[J]. 2019.
[24]
Hur J, Roth S . Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation[J]. 2019.
[25]
Long G, Kneip L, Alvarez J M, Learning Image Matching by Simply Watching Video[C]// European Conference on Computer Vision. Springer, Cham, 2016.
- Frame Rate Up-Conversion Algorithm Based on Infrared Image
Recommendations
Adaptive frame rate up-conversion based on motion classification
In this paper, a new technique on video frame rate up-conversion (FRUC) is presented by combining the adaptive motion classification (AMC) for image sequences and the mixed motion estimation (ME). In the proposed FRUC scheme, the AMC classifies ...
Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion
In this paper, we present a new motion compensated frame rate up-conversion algorithm that uses multiframes to enhance the accuracy of motion estimation. We also develop adaptive motion vector smoothing to correct outliers in a motion vector field. In ...
Dual Motion Estimation for Frame Rate Up-Conversion
In this letter, we present a new motion estimation algorithm for frame rate up-conversion. The proposed dual motion estimation algorithm enhances the estimation accuracy of motion vectors by using the unidirectional and bidirectional matching ratios of ...
Comments
Information & Contributors
Information
Published In
June 2021
276 pages
ISBN:9781450389792
DOI:10.1145/3474963
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 14 October 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCMS '21
ICCMS '21: 2021 The 13th International Conference on Computer Modeling and Simulation
June 25 - 27, 2021
VIC, Melbourne, Australia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 41Total Downloads
- Downloads (Last 12 months)6
- Downloads (Last 6 weeks)1
Reflects downloads up to 28 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format