research-article

Fast image/video upsampling

Authors:

Chi-Keung TangAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 27, Issue 5

Article No.: 153, Pages 1 - 7

https://doi.org/10.1145/1409060.1409106

Published: 01 December 2008 Publication History

Abstract

We propose a simple but effective upsampling method for automatically enhancing the image/video resolution, while preserving the essential structural information. The main advantage of our method lies in a feedback-control framework which faithfully recovers the high-resolution image information from the input data, without imposing additional local structure constraints learned from other examples. This makes our method independent of the quality and number of the selected examples, which are issues typical of learning-based algorithms, while producing high-quality results without observable unsightly artifacts. Another advantage is that our method naturally extends to video upsampling, where the temporal coherence is maintained automatically. Finally, our method runs very fast. We demonstrate the effectiveness of our algorithm by experimenting with different image/video data.

Supplementary Material

JPG File (a153-shan-mp4_hi.jpg)

Download
13.72 KB

MOV File (a153-shan-mp4_hi.mov)

Download
379.40 MB

References

[1]

Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 10.

[2]

Baker, S., and Kanade, T. 2000. Limits on super-resolution and how to break them. In CVPR, IEEE Computer Society, 2372--2379.

[3]

Bando, Y., and Nishita, T. 2007. Towards digital refocusing from a single photograph. In Pacific Graphics.

[4]

Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Eurographics Symposium on Rendering, Eurographics, 327--338.

[5]

Bishop, C. M., Blake, A., and Marthi, B. 2003. Super-resolutioin enhancement of video. In In 9th Conf. on Artificial Intelligence and Statistics.

[6]

Chan, T. F., Osher, S., and Shen, J. 2001. The digital tv filter and nonlinear denoising. IEEE Trans. on Image Processing 10, 2, 231--241.

Digital Library

[7]

Fattal, R. 2007. Image upsampling via imposed edges statistics. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 95.

[8]

Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., and Freeman, W. T. 2006. Removing camera shake from a single photograph. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006) 25, 787--794.

[9]

Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Computer Graphics and Applications 22, 56--65.

Digital Library

[10]

Frigo, M., and Johnson, S. G., 2006. FFTW Home Page. WWW page. http://www.fftw.org/.

[11]

Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. 2001. Image analogies. In SIGGRAPH, 327--340.

[12]

Huang, J. G., and Mumford, D. 1999. Statistics of natural images and models. In CVPR, 1541--1547.

[13]

Irani, M., and Peleg, S. 1993. Motion analysis for image enhancement: resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation 4, 324--335.

[14]

Keys, R. G. 1981. Cubic convolution interpolation for digital image processing. IEEE Trans. Acoustics, Speech, and Signal Processing 29, 1153--1160.

[15]

Kong, D., Han, M., Xu, W., Tao, H., and Gong, Y. H. 2006. Video super-resolution with scene-specific priors. In BMVC.

[16]

Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. 2007. Joint bilateral upsampling. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2007) 26, 96.

[17]

Mon, D., 2006. Video enhancer. WWW page. http://www.thedeemon.com/VideoEnhancer/.

[18]

Osher, S., Sole, A., and Vese, L. 2003. Image decomposition and restoration using total variation minimization and the H ^-1. Multiscale Modeling and Simulation 1, 3, 349--370.

[19]

Patti, A., Sezan, M., and Tekalp, A. 1997. Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. on Image Processing 6, 1064--1076.

Digital Library

[20]

QELabs, 2005. Qe super resolution. WWW page. http://www.qelabs.com/index.

[21]

Schultz, R. R., and Stevenson, R. L. 1996. Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 5, 996--1011.

Digital Library

[22]

Shan, Q., Jia, J. Y., and Agarwala, A. 2008. High-quality motion deblurring from a single image. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008).

[23]

Sun, J., Sun, J., Xu, Z. B., and Shum, H. Y. 2008. Image super-rosolution using gradient profile prior. In CVPR.

[24]

Tappen, M. F., Russell, B. C., and Freeman, W. T. 2003. Exploiting the sparse derivative prior for super-resolution and image demosaicing. In Intl. Workshop on Statistical and Computational Theories of Vision.

[25]

Tappen, M. F., Russell, B. C., and Freeman, W. T. 2004. Efficient graphical models for processing images. In CVPR, IEEE Computer Society, 673--680.

[26]

Tipping, M. E., and Bishop, C. M. 2002. Bayesian image super resolution. In NIPS, MIT Press, 1279--1286.

[27]

Yuan, L., Sun, J., Quan, L., and Shum, H. Y. 2008. Progressive inter-scale and intra-scale non-blind image deconvolution. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2008).

[28]

Zhao, W. Y., and Sawhney, H. S. 2002. Is super-resolution with optical flow feasible. In ECCV, Springer, 599--613.

Cited By

Zhu MLiu JWang F(2024)Blind Deblurring Method for CASEarth Multispectral Images Based on Inter-Band Gradient Similarity PriorSensors10.3390/s2419625924:19(6259)Online publication date: 27-Sep-2024
https://doi.org/10.3390/s24196259
Qiu YZhang HYang KZhai TLu YCao ZZhang Z(2024)High-Resolution Image Processing of Probe-Based Confocal Laser Endomicroscopy Based on Multistage Neural Networks and Cross-Channel Attention ModulePhotonics10.3390/photonics1102010611:2(106)Online publication date: 25-Jan-2024
https://doi.org/10.3390/photonics11020106
Li SGao YWu JCao L(2024)Blind deblurring of astronomical images using a SCGTV-based single-frame methodOptics Express10.1364/OE.53374832:20(35579)Online publication date: 18-Sep-2024
https://doi.org/10.1364/OE.533748
Show More Cited By

Index Terms

Fast image/video upsampling
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Image upsampling via imposed edge statistics
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

In this paper we propose a new method for upsampling images which is capable of generating sharp edges with reduced input-resolution grid-related artifacts. The method is based on a statistical edge dependency relating certain edge features of two ...
Fast image/video upsampling
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers

We propose a simple but effective upsampling method for automatically enhancing the image/video resolution, while preserving the essential structural information. The main advantage of our method lies in a feedback-control framework which faithfully ...
High-quality image deblurring with panchromatic pixels

Image deblurring has been a very challenging problem in recent decades. In this article, we propose a high-quality image deblurring method with a novel image prior based on a new imaging system. The imaging system has a newly designed sensor pattern ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 27, Issue 5

December 2008

552 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1409060

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008

Published in TOG Volume 27, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Research Grants Council, University Grants Committee, Hong Kong

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

268
Total Citations
View Citations
2,284
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)10

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu MLiu JWang F(2024)Blind Deblurring Method for CASEarth Multispectral Images Based on Inter-Band Gradient Similarity PriorSensors10.3390/s2419625924:19(6259)Online publication date: 27-Sep-2024
https://doi.org/10.3390/s24196259
Qiu YZhang HYang KZhai TLu YCao ZZhang Z(2024)High-Resolution Image Processing of Probe-Based Confocal Laser Endomicroscopy Based on Multistage Neural Networks and Cross-Channel Attention ModulePhotonics10.3390/photonics1102010611:2(106)Online publication date: 25-Jan-2024
https://doi.org/10.3390/photonics11020106
Li SGao YWu JCao L(2024)Blind deblurring of astronomical images using a SCGTV-based single-frame methodOptics Express10.1364/OE.53374832:20(35579)Online publication date: 18-Sep-2024
https://doi.org/10.1364/OE.533748
Qin KLi MFeng HYang HChen JChen Y(2024)Ground-to-air aircraft infrared image deblurring based on imaging degradation simulationOptics Express10.1364/OE.52457132:17(29721)Online publication date: 2-Aug-2024
https://doi.org/10.1364/OE.524571
Dong WWang QTao STian C(2024)Blind multi-Poissonian image deconvolution with sparse log-step gradient priorOptics Express10.1364/OE.51360432:6(9061)Online publication date: 28-Feb-2024
https://doi.org/10.1364/OE.513604
Bai HPan J(2024)Self-Supervised Deep Blind Video Super-ResolutionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336116846:7(4641-4653)Online publication date: Jul-2024
https://doi.org/10.1109/TPAMI.2024.3361168
Wang HChen JZheng YZeng T(2024)Navigating Beyond Dropout: An Intriguing Solution Towards Generalizable Image Super Resolution2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02412(25532-25543)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02412
Duan LZhong LZhang J(2024)Turbulent image deblurring using a deblurred blur kernelJournal of Optics10.1088/2040-8986/ad3e0e26:6(065702)Online publication date: 2-May-2024
https://doi.org/10.1088/2040-8986/ad3e0e
Gao HFeng M(2024)Blind deblurring text images via Beltrami regularizationImage and Vision Computing10.1016/j.imavis.2024.105080(105080)Online publication date: May-2024
https://doi.org/10.1016/j.imavis.2024.105080
Xu XLin YYe C(2024)Real-time prediction for temperature distribution on the cylinder head of dual-fuel engines via a novel deep learning frameworkExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122357238:PFOnline publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122357
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents