research-article

Visual media retargeting

Authors:

Olga SorkineAuthors Info & Claims

SIGGRAPH ASIA '09: ACM SIGGRAPH ASIA 2009 Courses

Article No.: 11, Pages 1 - 13

https://doi.org/10.1145/1665817.1665828

Published: 16 December 2009 Publication History

Abstract

The increasing variety of commonly used display devices, especially mobile devices, requires adapting visual media to different resolutions and aspect ratios - a process called "retargeting." The media retargeting problem is further accentuated by the explosion of image and video content on the web. This course presents a comparative overview of the latest research in visual-media retargeting. It focuses on content-aware approaches, which, contrary to traditional scaling and cropping, adapt to the salient information within the image or video and rescale the content while preserving visually important information.

Topics include:

•Algorithmic details and practical considerations of the retargeting pipeline, including its two main parts (saliency estimation and resizing operators).

•Recent trends in retargeting operators, namely discrete graph-based approaches, also known as seam carving.

•Continuous methods that operate by image and video warping.

•Temporally coherent video retargeting and multi-operator frameworks.

The course illuminates the theoretical foundations and practical issues involved in media retargeting, and provides attendees a comprehensive understanding of the state of the art. It includes many live demos of the various resizing techniques.

References

[1]

Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. Graph. 26, 3, 10.

Digital Library

[2]

Boykov, Y., and Jolly, M.-P. 2001. Interactive graph cuts for optimal boundary&region segmentation of objects in n-d images. In International Conference on Computer Vision, (ICCV), vol. I, 105--112.

[3]

Dalal, N., and Triggs, B. 2005. Histograms of oriented gradients for human detection. In International Conference on Computer Vision&Pattern Recognition, vol. 2, 886--893.

Digital Library

[4]

DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. In Proceedings of SIGGRAPH, 769--776.

Digital Library

[5]

Fan, X., Xie, X., Zhou, H.-Q., and Ma, W.-Y. 2003. Looking into video frames on small displays. In Multimedia '03, ACM, New York, NY, USA, 247--250.

Digital Library

[6]

Gal, R., Sorkine, O., and Cohen-Or, D. 2006. Feature-aware texturing. In Eurographics Symposium on Rendering, 297--303.

Digital Library

[7]

Harris, C., and Stephens, M. 1988. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, 147--151.

[8]

Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11, 1254--1259.

Digital Library

[9]

Krähenbühl, P., Lang, M., Hornung, A., and Gross, M. 2009. A system for retargeting of streaming video. ACM Trans. Graph. 28, 5.

Digital Library

[10]

Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3, 277--286.

Digital Library

[11]

Lombaert, H., Sun, Y., Grady, L., and Xu, C. 2005. A multilevel banded graph cuts method for fast image segmentation. In Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV), vol. 1, 259--265.

Digital Library

[12]

Roditty, L., and Zwick, U. 2004. On dynamic shortest paths problems. In In Proceedings of 12th Annual European Symposium on Algorithms (ESA), 580--591.

[13]

Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. ACM Trans. Graph. 27, 3.

Digital Library

[14]

Rubinstein, M., Shamir, A., and Avidan, S. 2009. Multioperator media retargeting. ACM Trans. Graph. 28, 3.

Digital Library

[15]

Santella, A., Agrawala, M., DeCarlo, D., Salesin, D., and Cohen, M. 2006. Gaze-based interaction for semiautomatic photo cropping. In Proceedings of CHI, 771--780.

Digital Library

[16]

Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., 489--498.

Digital Library

[17]

Shamir, A., and Avidan, S. 2009. Seam carving for media retargeting. Communications of the ACM 52, 1, 77--85.

Digital Library

[18]

Viola, P., and Jones, M. J. 2004. Robust real-time face detection. Int. J. Comput. Vision 57, 2, 137--154.

Digital Library

[19]

Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. Graph. 23, 3, 574--583.

Digital Library

[20]

Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graph. 24, 3, 585--594.

Digital Library

[21]

Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27, 5.

Digital Library

[22]

Wang, Y.-S., Fu, H., Sorkine, O., Lee, T.-Y., and Seidel, H.-P. 2009. Motion-aware temporal coherence for video resizing. ACM Trans. Graph. (Proceedings of ACM SIGGRAPH ASIA) 28, 5.

Digital Library

[23]

Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In IEEE International Conference on Computer Vision (ICCV).

[24]

Zabih, R., Miller, J., and Mai, K. 1995. A feature-based algorithm for detecting and classifying scene breaks. In ACM Multimedia, 189--200.

Digital Library

[25]

Zhang, Y.-F., Hu, S.-M., and Martin, R. R. 2008. Shrinkability maps for content-aware video resizing. Computer Graphics Forum 27, 7, 1797--1804.

Cited By

Jia ZZhang ZWang LTan T(2024)Human Image Generation: A Comprehensive SurveyACM Computing Surveys10.1145/366586956:11(1-39)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3665869
Fan RWang LLiu XIm SLam C(2023)Real-scene-constrained virtual scene layout synthesis for mixed realityThe Visual Computer10.1007/s00371-023-03167-440:9(6319-6339)Online publication date: 12-Dec-2023
https://doi.org/10.1007/s00371-023-03167-4
Liu LSun QYuan YZhai Y(2022)Saliency Preserving Image Retargeting Using Probability Density2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC )10.1109/IAEAC54830.2022.9929513(1951-1959)Online publication date: 3-Oct-2022
https://doi.org/10.1109/IAEAC54830.2022.9929513
Show More Cited By

Index Terms

Visual media retargeting
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation

Recommendations

A comparative study of image retargeting
SIGGRAPH ASIA '10: ACM SIGGRAPH Asia 2010 papers

The numerous works on media retargeting call for a methodological approach for evaluating retargeting results. We present the first comprehensive perceptual study and analysis of image retargeting. First, we create a benchmark of images and conduct a ...
Issues and Challenges in Content-Based Media Retargeting
CSNT '14: Proceedings of the 2014 Fourth International Conference on Communication Systems and Network Technologies

Smartphones are rapidly increasing in this technological world, in that visual media is the most important thing in mobile arena and due to visual graphical world everyone wants high graphical visual media. Now a day demands of videos and images are ...
Compressibility-Aware Media Retargeting With Structure Preserving

A number of algorithms have been proposed for intelligent image/video retargeting with image content retained as much as possible. However, they usually suffer from some artifacts in the results, such as ridge or structure twist. In this paper, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH ASIA '09: ACM SIGGRAPH ASIA 2009 Courses

December 2009

2555 pages

ISBN:9781450379311

DOI:10.1145/1665817

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 December 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SA09

Sponsor:

SIGGRAPH

SA09: SIGGRAPH ASIA 2009

December 16 - 19, 2009

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

132
Total Citations
View Citations
1,686
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jia ZZhang ZWang LTan T(2024)Human Image Generation: A Comprehensive SurveyACM Computing Surveys10.1145/366586956:11(1-39)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3665869
Fan RWang LLiu XIm SLam C(2023)Real-scene-constrained virtual scene layout synthesis for mixed realityThe Visual Computer10.1007/s00371-023-03167-440:9(6319-6339)Online publication date: 12-Dec-2023
https://doi.org/10.1007/s00371-023-03167-4
Liu LSun QYuan YZhai Y(2022)Saliency Preserving Image Retargeting Using Probability Density2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC )10.1109/IAEAC54830.2022.9929513(1951-1959)Online publication date: 3-Oct-2022
https://doi.org/10.1109/IAEAC54830.2022.9929513
Hsin HSu C(2022)SHE-Based Video Retargeting with Multiple Ground Control PointsProceedings of the Future Technologies Conference (FTC) 2022, Volume 210.1007/978-3-031-18458-1_54(797-805)Online publication date: 13-Oct-2022
https://doi.org/10.1007/978-3-031-18458-1_54
Dong ZWu WXu ZSun QYuan GLiu LFu X(2021)Tailored Reality: Perception-aware Scene Restructuring for Adaptive VR NavigationACM Transactions on Graphics10.1145/347084740:5(1-15)Online publication date: 8-Oct-2021
https://dl.acm.org/doi/10.1145/3470847
Li LLi YWu JMa LFang Y(2021)Quality Evaluation for Image Retargeting With Instance SemanticsIEEE Transactions on Multimedia10.1109/TMM.2020.301612423(2757-2769)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1109/TMM.2020.3016124
Valdez-Balderas DMuraveynyk OSmith T(2021)Fast Hybrid Image Retargeting2021 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP42928.2021.9506584(1849-1853)Online publication date: 19-Sep-2021
https://doi.org/10.1109/ICIP42928.2021.9506584
Hsin HYang CSu C(2021)Image Resizing in DCT DomainAdvances in Data Science and Information Engineering10.1007/978-3-030-71704-9_58(799-805)Online publication date: 30-Oct-2021
https://doi.org/10.1007/978-3-030-71704-9_58
Han XHou KDu DQiu YCui SZhou KYu Y(2020)CaricatureShop: Personalized and Photorealistic Caricature SketchingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.288600726:7(2349-2361)Online publication date: 1-Jul-2020
https://doi.org/10.1109/TVCG.2018.2886007
Liu YHan YYe ZLai Y(2020)Ranking-Preserving Cross-Source Learning for Image Retargeting Quality AssessmentIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.292399842:7(1798-1805)Online publication date: 1-Jul-2020
https://doi.org/10.1109/TPAMI.2019.2923998
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents