research-article

CASAIR: Content and Shape-Aware Image Retargeting and Its Applications

Authors:

Yu-Tseh Jason Chi,

Adrian M. Peter,

Jeffrey HoAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 25, Issue 5

Pages 2222 - 2232

https://doi.org/10.1109/TIP.2016.2528040

Published: 01 May 2016 Publication History

Abstract

This paper proposes a novel image-retargeting algorithm that can retarget images to a large family of non-rectangular shapes. Specifically, we study image retargeting from a broader perspective that includes the content as well as the shape of an image, and the proposed content and shape-aware image-retargeting (CASAIR) algorithm is driven by the dual objectives of image content preservation and image domain transformation, with the latter defined by an application-specific target shape. The algorithm is based on the idea of seam segment carving that successively removes low-cost seam segments from the image to simultaneously achieve the two objectives, with the selection of seam segments determined by a cost function incorporating inputs from image content and target shape. To provide a complete characterization of shapes that can be obtained using CASAIR, we introduce the notion of bhv-convex shapes, and we show that bhv-convex shapes are precisely the family of shapes that can be retargeted to by CASAIR. The proposed algorithm is simple in both its design and implementation, and in practice, it offers an efficient and effective retargeting platform that provides its users with considerable flexibility in choosing target shapes. To demonstrate the potential of CASAIR for broadening the application scope of image retargeting, this paper also proposes a smart camera–projector system that incorporates CASAIR. In the context of ubiquitous display, CASAIR equips the camera–projector system with the capability of retargeting images online in order to maximize the quality and fidelity of the displayed images whenever the situation demands.

References

[1]

S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graph., vol. 26, no. 3, Jul. 2007, Art. no.

Digital Library

[2]

Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, “Optimized scale-and-stretch for image resizing,” ACM Trans. Graph., vol. 27, no. 5, Dec. 2008, Art. no.

Digital Library

[3]

Z. Karni, D. Freedman, and C. Gotsman, “Energy-based image deformation,” Comput. Graph. Forum, vol. 28, no. 5, pp. 1257–1268, 2009.

[4]

Y. Ding, J. Xiao, and J. Yu, “Importance filtering for image retargeting,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 89–96.

[5]

V. Setlur, S. Takagi, R. Raskar, M. Gleicher, and B. Gooch, “Automatic image retargeting,” in Proc. 4th Int. Conf. Mobile Ubiquitous Multimedia, 2005, pp. 59–68.

[6]

A. Santella, M. Agrawala, D. DeCarlo, D. Salesin, and M. Cohen, “Gaze-based interaction for semi-automatic photo cropping,” in Proc. SIGCHI Conf. Human Factors Comput. Syst., 2006, pp. 771–780.

[7]

M. Rubinstein, A. Shamir, and S. Avidan, “Improved seam carving for video retargeting,” ACM Trans. Graph., vol. 27, no. 3, Aug. 2008, Art. no.

[8]

M. Rubinstein, A. Shamir, and S. Avidan, “Multi-operator media retargeting,” ACM Trans. Graph., vol. 28, no. 3, 2009, Art. no.

[9]

D. Han, X. Wu, and M. Sonka, “Optimal multiple surfaces searching for video/image resizing—A graph-theoretic approach,” in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 1026–1033.

[10]

M. Grundmann, V. Kwatra, M. Han, and I. Essa, “Discontinuous seam-carving for video retargeting,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 569–576.

[11]

J. Sun and H. Ling, “Scale and object aware image retargeting for thumbnail browsing,” in Proc. IEEE ICCV, Nov. 2011, pp. 1511–1518.

[12]

L. Wolf, M. Guttmann, and D. Cohen-Or, “Non-homogeneous content-driven video-retargeting,” in Proc. IEEE 11th Int. Conf. Comput. Vis. (ICCV), Oct. 2007, pp. 1–6.

[13]

Y.-S. Wang, H.-C. Lin, O. Sorkine, and T.-Y. Lee, “Motion-based video retargeting with optimized crop-and-warp,” ACM Trans. Graph., vol. 29, no. 4, 2010, Art. no.

[14]

Y. Guo, F. Liu, J. Shi, Z.-H. Zhou, and M. Gleicher, “Image retargeting using mesh parametrization,” IEEE Trans. Multimedia, vol. 11, no. 5, pp. 856–867, Aug. 2009.

Digital Library

[15]

Y. Pritch, E. Kav-Venaki, and S. Peleg, “Shift-map image editing,” in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 151–158.

[16]

E. Kav-Venaki and S. Peleg, “Feedback retargeting,” in Proc. Media Retargeting Workshop ECCV, Crete, Greece, Sep. 2010, pp. 145–155.

[17]

S. Qi and J. Ho, “Seam segment carving: Retargeting images to irregularly-shaped image domains,” in Proc. 12th ECCV, Oct. 2012, pp. 314–326.

[18]

R. Sukthankar, R. G. Stockton, and M. D. Mullin, “Smarter presentations: Exploiting homography in camera-projector systems,” in Proc. 8th IEEE Int. Conf. Comput. Vis. (ICCV), vol. 1. 2001, pp. 247–253.

[19]

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2004.

[20]

M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Trans. Graph., vol. 29, no. 5, 2010, Art. no.

[21]

T. Liu et al., “Learning to detect a salient object,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353–367, Feb. 2011.

Digital Library

Cited By

Garg ASingh A(2022)Performance analysis of seam diversion based image retargeting technique based on edge detection operatorsMultimedia Tools and Applications10.1007/s11042-022-14157-482:15(23207-23250)Online publication date: 16-Nov-2022
https://dl.acm.org/doi/10.1007/s11042-022-14157-4
Garg ANayyar ASingh A(2022)Improved seam carving for structure preservation using efficient energy functionMultimedia Tools and Applications10.1007/s11042-022-12003-181:9(12883-12924)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s11042-022-12003-1
Tang ZYao JZhang Q(2022)Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contentsMultimedia Tools and Applications10.1007/s11042-021-11376-z81:1(1501-1522)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s11042-021-11376-z
Show More Cited By

Index Terms

CASAIR: Content and Shape-Aware Image Retargeting and Its Applications
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Computer graphics
    1. Image manipulation
    2. Shape modeling

Index terms have been assigned to the content through auto-classification.

Recommendations

Content-Aware Image Retargeting Using Line-Based MLS Deformation
LNCS on Transactions on Edutainment XIII - Volume 10092

Content-aware image retargeting, also known as seam carving or content aware scaling, is an algorithm for image resizing developed in recent years. In this paper, we present a content-aware image retargeting method by combining the line-based Moving ...
CMAIR: content and mask-aware image retargeting
Abstract
In recent years, more and more image retargeting techniques have been proposed to facilitate our daily life, in particular those based on the use of seam carving, warping or the combination of them. However, these techniques can only retarget the ...
Seam carving for content-aware image resizing
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports content-aware image resizing for both reduction and expansion. A seam is ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 25, Issue 5

May 2016

488 pages

ISSN:1057-7149

Issue’s Table of Contents

Copyright © 2016.

Publisher

IEEE Press

Publication History

Published: 01 May 2016

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Garg ASingh A(2022)Performance analysis of seam diversion based image retargeting technique based on edge detection operatorsMultimedia Tools and Applications10.1007/s11042-022-14157-482:15(23207-23250)Online publication date: 16-Nov-2022
https://dl.acm.org/doi/10.1007/s11042-022-14157-4
Garg ANayyar ASingh A(2022)Improved seam carving for structure preservation using efficient energy functionMultimedia Tools and Applications10.1007/s11042-022-12003-181:9(12883-12924)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s11042-022-12003-1
Tang ZYao JZhang Q(2022)Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contentsMultimedia Tools and Applications10.1007/s11042-021-11376-z81:1(1501-1522)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1007/s11042-021-11376-z
Zeng KHu JGong YWattanachote KYu RLuo X(2020)Vertical Retargeting for Stereoscopic Images via Stereo Seam CarvingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/340829516:4(1-22)Online publication date: 17-Dec-2020
https://dl.acm.org/doi/10.1145/3408295
Chang HShih TChang CTavanapong W(2019)CMAIR: content and mask-aware image retargetingMultimedia Tools and Applications10.1007/s11042-019-7462-278:15(21731-21758)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s11042-019-7462-2
Song YTang FDong WZhang XDeussen OLee TBoll SMu Lee KLuo JZhu WByun HWen Chen CLienhart RMei T(2018)Photo Squarization by Deep Multi-Operator RetargetingProceedings of the 26th ACM international conference on Multimedia10.1145/3240508.3240623(1047-1055)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3240508.3240623
Kiess JKopf SGuthier BEffelsberg W(2018)A Survey on Content-Aware Image and Video RetargetingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/323159814:3(1-28)Online publication date: 24-Jul-2018
https://dl.acm.org/doi/10.1145/3231598
Zhu BChen YWang JLiu SZhang BTang MLiu QLienhart RWang HChen SBoll SChen PFriedland GLi JYan S(2017)Fast Deep Matting for Portrait Animation on Mobile PhoneProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123286(297-305)Online publication date: 23-Oct-2017
https://dl.acm.org/doi/10.1145/3123266.3123286

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents