Deep stereoscopic image saliency inspired stereoscopic image thumbnail generation
Pages 42749 - 42767
Abstract
In this paper, we propose a stereoscopic image thumbnail generation method guided by the stereoscopic image saliency. Specifically, we utilize an uncertain-weighted fusion mechanism to combine the spatial saliency information with the saliency driven by depth cues, generating the dense stereoscopic saliency fixation map. Subsequently, the obtained dense fixation map is converted into a salient object map through a saliency optimization module, which provides the object-level saliency cues for the thumbnail generation task. Under the guidance of the obtained salient object map, a cropping window is employed to cut out the most salient region and generate the stereoscopic thumbnails, such that the disparity distribution of the original image can be well preserved, and avoid sharply deforming certain structured objects in the subsequent warping operation. Finally, the warping operation is utilized to adjust the aspect ratio of the stereoscopic thumbnail to the target size. Qualitative and quantitative results demonstrate that our proposed method achieves superior performance than the state-of-the-art benchmarks on the public datasets.
References
[1]
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, and Süsstrunk S Slic superpixels compared to state-of-the-art superpixel methods IEEE Trans Pattern Anal Mach Intell 2012 34 11 2274-2282
[2]
Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM SIGGRAPH 2007, pp 10–es
[3]
B AKA, B MSA, C KD, and D RGAB Contextual encoder-decoder network for visual saliency prediction Neural Netw 2020 129 261-270
[4]
Banitalebi-Dehkordi A, Pourazad MT, and Nasiopoulos P A learning-based visual saliency prediction model for stereoscopic 3D video (LBVS-3D) Multimed Tools Appl 2017 76 23859-23890
[5]
Chai X, Shao F, Jiang Q, and Ho YS MSTGAR: multioperator-based stereoscopic thumbnail generation with arbitrary resolution IEEE Trans Multimed 2019 22 5 1208-1219
[6]
Chang CH, Liang CK, and Chuang YY Content-aware display adaptation and interactive editing for stereoscopic images IEEE Trans Multimed 2011 13 4 589-601
[7]
Che Z, Borji A, Zhai G, Min X, Guo G, and Callet PL How is gaze influenced by image transformations? Dataset and model IEEE Trans Image Process 2020 29 2287-2300
[8]
Chen J, Bai G, Liang S, Li Z Automatic image cropping: a computational complexity study (2016)
[9]
Cho D, Park J, Oh TH, Tai YW, So Kweon I (2017) Weakly-and self-supervised learning for content-aware deep image retargeting. In: Proceedings of the IEEE international conference on computer vision, pp 4558–4567
[10]
Cong R, Lei J, Zhang C, Huang Q, Cao X, and Hou C Saliency detection for stereoscopic images based on depth confidence analysis and multiple cues fusion IEEE Signal Process Lett 2016 23 6 819-823
[11]
Dong WM, Bao GB, Zhang X, and Paul JC Fast multi-operator image resizing and evaluation J Comput Sci Technol 2012 27 1 12-134
[12]
Fan S, Shen Z, Jiang M, Koenig BL, Xu J, Kankanhalli MS, Zhao Q (2018) Emotional attention: a study of image sentiment and visual attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7521–7531
[13]
Fan X, Lei J, Liang J, Fang Y, Cao X, and Ling N Unsupervised stereoscopic image retargeting via view synthesis and stereo cycle consistency losses Neurocomputing 2021 447 161-171
[14]
Fang Y, Wang J, Narwaria M, Le Callet P, and Lin W Saliency detection for stereoscopic images IEEE Trans Image Process 2014 23 6 2625-2636
[15]
Fang Y, Wang Z, Lin W, and Fang Z Video saliency incorporating spatiotemporal cues and uncertainty weighting IEEE Trans Image Process 2014 23 9 3910-3921
[16]
Fu Z, Shao F, Jiang Q, Chao M, Ho YS (2020) Subjective and objective quality assessment for stereoscopic 3d image retargeting. IEEE Transactions on Multimedia
[17]
Harel J, Koch C, and Perona P Graph-based visual saliency Adv Neural Inform Process Syst 2006 19 545-552
[18]
Huang X, Shen C, Boix X, Zhao Q (2015) SALICON: reducing the semantic gap in saliency prediction by adapting deep neural networks pp 262–270 (2015). IEEE International Conference on Computer Vision, Santiago, pp 11–18.
[19]
Itti L, Koch C, and Niebur E A model of saliency-based visual attention for rapid scene analysis IEEE Trans Pattern Anal Mach Intell 1998 20 11 1254-1259
[20]
Jiang Q, Shao F, Jiang G, Yu M, Peng Z, and Yu C A depth perception and visual comfort guided computational model for stereoscopic 3d visual saliency Signal Process: Image Commun 2015 38 57-69
[21]
Jung YJ, Kim H, and Ro YM Critical binocular asymmetry measure for the perceptual quality assessment of synthesized stereo 3d images in view synthesis IEEE Trans Circuits Syst Video Technol 2016 26 7 1201-1214
[22]
Kiess J, Guthier B, Kopf S, Effelsberg W (2012) Seamcrop for image retargeting. In: Multimedia on mobile devices 2012; and multimedia content access: algorithms and systems VI, vol 8304, p 83040K
[23]
Lowe DG Distinctive image features from scale-invariant keypoints Int J Comput Vis 2004 60 2 91-110
[24]
Mao Y, Jiang Q, Cong R, Gao W, Shao F, Kwong S (2021) Cross-modality fusion and progressive integration network for saliency prediction on stereoscopic 3d images. IEEE Trans Multimedia, 1–1.
[25]
Nguyen AD, Kim J, Oh H, Kim H, Lin W, and Lee S Deep visual saliency on stereoscopic images IEEE Trans Image Process 2018 28 4 1939-1953
[26]
Niu Y, Chen J, Ke X, and Chen J Stereoscopic image saliency detection optimization: a multi-cue-driven approach IEEE Access 2019 PP 99 1-1
[27]
Rubinstein M, Shamir A, and Avidan S Multi-operator media retargeting ACM Trans Graph (TOG) 2009 28 3 1-11
[28]
Shao F, Lin W, Lin W, Jiang Q, and Jiang G QoE-guided warping for stereoscopic image retargeting IEEE Trans Image Process 2017 26 10 4790-4805
[29]
Suh B, Ling H, Bederson BB, Jacobs DW (2003) Automatic thumbnail cropping and its effectiveness. In: Proceedings of the 16th annual ACM symposium on user interface software and technology, pp 95–104
[30]
Sun J and Ling H Scale and object aware image thumbnailing Int J Comput Vis 2013 104 2 135-153
[31]
Tan W, Yan B, Lin C, and Niu X Cycle-ir: deep cyclic image retargeting IEEE Trans Multimed 2019 22 7 1730-1743
[32]
Wang YS, Tai CL, Sorkine O, and Lee TY Optimized scale-and-stretch for image resizing ACM Trans Graph 2008 27 5 118
[33]
Wang W, Shen J, Yu Y, and Ma KL Stereoscopic thumbnail creation via efficient stereo saliency detection IEEE Trans Vis Comput Graph 2016 23 8 2014-2027
[34]
Wang X, Shao F, Jiang Q, Fu Z, Chao M, Gu K, Ho YS (2021) Combining retargeting quality and depth perception measures for quality evaluation of retargeted stereopairs. IEEE Trans Multimedia, 1–1.
[35]
Wu L, Cao L, Xu M, and Wang J A hybrid image retargeting approach via combining seam carving and grid warping J Multimed 2014 9 4 483
[36]
Wang W, Shen J, Dong X, Borji A, and Yang R Inferring salient objects from human fixations IEEE Trans Pattern Anal Mach Intell 2019 42 8 1913-1927
[37]
Zhang Q, Wang X, Jiang J, Ma L (2016) Deep learning features inspired saliency detection of 3d images. In: Pacific rim conference on multimedia, pp 580–589. Springer
[38]
Zhang Y, Fang Y, Lin W, Zhang X, Li L (2016) Backward registration-based aspect ratio similarity for image retargeting quality assessment. IEEE Trans Image Process, 1–1
[39]
Zhang L, Li K, Ou Z, and Wang F Seam warping: a new approach for image retargeting for small displays Soft Comput 2017 21 2 447-457
[40]
Zhang Q, Wang X, Wang S, Sun Z, and Jiang J Learning to explore saliency for stereoscopic videos via component-based interaction IEEE Trans Image Process 2020 29 99 5722-5736
[41]
Zhou Y, Chen Z, and Li W Weakly supervised reinforced multi-operator image retargeting IEEE Trans Circuits Syst Video Technol 2020 31 1 126-139
Index Terms
- Deep stereoscopic image saliency inspired stereoscopic image thumbnail generation
Index terms have been assigned to the content through auto-classification.
Recommendations
Visual comfort prediction for stereoscopic image using stereoscopic visual saliency
Perceptually salient regions of stereoscopic images significantly affect visual comfort (VC). In this paper, we propose a new objective approach for predicting VC of stereoscopic images according to visual saliency. The proposed approach includes two ...
Saliency-based stereoscopic image retargeting
In this paper, we propose a new saliency based stereoscopic image retargeting method based on the characteristics of the Human Visual System (HVS). A new stereoscopic saliency detection method is designed by adopting low-level features of intensity, ...
Learning visual saliency from human fixations for stereoscopic images
In the previous years, a lot of saliency detection algorithms have been designed for saliency computation of visual content. Recently, stereoscopic display techniques have developed rapidly, which results in much requirement of stereoscopic saliency ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 December 2022
Accepted: 13 July 2022
Revision received: 27 September 2021
Received: 30 June 2021
Author Tags
Qualifiers
- Research-article
Funding Sources
- national natural science foundation of china
- shenzhen fundamental research program
- shenzhen fundamental research program
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Jan 2025