Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Local and global sparse representation for no-reference quality assessment of stereoscopic images

Published: 01 January 2018 Publication History

Abstract

No-reference/blind quality assessment of stereoscopic 3D images is much more challenging than 2D images due to the poor understanding of binocular vision. In this paper, we propose a BLind Quality Evaluator for stereoscopic 3D images by learning Local and Global Sparse Representations (BLQELGSR). Specifically, at the training stage, we first construct a large-scale training set by simulating some common distortions that are likely encountered by stereoscopic images, and propose a multi-modal sparse representation framework to characterize the relationship between the feature and quality spaces for all sources of information from left, right and cyclopean views in local and global manners. At the testing stage, based on the derived 3D quality prediction framework, the local and global quality scores from different sources are predicted and combined to drive a final 3D quality score. Experimental results on three 3D image quality databases show that in comparison with the existing methods, the devised BLQELGSR can achieve better prediction performance to be in line with subjective assessment.

References

[1]
B. Appina, S. Khan, S.S. Channappayya, No-reference stereoscopic image quality assessment using natural scene statistics, Signal Process. Image Commun., 43 (2016) 1-14.
[2]
S. Bahrampour, N.M. Nasrabadi, A. Ray, W.K. Jenkins, Multimodal task-driven dictionary learning for image classification, IEEE Trans. Image Process., 25 (2016) 24-38.
[3]
A. Benoit, P. Le Callet, P. Campisi, R. Cousseau, Quality assessment of stereoscopic images, EURASIP J. Image Video Process., 2008 (2009) 659024.
[4]
R. Bensalma, M.-C. Larabi, A perceptual metric for stereoscopic image quality assessment based on the binocular energy, Multidimens. Syst. Signal Process. (2013) 1-36.
[5]
M.-J. Chen, L.K. Cormack, A.C. Bovik, No-reference quality assessment of natural stereopairs, IEEE Trans. Image Process., 22 (2013) 3379-3391.
[6]
M.-J. Chen, C.-C. Su, D.-K. Kwon, L.K. Cormack, A.C. Bovik, Full-reference quality assessment of stereopairs accounting for rivalry, Signal Process. Image Commun., 28 (2013) 1143-1155.
[7]
K. Gu, G. Zhai, X. Yang, W. Zhang, No-reference stereoscopic IQA approach: from nonlinear effect to parallax compensation, J. Electr. Comput. Eng., 2012 (2012).
[8]
K. Gu, G. Zhai, X. Yang, W. Zhang, Using free energy principle for blind image quality assessment, IEEE Trans. Multimedia, 17 (2015) 50-63.
[9]
L. He, D. Tao, X. Li, X. Gao, Sparse representation for blind image quality assessment, IEEE, 2012.
[10]
S. Hochstein, M. Ahissar, View from the top: hierarchies and reverse hierarchies in the visual system, Neuron, 36 (2002) 791-804.
[11]
Q. Jiang, F. Shao, G. Jiang, M. Yu, Z. Peng, Supervised dictionary learning for blind image quality assessment using quality-constraint sparse coding, J. Vis. Commun. Image Represent., 33 (2015) 123-133.
[12]
Q. Jiang, F. Shao, W. Lin, G. Jiang, On predicting visual comfort of stereoscopic images: a learning to rank based approach, IEEE Signal Process. Lett., 23 (2016) 302-306.
[13]
Z. Jiang, Z. Lin, L.S. Davis, Label consistent K-SVD: learning a discriminative dictionary for recognition, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013) 2651-2664.
[14]
L. Kang, P. Ye, Y. Li, D. Doermann, Convolutional neural networks for no-reference image quality assessment, 2014.
[15]
P. Le Callet, S. Mller, A. Perkis, Qualinet white paper on definitions of quality of experience, 2012.
[16]
P. Lebreton, A. Raake, M. Barkowsky, P. Le Callet, Evaluating depth perception of 3D stereoscopic videos, IEEE J. Sel. Top. Signal Process., 6 (2012) 710-720.
[17]
K. Lee, S. Lee, 3D perception based quality pooling: stereopsis, binocular rivalry, and binocular suppression, IEEE J. Sel. Top. Signal Process., 9 (2015) 533-545.
[18]
Y.-H. Lin, J.-L. Wu, Quality assessment of stereoscopic 3D image compression by binocular integration behaviors, IEEE Tans. Image Process., 23 (2014) 1527-1542.
[19]
P. Marziliano, F. Dufaux, S. Winkler, T. Ebrahimi, A no-reference perceptual blur metric, IEEE, 2002.
[20]
A. Mittal, A.K. Moorthy, A.C. Bovik, No-reference image quality assessment in the spatial domain, IEEE Trans. Image Process., 21 (2012) 4695-4708.
[21]
A. Mittal, R. Soundararajan, A.C. Bovik, Making a completely blind image quality analyzer, IEEE Signal Process. Lett., 20 (2013) 209-212.
[22]
D.C. Mocanu, G. Exarchakos, A. Liotta, Deep learning for objective quality assessment of 3D images, IEEE, 2014.
[23]
A.K. Moorthy, A.C. Bovik, Blind image quality assessment: from natural scene statistics to perceptual quality, IEEE Trans. Image Process., 20 (2011) 3350-3364.
[24]
A.K. Moorthy, C.-C. Su, A. Mittal, A.C. Bovik, Subjective evaluation of stereoscopic image quality, Signal Process. Image Commun., 28 (2013) 870-883.
[25]
T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell., 24 (2002) 971-987.
[26]
J. Park, H. Oh, S. Lee, A.C. Bovik, 3D visual discomfort predictor: analysis of disparity and neural activity statistics, IEEE Trans. Image Process., 24 (2015) 1101-1114.
[27]
S. Ryu, K. Sohn, No-reference quality assessment for stereoscopic images based on binocular quality perception, IEEE Trans. Circuits Syst. Video Technol., 24 (2014) 591-602.
[28]
M.A. Saad, A.C. Bovik, C. Charrier, Blind image quality assessment: a natural scene statistics approach in the DCT domain, IEEE Trans. Image Process., 21 (2012) 3339-3352.
[29]
Z. Sazzad, R. Akhter, J. Baltes, Y. Horita, Objective no-reference stereoscopic image quality prediction based on 2D image features and relative disparity, Adv. Multimedia, 2012 (2012) 8.
[30]
F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, Q. Dai, Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties, IEEE Trans. Image Process., 24 (2015) 2971-2983.
[31]
F. Shao, W. Lin, S. Gu, G. Jiang, T. Srikanthan, Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics, IEEE Trans. Image Process., 22 (2013) 1940-1953.
[32]
F. Shao, W. Lin, S. Wang, G. Jiang, M. Yu, Q. Dai, Learning receptive fields and quality lookups for blind quality assessment of stereoscopic images, IEEE Trans. Cybern., 46 (2016) 730-743.
[33]
F. Shao, W. Tian, W. Lin, G. Jiang, Q. Dai, Toward a blind deep quality evaluator for stereoscopic images based on monocular and binocular interactions, IEEE Trans. Image Process., 25 (2016) 2059-2074.
[34]
H.R. Sheikh, A.C. Bovik, Image information and visual quality, IEEE Trans. Image Process., 15 (2006) 430-444.
[35]
S. Shekhar, V.M. Patel, N.M. Nasrabadi, R. Chellappa, Joint sparse representation for robust multimodal biometrics recognition, IEEE Trans. Pattern Anal. Mach. Intell., 36 (2014) 113-126.
[36]
C.-C. Su, L.K. Cormack, A.C. Bovik, Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation, IEEE Trans. Image Process., 24 (2015) 1685-1699.
[37]
H. Tang, N. Joshi, A. Kapoor, Blind image quality assessment using semi-supervised rectifier networks, 2014.
[38]
P.V. Vu, D.M. Chandler, A fast wavelet-based algorithm for global and local image sharpness estimation, IEEE Signal Process. Lett., 19 (2012) 423-426.
[39]
J. Wang, A. Rehman, K. Zeng, S. Wang, Z. Wang, Quality prediction of asymmetrically distorted stereoscopic 3D images, IEEE Trans. Image Process., 24 (2015) 3400-3414.
[40]
Z. Wang, A. Bovik, A universal image quality index, IEEE Signal Process Lett., 9 (2002) 81-84.
[41]
Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004) 600-612.
[42]
Z. Wang, H.R. Sheikh, A.C. Bovik, No-reference perceptual quality assessment of JPEG compressed images, IEEE, 2002.
[43]
W. Xue, X. Mou, L. Zhang, A.C. Bovik, X. Feng, Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features, IEEE Trans. Image Process., 23 (2014) 4850-4862.
[44]
W. Xue, L. Zhang, X. Mou, Learning without human scores for blind image quality assessment, 2013.
[45]
J. Yang, Y. Wang, B. Li, W. Lu, Q. Meng, Z. Lv, D. Zhao, Z. Gao, Quality assessment metric of stereo images considering cyclopean integration and visual saliency, Inf. Sci. (NY), 373 (2016) 251-268.
[46]
G.N. Yilmaz, A no reference depth perception assessment metric for 3D video, Multimed. Tools Appl., 74 (2015) 6937-6950.
[47]
L. Zhang, L. Zhang, A.C. Bovik, A feature-enriched completely blind image quality evaluator, IEEE Trans. Image Process., 24 (2015) 2579-2591.
[48]
L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: a feature similarity index for image quality assessment, IEEE Trans. Image Process., 20 (2011) 2378-2386.
[49]
W. Zhang, C. Qu, L. Ma, J. Guan, R. Huang, Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network, Pattern Recognit., 59 (2016) 176-187.
[50]
Y. Zhang, D.M. Chandler, 3D-MAD: a full reference stereoscopic image quality estimator based on binocular lightness and contrast perception, IEEE Trans. Image Process., 24 (2015) 3810-3825.

Cited By

View all
  • (2021)An Adaptive Smoothness Parameter Strategy for Variational Optical Flow ModelScientific Programming10.1155/2021/75946362021Online publication date: 1-Jan-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Information Sciences: an International Journal
Information Sciences: an International Journal  Volume 422, Issue C
January 2018
543 pages

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 January 2018

Author Tags

  1. Multi-modal sparse representation
  2. No-reference quality assessment
  3. Stereoscopic 3D image

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2021)An Adaptive Smoothness Parameter Strategy for Variational Optical Flow ModelScientific Programming10.1155/2021/75946362021Online publication date: 1-Jan-2021

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media