Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

CoDIQE3D: A completely blind, no-reference stereoscopic image quality estimator using joint color and depth statistics

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we present an unsupervised, completely blind, no-reference (NR) stereoscopic (S3D) image quality prediction model to assess the perceptual quality of natural S3D images. We study the joint dependencies between color and depth features of S3D images and empirically model these dependencies by using a bivariate generalized Gaussian distribution (BGGD). We compute the parameters of BGGD, and we also obtain the determinant and the coherence values from the covariance matrix of the proposed BGGD model. We extract the features of BGGD model and covariance matrix from the reference S3D image, followed by multivariate Gaussian (MVG) distribution modeling on the predicted features of the reference. We estimate the joint color and depth quality of the S3D images by computing the likelihood of the image features with respect to the reference MVG model. We apply the popular 2D unsupervised NIQE model on individual stereo views to estimate the overall spatial quality of the S3D images. Finally, we pool the likelihood scores and the spatial NIQE scores to achieve the estimation for the overall perceived quality of the S3D images. The performance of the proposed model is evaluated on the MICT, LIVE Phase I and II S3D image datasets. The results indicate consistent and robust performance for all datasets. Our proposed estimator is completely blind, as it requires neither training on subjective scores nor reference S3D images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Statista (Julia Stoll): Number of digital 3D cinema screens worldwide from 2006 to 2019. https://www.statista.com/statistics/271863/number-of-3d-cinema-screens-worldwide/ (2020)

  2. Jordan, J.R., III., Bovik, A.C.: Using chromatic information in edge-based stereo correspondence. CVGIP Image Underst. 54(1), 98–118 (1991)

    Article  MATH  Google Scholar 

  3. Jordan, J.R., III., Bovik, A.C.: Using chromatic information in dense stereo correspondence. Pattern Recogn. 25(4), 367–383 (1992)

    Article  Google Scholar 

  4. Su, C.-C., Cormack, L.K., Bovik, A.C.: Color and depth priors in natural images. IEEE Trans. Image Process. 22(6), 2259–2274 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Appina, B., Khan, S., Channappayya, S.S.: No-reference stereoscopic image quality assessment using natural scene statistics. Signal Process.: Image Commun. 43, 1–14 (2016)

    Google Scholar 

  6. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a completely blind image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2013)

    Article  Google Scholar 

  7. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  8. Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: Conference on Signals, Systems and Computers, vol. 2, pp. 1398–1402, IEEE (2003)

  9. Cai, R., Fang, M.: Blind image quality assessment by simulating the visual cortex. Vis. Comput. 1–18 (2022)

  10. Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24(8), 2579–2591 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Ji, J., Xiang, K., Wang, X.: SCVS: blind image quality assessment based on spatial correlation and visual saliency. Vis. Comput. 1, 23 (2022)

    Google Scholar 

  12. Joshi, P., Prakash, S., Rawat, S.: Continuous wavelet transform-based no-reference quality assessment of deblocked images. Vis. Comput. 34(12), 1739–1748 (2018)

    Article  Google Scholar 

  13. Campisi, P., Le Callet, P., Marini, E.: Stereoscopic images quality assessment. In: European Signal Processing Conference, pp. 2110–2114. IEEE (2007)

  14. Gorley, P., Holliman, N.: Stereoscopic image quality metrics and compression. In: Electronic Imaging, pp. 45–56. International Society for Optics and Photonics (2008)

  15. Chen, M.-J., Su, C.-C., Kwon, D.-K., Cormack, L.K., Bovik, A.C.: Full-reference quality assessment of stereopairs accounting for rivalry. Signal Process.: Image Commun. 28(9), 1143–1155 (2013)

    Google Scholar 

  16. Benoit, A., Le Callet, P., Campisi, P., Cousseau, R.: Quality assessment of stereoscopic images. EURASIP J. Image Video Process. 2008, 1–13 (2009)

    Article  Google Scholar 

  17. Khan, M.S., Channappayya, S.S.: Sparsity based stereoscopic image quality assessment. In: Asilomar Conference on Signals, Systems and Computers, pp. 1858–1862. IEEE (2016)

  18. Khan, Md.S., Appina, B., Channappayya, S.: Full-reference stereo image quality assessment using natural stereo scene statistics. IEEE Signal Process. Lett. 22, 1985–1989 (2015)

    Article  Google Scholar 

  19. Khan, S., Channappayya, S.S.: Estimating depth-salient edges and its application to stereoscopic image quality assessment. IEEE Trans. Image Process. 27(12), 5892–5903 (2018)

    Article  MathSciNet  Google Scholar 

  20. Bensalma, R., Larabi, M.-C.: A perceptual metric for stereoscopic image quality assessment based on the binocular energy. Multidimens. Syst. Signal Process. 24(2), 281–316 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Bensalma, R., Larabi, M.C.: Towards a perceptual quality metric for color stereo images. In International Conference on Image Processing, pp. 4037–4040. IEEE (2010)

  22. Jiang, Q., Zhou, W., Chai, X., Yue, G., Shao, F., Chen, Z.: A full-reference stereoscopic image quality measurement via hierarchical deep feature degradation fusion. IEEE Trans. Instrum. Meas. 69(12), 9784–9796 (2020)

    Article  Google Scholar 

  23. Akhter, R., Sazzad, Z.P., Horita, Y., Baltes, J.: No-reference stereoscopic image quality assessment. In: IS &T/SPIE Electronic Imaging, pp. 271–282. International Society for Optics and Photonics (2010)

  24. Sazzad, Z.P., Yamanaka, S., Kawayokeita, Y., Horita, Y.: Stereoscopic image quality prediction. In: International Workshop on Quality of Multimedia Experience, pp. 180–185, IEEE (2009)

  25. Ryu, S., Sohn, K.: No-reference quality assessment for stereoscopic images based on binocular quality perception. IEEE Trans. Circuits Syst. Video Technol. 24(4), 591–602 (2013)

    Google Scholar 

  26. Appina, B.: A ‘Complete Blind’ No-Reference Stereoscopic Image Quality Assessment Algorithm. In: International Conference on Signal Processing and Communications (SPCOM), pp. 1–5. IEEE (2020)

  27. Shao, F., Lin, W., Wang, S., Jiang, G., Yu, M.: Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook. IEEE Trans. Broadcast. 61, 154–165 (2015)

    Article  Google Scholar 

  28. Jiang, Q., Duan, F., Shao, F.: 3D visual attention for stereoscopic image quality assessment. J. Softw. 9(7), 1841–1847 (2014)

    Article  Google Scholar 

  29. Chen, M.-J., Cormack, L.K., Bovik, A.C.: No-reference quality assessment of natural stereopairs. IEEE Trans. Image Process. 22(9), 3379–3391 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  30. Zhou, W., Yu, L., Zhou, Y., Qiu, W., Wu, M.-W., Luo, T.: Blind quality estimator for 3D images based on binocular combination and extreme learning machine. Pattern Recogn. 71, 207–217 (2017)

    Article  Google Scholar 

  31. Su, C.-C., Cormack, L.K., Bovik, A.C.: Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Trans. Image Process. 24(5), 1685–1699 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  32. Shi, Y., Guo, W., Niu, Y., Zhan, J.: No-reference stereoscopic image quality assessment using a multi-task CNN and registered distortion representation. Pattern Recogn. 100, 1–12 (2020)

    Article  Google Scholar 

  33. Zhang, W., Qu, C., Ma, L., Guan, J., Huang, R.: Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network. Pattern Recogn. 59, 176–187 (2016)

    Article  Google Scholar 

  34. Liu, Y., Yan, W., Zheng, Z., Huang, B., Yu, H.: Blind stereoscopic image quality assessment accounting for human monocular visual properties and binocular interactions. IEEE Access 8, 33666–33678 (2020)

    Article  Google Scholar 

  35. Yildiz, Z.C., Oztireli, A.C., Capin, T.: A machine learning framework for full-reference 3D shape quality assessment. Vis. Comput. 36(1), 127–139 (2020)

    Article  Google Scholar 

  36. Den Ouden, H., Van Ee, R., De Haan, E.: Colour helps to solve the binocular matching problem. J. Physiol. 567(2), 665–671 (2005)

    Article  Google Scholar 

  37. Daniel, Y., Roe, A.W., Gilbert, C.D.: A hierarchy of the functional organization for color, form and disparity in primate visual area v2. Vision. Res. 41(10–11), 1333–1349 (2001)

    Google Scholar 

  38. Nasr, S., Polimeni, J.R., Tootell, R.B.: Interdigitated color-and disparity-selective columns within human visual cortical areas v2 and v3. J. Neurosci. 36(6), 1841–1857 (2016)

    Article  Google Scholar 

  39. Fine, I., MacLeod, D., Boynton, G.M.: Surface segmentation based on the luminance and color statistics of natural scenes. J. Vis. 2(10), 66–66 (2002)

    Article  Google Scholar 

  40. Chen, M., Bovik, A.C., Cormack, L.K.: Study on distortion conspicuity in stereoscopically viewed 3D images. In: IEEE 10th IVMSPWorkshop: Perception and Visual Signal Analysis, pp. 24–29 (2011)

  41. Su, C.C., Bovik, A.C., Cormack, L. K.: Statistical model of color and disparity with application to Bayesian stereopsis. In: Southwest Symposium on Image Analysis and Interpretation, pp. 169–172. IEEE (2012)

  42. Su, C.C., Bovik, A.C., Cormack, L.K.: Natural scene statistics of color and range. In: International Conference on Image Processing, pp. 257–260, IEEE (2011)

  43. Su, C.-C., Cormack, L.K., Bovik, A.C.: Bivariate statistical modeling of color and range in natural scenes. In: IS &T/SPIE Electronic Imaging, International Society for Optics and Photonics, pp. 391–400 (2014)

  44. Pascal, F., Bombrun, L., Tourneret, J.-Y., Berthoumieu, Y.: Parameter estimation for multivariate generalized Gaussian distributions. IEEE Trans. Signal Process. 61(23), 5960–5971 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  45. Saad, M.A., Bovik, A.C., Charrier, C.: Blind prediction of natural video quality. IEEE Trans. Image Process. 23(3), 1352–1365 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  46. Simoncelli, E.P., Freeman, W.T.: The steerable pyramid: a flexible architecture for multi-scale derivative computation. In: Proceedings., International Conference on Image Processing, vol. 3, pp. 444–447. IEEE (1995)

  47. Palm, C., Keysers, D., Lehmann, T., Spitzer, K.: Gabor filtering of complex hue/saturation images for color texture classification. In: Proceeding of the JCIS, pp. 45–49, Citeseer (2000)

  48. Jung, Y.J., Sohn, H., Lee, S.-I., Park, H.W., Ro, Y.M.: Predicting visual discomfort of stereoscopic images using human attention model. IEEE Trans. Circuits Syst. Video Technol. 23(12), 2077–2082 (2013)

    Article  Google Scholar 

  49. Moorthy, A.K., Su, C.-C., Mittal, A., Bovik, A.C.: Subjective evaluation of stereoscopic image quality. Signal Process. Image Commun. 28(8), 870–883 (2013)

    Article  Google Scholar 

  50. Wang, J., Rehman, A., Zeng, K., Wang, S.,Wang, Z.: Quality prediction of asymmetrically distorted stereoscopic 3D images. IEEE Trans. Image Process. 24(11), 3400–3414 (2015)

  51. VQEG Final Report From the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment, Phase II. [online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv-phase-ii/frtv-phase-ii.aspx (2003)

  52. Testolina, P., Barbato, F., Michieli, U., Giordani, M., Zanuttigh, P., Zorzi, M.: SELMA: SEmantic large-scale multimodal acquisitions in variable weather, daytime and viewpoints. arXiv preprint arXiv:2204.09788 (2022)

  53. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Conference on Robot Learning, pp. 1–16. PMLR (2017)

  54. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)

    MATH  Google Scholar 

Download references

Acknowledgements

The research reported in this paper was supported in part by the Department of Science and Technology - Science and Engineering Research Board, Government of India under Grant SRG/2020/000336. The work was also supported by project no. BME-NVA-02, implemented with the support provided by the Ministry of Innovation and Technology of Hungary from the National Research, Development and Innovation Fund, financed under the TKP2021 funding scheme.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Indian Institute of Information Technology, Design and Manufacturing, Kancheepuram, Chennai, India

    Ajay Kumar Reddy Poreddy

  2. Department of Networked Systems and Services, Budapest University of Technology and Economics, Budapest, Hungary

    Peter A. Kara

  3. Department of Electrical Engineering, Indian Institute of Technology Hyderabad, Kandi, India

    Roopak R. Tamboli

  4. Sigma Technology, Budapest, Hungary

    Aniko Simon

  5. Department of Electrical Engineering, Indian Institute of Technology Indore, Simrol, Madhya Pradesh, India

    Balasubramanyam Appina

Authors
  1. Ajay Kumar Reddy Poreddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter A. Kara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roopak R. Tamboli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aniko Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Balasubramanyam Appina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Balasubramanyam Appina.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Poreddy, A.K.R., Kara, P.A., Tamboli, R.R. et al. CoDIQE3D: A completely blind, no-reference stereoscopic image quality estimator using joint color and depth statistics. Vis Comput 39, 6743–6753 (2023). https://doi.org/10.1007/s00371-022-02760-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02760-3

Keywords

Search

Navigation