research-article

VR IQA NET: Deep Virtual Reality Image Quality Assessment Using Adversarial Learning

Authors:

Heaun-Taek Lim,

Yang Man RaAuthors Info & Claims

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Pages 6737 - 6741

https://doi.org/10.1109/ICASSP.2018.8461317

Published: 15 April 2018 Publication History

Abstract

In this paper, we propose a novel virtual reality image quality assessment (VR IQA) with adversarial learning for omnidirectional images. To take into account the characteristics of the omnidirectional image, we devise deep networks including novel quality score predictor and human perception guider. The proposed quality score predictor automatically predicts the quality score of distorted image using the latent spatial and position feature. The proposed human perception guider criticizes the predicted quality score of the predictor with the human perceptual score using adversarial learning. For evaluation, we conducted extensive subjective experiments with omnidirectional image dataset. Experimental results show that the proposed VR IQA metric outperforms the 2-D IQA and the state-of-the-arts VR IQA.

5. References

[1]

E. Upenik, M. Rerabek, and T. Ebrahimi, “On the performance of objective metrics for omnidirectional visual content,” International Conference on Quality of Multimedia Experience, June 2017.

[2]

K. T. Ng, S. C. Chan, and H. Y. Shum, “Data compression and transmission aspects of panoramic videos,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 1, pp. 82–95, Jan. 2005.

[3]

C. Grunheit, A. Smolic, and T. Wiegand, “Efficient representation and interactive streaming of high-resolution panoramic views,” IEEE International Conference on Image Processing, Sep. 2002.

[4]

P. R. Alface, Patrice, J. F. Macq, and N. Verzijp, “Interactive Omnidirectional Video Delivery: A Bandwidth-Effective Approach,” Bell Labs Technical Journal, vol. 16, no. 4, pp. 135–147, March 2012.

[5]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.

Digital Library

[6]

Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” Asilomar Conference on Signals, Systems and Computers, vol. 2, pp. 1398–1402, Nov. 2003.

[7]

H. R. Sheikh, and A. C. Bovik, “Image information and visual quality,” IEEE Transactions on image processing, vol. 15, no. 2, pp. 430–444, Jan. 2006.

Digital Library

[8]

Y. Matt, H. Lakshman, and B. Girod, “A framework to evaluate omnidirectional video coding schemes,” IEEE International Symposium on Mixed and Augmented Reality, Sep. 2015.

[9]

S. Yule, A. Lu, and Y. Lu, “Weighted-to-Spherically-Uniform Quality Evaluation for Omnidirectional Video,” IEEE signal processing letters, vol. 24, no. 9, pp. 1408–1412, June 2017.

[10]

V. Zakharchenko, K. P. Choi, and J. H. Park, “Quality metric for spherical panoramic video,” Proc. SPIE 9970, Optics and Photonics for Information Processing X, pp. 99700C, Sep. 2016.

[11]

S. Bosse, D. Maniry, K. R. Müller, T. Wiegand, and W. Samek, “Neural network-based full-reference image quality assessment,” Picture Coding Symposium, Dec. 2016.

[12]

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in Neural Information Processing Systems, 2014.

Digital Library

[13]

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” The IEEE Conference on Computer Vision and Pattern Recognition, June 2016.

[14]

J. Xiao, K. A. Ehinger, A. Oliva, and A. Torralba, “Recognizing Scene Viewpoint using Panoramic Place Representation,” The IEEE Conference on Computer Vision and Pattern Recognition, June 2012.

[15]

G. K. Wallace, “The JPEG still picture compression standard,” IEEE Transactions on Consumer Electronics, vol. 38, no. 1, pp. 18–34, Feb. 1992.

Digital Library

[16]

A. Skodras, C. Christopoulos, and T. Ebrahimi, “The JPEG 2000 still image compression standard,” IEEE Signal processing magazine, vol. 18, no. 5, pp. 36–58, Sep. 2001.

[17]

G. J. Sullivan, J. Ohm, W. J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Transactions on circuits and systems for video technology, vol. 22, no. 12, pp. 1649–1668, Sep. 2012.

Digital Library

[18]

https://ffmpeg.org/.

[19]

M. Abadi, et al., “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” 2016, Software available from tensorflow.org.

[20]

Methodology for the Subjective Assessment of the Quality of Television Pictures, document Rec. ITU-R BT.500–13, Jan. 2012.

[21]

Subjective Methods for the Assessment of Stereoscopic 3DTV Systems, document ITU-R BT.2021, 2012.

[22]

J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmilleret, “Striving for simplicity: The all convolutional net,” International Conference on Learning Representations, May 2015.

Cited By

Chen JNiu Y(2024)Two-stream network with viewport selection for blind omnidirectional video quality assessmentMultimedia Tools and Applications10.1007/s11042-023-15739-683:4(12139-12157)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15739-6
Hirzle TMüller FDraxler FSchmitz MKnierim PHornbæk K(2023)When XR and AI Meet - A Scoping Review on Extended Reality and Artificial IntelligenceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581072(1-45)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581072
Fu JHou CZhou WXu JChen ZMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Adaptive Hypergraph Convolutional Network for No-Reference 360-degree Image Quality AssessmentProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548337(961-969)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548337

Index Terms

VR IQA NET: Deep Virtual Reality Image Quality Assessment Using Adversarial Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Index terms have been assigned to the content through auto-classification.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Apr 2018

17916 pages

Copyright © 2018.

Publisher

IEEE Press

Publication History

Published: 15 April 2018

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen JNiu Y(2024)Two-stream network with viewport selection for blind omnidirectional video quality assessmentMultimedia Tools and Applications10.1007/s11042-023-15739-683:4(12139-12157)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11042-023-15739-6
Hirzle TMüller FDraxler FSchmitz MKnierim PHornbæk K(2023)When XR and AI Meet - A Scoping Review on Extended Reality and Artificial IntelligenceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581072(1-45)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581072
Fu JHou CZhou WXu JChen ZMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Adaptive Hypergraph Convolutional Network for No-Reference 360-degree Image Quality AssessmentProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548337(961-969)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548337

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents