tutorial

Automation of Subjective Video Quality Measurements

Authors:

Juan Pablo GarellaAuthors Info & Claims

LANC '14: Proceedings of the Latin America Networking Conference on LANC 2014

Article No.: 7, Pages 1 - 5

https://doi.org/10.1145/2684083.2684090

Published: 18 September 2014 Publication History

Get Access

Abstract

Quality of Experience is utterly important in Communication Systems, especially in audiovisual services. Audio and video quality, as perceived by the users, is one of the most important aspects that affect the Quality of Experience. The most accurate methods for measuring the perceived audio and video quality of a video clip are the subjective tests, where the video sequences are presented to different viewers, and opinions are averaged. Subjective measurements are also necessary to develop and calibrate objective models for video quality estimation. Different methods for performing these tests are standardized, and normally require many manual setups, data-entry and/or special hardware devices. This paper presents the design, development and deployment of a software based system to automate the subjective tests for video quality measurement. The application is available for download and use for the academic community.

References

[1]

Reiter, U. "Quality of experience: a buzzword or the key to successful multimedia delivery across networks?", Proceedings of the 6th Latin America Networking Conference, October 2011(pp. 20--24). ACM.

Digital Library

Google Scholar

[2]

J. Joskowicz, R. Sotelo y J. Ardao, "Towards a General Parametric Model for Perceptual Video Quality Estimation", IEEE Transactions on Broadcasting, vol. 59, n° 4, pp. 569--579, 2013.

Crossref

Google Scholar

[3]

Casadesus, L., Fernández-Navajas, J., Sequeira, L., Quintana, I., Saldana, J., & Ruiz-Mas, J. "IPTV Quality assessment system", Proceedings of the 7th Latin American Networking Conference, October 2012 (pp. 52--58). ACM.

Digital Library

Google Scholar

[4]

ITU, "Recommendation ITU-R BT.500-13, Methodology for the subjective assessment of the quality of television pictures", Jan. 2012.

Google Scholar

[5]

ITU, "Recommnedation ITU-R BT.710-4 Subjective Assessment Methods for Image Quality in High Definition Television", Nov. 1998.

Google Scholar

[6]

ITU, "Recommendation ITU-T P.910 Subjective video quality assessment methods", April 2008.

Google Scholar

[7]

ITU, "Recommendation ITU-T P.913 Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment", January 2014.

Google Scholar

[8]

B. P. van den Berge, "Influence of level of motion on perceived video quality in MPEG-2 coding," Master Thesis, Faculty of Industrial Engineering & Innovation Sciences, Human Technology Interaction, Eindhoven University of Technology, 2009.

Google Scholar

[9]

Q. Huynh-Thu, M.-N. Garcia, F. Speranza, P. Corriveau and A. Raake, "Study of Rating Scales for Subjective Quality Assessment of High-Definition Video", IEEE Transactions on Broadcasting, vol. 57, no. 1, March 2011.

Crossref

Google Scholar

[10]

K. Seshadrinathan, R. Soundararajan, A. Conrad Bovik and L. K. Cormack, "Study of Subjective and Objective Quality Assessment of Video", IEEE Transactions on Image Processing, vol. 19, no. 6, June 2010.

Digital Library

Google Scholar

[11]

O. Nemethova, M. Ries and D. Antitza, "Test Equipment of Time-Variant Subjective Perceptual Video Quality in Mobile Terminals," proceedings of HCI, Phoenix, USA, November 14--16, 2005.

Google Scholar

[12]

T. Liu, G. Cash, N. Narvekar and J. Bloom, "Continuous mobile video subjective quality assessment using gaming steering wheel," Sixth International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM), January 2012.

Google Scholar

[13]

V. L. Organization, "VLC Media Player," {Online}. Available: http://www.videolan.org/vlc/. (Accessed: 9 June 2014)

Google Scholar

[14]

"VQI: Video Quality Indicators for DTV," {Online}. Available: http://www2.um.edu.uy/ingenieria/vqi/. (Accessed: 9 June 2014)

Google Scholar

Cited By

View all

Murron MJoskowicz JSotelo R(2021)Quality of Service and Quality of Experience in BroadcastingWiley Encyclopedia of Electrical and Electronics Engineering10.1002/047134608X.W8415(1-10)Online publication date: 21-Jun-2021
https://doi.org/10.1002/047134608X.W8415
Sotelo RJoskowicz JGonzalez A(2018)4K Video Quality Assessment: An Application to Support Subjective Tests2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB)10.1109/BMSB.2018.8436745(1-5)Online publication date: Jun-2018
https://doi.org/10.1109/BMSB.2018.8436745
Sotelo R(2015)Subjective and objective video quality assessment. A parametric model for digital television coded with H.2642015 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON)10.1109/Chilecon.2015.7400414(437-440)Online publication date: Oct-2015
https://doi.org/10.1109/Chilecon.2015.7400414
Show More Cited By

Index Terms

Automation of Subjective Video Quality Measurements
1. Information systems
  1. World Wide Web
    1. Web applications
      1. Internet communications tools
        Web conferencing

Recommendations

3D video subjective quality: a new database and grade comparison study

This paper presents a research study on the subjective assessment of 3D video quality using a newly constructed 3D video database (3DVCL@FER). This database consists of 8 original 3D video sequences, each degraded with 22 different degradation types, ...
Subjective Multimedia Quality Assessment

The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the ...
Subjective and objective quality evaluation of watermarked audio tracks
WEDELMUSIC'02: Proceedings of the Second international conference on Web delivering of music

This paper presents methods for the quality evaluation of watermarked audio data. Subjective and objective methods used in testing transparency as well as approaches rating the quality of the watermarked copies with regard to the original ones are ...

Comments

Information & Contributors

Information

Published In

LANC '14: Proceedings of the Latin America Networking Conference on LANC 2014

September 2014

91 pages

ISBN:9781450332804

DOI:10.1145/2684083

General Chair:
Eduardo Cerqueira
Federal University of Para, BR

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

Video Subjective Tests

Qualifiers

Tutorial
Research
Refereed limited

Conference

LANC '14

LANC '14: Latin America Networking Conference

September 18 - 19, 2014

Montevideo, Uruguay

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
72
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Murron MJoskowicz JSotelo R(2021)Quality of Service and Quality of Experience in BroadcastingWiley Encyclopedia of Electrical and Electronics Engineering10.1002/047134608X.W8415(1-10)Online publication date: 21-Jun-2021
https://doi.org/10.1002/047134608X.W8415
Sotelo RJoskowicz JGonzalez A(2018)4K Video Quality Assessment: An Application to Support Subjective Tests2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB)10.1109/BMSB.2018.8436745(1-5)Online publication date: Jun-2018
https://doi.org/10.1109/BMSB.2018.8436745
Sotelo R(2015)Subjective and objective video quality assessment. A parametric model for digital television coded with H.2642015 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON)10.1109/Chilecon.2015.7400414(437-440)Online publication date: Oct-2015
https://doi.org/10.1109/Chilecon.2015.7400414
Sotelo RJoskowicz JGarella JDuran DJuayek M(2015)Subjective video quality test: Methodology, database and experience2015 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting10.1109/BMSB.2015.7177254(1-6)Online publication date: Jun-2015
https://doi.org/10.1109/BMSB.2015.7177254

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

3D video subjective quality: a new database and grade comparison study

Subjective Multimedia Quality Assessment

Subjective and objective quality evaluation of watermarked audio tracks

Comments

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tag

Qualifiers

Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

3D video subjective quality: a new database and grade comparison study

Subjective Multimedia Quality Assessment

Subjective and objective quality evaluation of watermarked audio tracks

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tag

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations