research-article

Measuring the Quality of Experience of Web users

Authors:

Dario RossiAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 46, Issue 4

Pages 8 - 13

https://doi.org/10.1145/3027947.3027949

Published: 23 December 2016 Publication History

Abstract

Measuring quality of Web users experience (WebQoE) faces the following trade-off. On the one hand, current practice is to resort to metrics, such as the document completion time (onLoad), that are simple to measure though knowingly inaccurate. On the other hand, there are metrics, like Google’s SpeedIndex, that are better correlated with the actual user experience, but are quite complex to evaluate and, as such, relegated to lab experiments. In this paper, we first provide a comprehensive state of the art on the metrics and tools available for WebQoE assessment. We then apply these metrics to a representative dataset (the Alexa top-100 webpages) to better illustrate their similarities, differences, advantages, and limitations. We next introduce novel metrics, inspired by Google’s SpeedIndex, that offer significant advantage in terms of computational complexity, while maintaining a high correlation with the SpeedIndex. These properties make our proposed metrics highly relevant and of practical use.

References

[1]

http://www.showslow.com.

[2]

https://www.telecom-paristech.fr/ drossi/webmos.

[3]

dynaTrace. http://dynatrace.com/.

[4]

Alexa Internet Inc. http://www.alexa.com.

[5]

Brutlag, J., Abrams, Z., and Meenan, P. Above the fold time: Measuring web page performance visually. http://conferences.oreilly.com/velocity/velocity-mar2011/public/schedule/detail/18692.

[6]

Butkiewicz, M., Wang, D., Wu, Z., Madhyastha, H. V., and Sekar, V. Klotski: Reprioritizing Web Content to Improve User Experience on Mobile Devices. In USENIX NSDI (Oakland, CA, May 2015), USENIX Association, pp. 439–453.

[7]

Erman, J., Gopalakrishnan, V., Jana, R., and Ramakrishnan, K. K. Towards a SPDY'Ier Mobile Web? In ACM CoNEXT (December 2013), pp. 303–314.

Digital Library

[8]

Google Inc. https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index.

[9]

Google Inc. https://googlewebmastercentral.blogspot.fr/2010/04/using-site-speed-in-web-search-ranking.html.

[10]

Google Inc. https://www.youtube.com/watch?v=muSIzHurn4U .

[11]

Google Inc. http://webpagetest.org.

[12]

Google Inc. Pagespeed insights. https://developers.google.com/speed/docs/insights/rules.

[13]

International Telecommunication Union (ITU). Subjective testing methodology for web browsing. Tech. rep., International Telecommunication Union, Feb. 2014.

[14]

M. Varvello, K. Schomp, D. Naylor, J. Blackburn, A. Finamore, and K. Papagiannaki. Is The Web HTTP/2 Yet? In Passive and Active Measurement (PAM) (Apr. 2016).

[15]

Miller, R. B. Response time in man-computer conversational transactions. In Proc. AFIPS Fall Joint Computer Conference (1968), ACM, pp. 267–277.

Digital Library

[16]

Nielsen, J. Response times: The 3 important limits. https://www.nngroup.com/articles/response-times-3-important-limits/.

[17]

Qian, F., Gopalakrishnan, V., Halepovic, E., Sen, S., and Spatscheck, O. TM3: Flexible Transport-layer Multi-pipe Multiplexing Middlebox Without Head-of-line Blocking. In ACM CoNEXT (December 2015).

[18]

Wang, X. S., Balasubramanian, A., Krishnamurthy, A., and Wetherall, D. How Speedy is SPDY? In USENIX NSDI (Seattle, WA, Apr. 2014), USENIX Association, pp. 387–399.

[19]

Wang, X. S., Krishnamurthy, A., and Wetherall, D. Speeding up Web Page Loads with Shandian. In USENIX NSDI (2016).

[20]

World Wide Web Consortium (W3C). https://www.w3.org/TR/2012/REC-navigation-timing-20121217.

[21]

Yahoo Inc. Yslow Ruleset Matrix. http://yslow.org/ruleset-matrix/.

Cited By

Zhang XJin SHe YHassan AMao ZQian FZhang ZChua TNgo CKa-Wei Lee RKumar RLauw H(2024)QUIC is not Quick Enough over Fast InternetProceedings of the ACM Web Conference 202410.1145/3589334.3645323(2713-2722)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645323
Perna GTrevisan MGiordano DPerdices DMellia M(2024)Monitoring Web QoE in Satellite Networks from Passive Measurements2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454720(669-676)Online publication date: 6-Jan-2024
https://doi.org/10.1109/CCNC51664.2024.10454720
Yuan ZRen KWang GMiao XChen HDuh WHuang HKato MMothe JPoblete B(2023)Hydrus: Improving Personalized Quality of Experience in Short-form Video ServicesProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591696(1127-1136)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591696
Show More Cited By

Index Terms

Measuring the Quality of Experience of Web users
1. Information systems
  1. World Wide Web
2. Networks
  1. Network performance evaluation
    1. Network measurement
    2. Network performance analysis
  2. Network protocols
    1. Application layer protocols

Recommendations

Perceived Performance of Top Retail Webpages In the Wild: Insights from Large-scale Crowdsourcing of Above-the-Fold QoE
Internet QoE '17: Proceedings of the Workshop on QoE-based Analysis and Management of Data Communication Networks

Clearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in ...
Measuring the Quality of Experience of Web users
Internet-QoE '16: Proceedings of the 2016 workshop on QoE-based Analysis and Management of Data Communication Networks

Measuring quality of Web users experience (WebQoE) faces the following trade-off. On the one hand, current practice is to resort to metrics, such as the document completion time (onLoad), that are simple to measure though knowingly inaccurate. On the ...
Perceived Performance of Top Retail Webpages In the Wild

Clearly, no one likes webpages with poor quality of experience (QoE). Being perceived as slow or fast is a key element in the overall perceived QoE of web applications. While extensive effort has been put into optimizing web applications (both in ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 46, Issue 4

October 2016

40 pages

ISSN:0146-4833

DOI:10.1145/3027947

Issue’s Table of Contents

Copyright © 2016 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 December 2016

Published in SIGCOMM-CCR Volume 46, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
555
Total Downloads

Downloads (Last 12 months)54
Downloads (Last 6 weeks)4

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XJin SHe YHassan AMao ZQian FZhang ZChua TNgo CKa-Wei Lee RKumar RLauw H(2024)QUIC is not Quick Enough over Fast InternetProceedings of the ACM Web Conference 202410.1145/3589334.3645323(2713-2722)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645323
Perna GTrevisan MGiordano DPerdices DMellia M(2024)Monitoring Web QoE in Satellite Networks from Passive Measurements2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454720(669-676)Online publication date: 6-Jan-2024
https://doi.org/10.1109/CCNC51664.2024.10454720
Yuan ZRen KWang GMiao XChen HDuh WHuang HKato MMothe JPoblete B(2023)Hydrus: Improving Personalized Quality of Experience in Short-form Video ServicesProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591696(1127-1136)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591696
Garcia Murça SPerdices DLópez de Vergara J(2023)Analysis of Top Internet Websites Load Times2023 IEEE Virtual Conference on Communications (VCC)10.1109/VCC60689.2023.10474816(264-269)Online publication date: 28-Nov-2023
https://doi.org/10.1109/VCC60689.2023.10474816
Baskaran SFaisal TWang CLopez DOrdonez-Lucena JArribas I(2023)The Role of DLT for Beyond 5G Systems and Services: A VisionIEEE Communications Standards Magazine10.1109/MCOMSTD.0004.22000537:1(32-38)Online publication date: Mar-2023
https://doi.org/10.1109/MCOMSTD.0004.2200053
Yan YZheng YLiu XMedvidovic NWang WGrundy JPollock LPenta M(2023)AdHere: Automated Detection and Repair of Intrusive AdsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00051(486-498)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00051
van Riet JMalavolta IGhaleb T(2023)Optimize along the wayJournal of Systems and Software10.1016/j.jss.2022.111593198:COnline publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1016/j.jss.2022.111593
Xilogianni CDoukas FDrivas IKouis D(2022)Speed Matters: What to Prioritize in Optimization for Faster WebsitesAnalytics10.3390/analytics10200121:2(175-192)Online publication date: 25-Nov-2022
https://doi.org/10.3390/analytics1020012
Jha NTrevisan MVassio LMellia M(2022)The Internet with Privacy Policies: Measuring The Web Upon ConsentACM Transactions on the Web10.1145/355535216:3(1-24)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1145/3555352
Casas PWassermann SWehner NSeufert MHossfeld T(2022)Not all Web Pages are Born the Same Content Tailored Learning for Web QoE Inference2022 IEEE International Symposium on Measurements & Networking (M&N)10.1109/MN55117.2022.9887781(1-6)Online publication date: 18-Jul-2022
https://doi.org/10.1109/MN55117.2022.9887781
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents