article

Measuring thin-client performance using slow-motion benchmarking

Authors:

Naomi NovikAuthors Info & Claims

ACM Transactions on Computer Systems (TOCS), Volume 21, Issue 1

Pages 87 - 115

https://doi.org/10.1145/592637.592640

Published: 01 February 2003 Publication History

Abstract

Modern thin-client systems are designed to provide the same graphical interfaces and applications available on traditional desktop computers while centralizing administration and allowing more efficient use of computing resources. Despite the rapidly increasing popularity of these client-server systems, there are few reliable analyses of their performance. Industry standard benchmark techniques commonly used for measuring desktop system performance are ill-suited for measuring the performance of thin-client systems because these benchmarks only measure application performance on the server, not the actual user-perceived performance on the client. To address this problem, we have developed slow-motion benchmarking, a new measurement technique for evaluating thin-client systems. In slow-motion benchmarking, performance is measured by capturing network packet traces between a thin client and its respective server during the execution of a slow-motion version of a conventional benchmark application. These results can then be used either independently or in conjunction with conventional benchmark results to yield an accurate and objective measure of the performance of thin-client systems. We have demonstrated the effectiveness of slow-motion benchmarking by using this technique to measure the performance of several popular thin-client systems in various network environments on Web and multimedia workloads. Our results show that slow-motion benchmarking solves the problems with using conventional benchmarks on thin-client systems and is an accurate tool for analyzing the performance of these systems.

References

[1]

Broadway. Broadway/X web FAQ. Available at http://www.broadwayinfo.com/bwfaq.htm.

[2]

Charon Systems. Available at http://www.charon.com.

[3]

Citrix Systems 1998. Citrix MetaFrame 1.8 Backgrounder, Citrix White Paper. Boca Raton, Fla.

[4]

Cumberland, B. C., Carius, G., and Muir, A. 1999. Microsoft Windows NT Server 4.0, Terminal Server Edition: Technical Reference. Microsoft Press, Redmond, Wash.

[5]

Curtis, J. 1998. Port mirroring: The duplex paradox. Network World Fusion.

[6]

Danskin, J. and Hanrahan, P. 1994. Profiling the X protocol. In Proceedings of the SIGMETRICS Conference on Measurement and Modeling of Computer Systems, SIGMETRICS (Nashville, Ten.).

[7]

Expand Networks 2000. Enterprise Caching and Citrix ICA: Performance Benchmark Test. Expand Networks, Roseland, N.J.

[8]

Expertcity. 2000. Desktop Streaming Technology and Security, Expertcity White Paper. Santa Barbara, Calif.

[9]

Howard, B. 2000. Thin is back. PC Magazine, Ziff Davis Media 19, 7 (Apr.).

[10]

LapLink.com, Inc. 1999. LapLink 2000 Users Guide. LapLink.com, Inc., Bothell, Wash.

[11]

Mathers, T. W. and Genoway, S. P. 1998. Windows NT Thin Client Solutions: Implementing Terminal Server and Citrix MetaFrame. Macmillan Technical, Indianapolis.

[12]

Microsoft. 2000a. Microsoft Windows NT Server 4.0, Terminal Server Edition: An Architectural Overview, Technical White Paper. Microsoft Corporation, Redmond, Wash.

[13]

Microsoft. 2000b. Windows 2000 Terminal Services Capacity and Scaling, Technical White Paper. Microsoft Corporation, Redmond, Wash.

[14]

Nieh, J., Yang, S. J., and Novik, N. 2000. A comparison of thin-client computing architectures. Tech. Rep. CUCS-022-00, Department of Computer Science, Columbia University, November.

[15]

Nielsen, J. 1994. Usability Engineering. Morgan Kaufman, San Francisco.

[16]

PC Anywhere. Available at http://www.symantec.com/pcanywhere.

[17]

Personable. Available at http://www.personable.com.

[18]

Runaware. Available at http://www.runaware.com.

[19]

Scheifler, R. W. and Gettys, J. 1986. The X Window System. ACM Trans. Graph. 5, 2 (Apr.).

[20]

Schmidt, B. K., Lam, M. S., and Northcutt, J. D. 1999. The interactive performance of SLIM: A stateless, thin-client architecture. In Proceedings of the Seventeenth ACM Symposium on Operating Systems Principles.

[21]

Shaw, A., Burgess, K. R., Pullan, J. M., and Cartwright, P. C. 2000. Method of Displaying an Application on a Variety of Client Devices in a Client/Server Network. US Patent US6104392.

[22]

Shunra Software. The Cloud. Available at http://www.shunra.com.

[23]

Sun Microsystems. Sun Ray 1 Enterprise Appliance. Available at http://www.sun.com/products/sunray1.

[24]

Tarantella, Inc. 1998. Tarantella Web-Enabling Software: The Adaptive Internet Protocol, A SCO Technical White Paper. Tarantella, Inc.

[25]

Tolly Research 2000. Thin-Client Networking: Bandwidth Consumption Using Citrix ICA. Tolly Research.

[26]

Virtual Network Computing. Available at http://www.uk.research.att.com/vnc.

[27]

WildPackets, Inc. Etherpeek 4. Available at http://www.wildpackets.com.

[28]

Wong, A. Y. and Seltzer, M. 1999. Evaluating Windows NT terminal server performance. In Proceedings of the Third USENIX Windows NT Symposium (USENIX, Seattle).

[29]

Yang, S. J., Nieh, J., and Novik, N. 2001. Measuring thin-client performance using slow-motion benchmarking. In Proceedings of USENIX 2001 Annual Technical Conference (USENIX, Boston).

[30]

Ziff Davis Media. i-Bench version 1.5. Available at http://www.i-bench.com.

Cited By

Arellano-Uson JMagaña EMorato DIzal M(2024)Survey on Quality of Experience Evaluation for Cloud-Based Interactive ApplicationsApplied Sciences10.3390/app1405198714:5(1987)Online publication date: 28-Feb-2024
https://doi.org/10.3390/app14051987
Liu TLucas JHe SLiu TWang XWang W(2024)Improving Resource and Energy Efficiency for Cloud 3D through Excessive Rendering ReductionProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650064(317-332)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650064
Arellano-Uson JMagaña EMorató DIzal M(2024)Interactivity Anomaly Detection in Remote Work Scenarios Using LSTMIEEE Access10.1109/ACCESS.2024.337240512(34402-34416)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3372405
Show More Cited By

Index Terms

Measuring thin-client performance using slow-motion benchmarking
1. Applied computing
  1. Computers in other domains
    1. Personal computers and PC applications
2. General and reference
  1. Cross-computing tools and techniques

Recommendations

On the performance of wide-area thin-client computing

While many application service providers have proposed using thin-client computing to deliver computational services over the Internet, little work has been done to evaluate the effectiveness of thin-client computing in a wide-area network. To assess ...
Web browsing performance of wireless thin-client computing
WWW '03: Proceedings of the 12th international conference on World Wide Web

Web applications are becoming increasingly popular for mobile wireless systems. However, wireless networks can have high packet loss rates, which can degrade web browsing performance on wireless systems. An alternative approach is wireless thin-client ...
Improving web browsing performance on wireless pdas using thin-client computing
WWW '04: Proceedings of the 13th international conference on World Wide Web

Web applications are becoming increasingly popular for mobile wireless PDAs. However, web browsing on these systems can be quite slow. An alternative approach is handheld thin-client computing, in which the web browser and associated application logic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer Systems

ACM Transactions on Computer Systems Volume 21, Issue 1

February 2003

115 pages

ISSN:0734-2071

EISSN:1557-7333

DOI:10.1145/592637

Issue’s Table of Contents

Copyright © 2003 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 February 2003

Published in TOCS Volume 21, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

123
Total Citations
View Citations
1,996
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Arellano-Uson JMagaña EMorato DIzal M(2024)Survey on Quality of Experience Evaluation for Cloud-Based Interactive ApplicationsApplied Sciences10.3390/app1405198714:5(1987)Online publication date: 28-Feb-2024
https://doi.org/10.3390/app14051987
Liu TLucas JHe SLiu TWang XWang W(2024)Improving Resource and Energy Efficiency for Cloud 3D through Excessive Rendering ReductionProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650064(317-332)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650064
Arellano-Uson JMagaña EMorató DIzal M(2024)Interactivity Anomaly Detection in Remote Work Scenarios Using LSTMIEEE Access10.1109/ACCESS.2024.337240512(34402-34416)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3372405
Dong HKinfe AYu JLiu QKilper DWilliams RVeeraraghavan M(2023)Towards Enabling Residential Virtual-Desktop ComputingIEEE Transactions on Cloud Computing10.1109/TCC.2021.311390511:1(745-762)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TCC.2021.3113905
Arellano-Uson JMagaña EMorato DIzal M(2023)Evaluation of RTT as an Estimation of Interactivity Time for QoE Evaluation in Remote Desktop Environments2023 33rd International Telecommunication Networks and Applications Conference10.1109/ITNAC59571.2023.10368539(240-245)Online publication date: 29-Nov-2023
https://doi.org/10.1109/ITNAC59571.2023.10368539
Kelkkanen VFiedler MLindero D(2021)Synchronous Remote Rendering for VRInternational Journal of Computer Games Technology10.1155/2021/66766442021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/6676644
Benamer WElberkawi ENeihum NAnwiji AYouns M(2021)Green Computing case study: calls for proposing solutions for the Arabian Gulf Oil CompanyE3S Web of Conferences10.1051/e3sconf/202122901063229(01063)Online publication date: 25-Jan-2021
https://doi.org/10.1051/e3sconf/202122901063
Arellano-Uson JMagaña EMorató DIzal M(2021)Protocol-agnostic method for monitoring interactivity time in remote desktop servicesMultimedia Tools and Applications10.1007/s11042-021-10708-380:13(19107-19135)Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1007/s11042-021-10708-3
Demestichas KDaskalakis E(2020)Information and Communication Technology Solutions for the Circular EconomySustainability10.3390/su1218727212:18(7272)Online publication date: 4-Sep-2020
https://doi.org/10.3390/su12187272
Liu THe SHuang STsang DTang LMars JWang W(2020)A Benchmarking Framework for Interactive 3D Applications in the Cloud2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)10.1109/MICRO50266.2020.00076(881-894)Online publication date: Oct-2020
https://doi.org/10.1109/MICRO50266.2020.00076
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents