Article

Software performance testing based on workload characterization

Authors:

Alberto Avritzer,

Joe Kondek,

Danielle Liu,

Elaine J. WeyukerAuthors Info & Claims

WOSP '02: Proceedings of the 3rd international workshop on Software and performance

Pages 17 - 24

https://doi.org/10.1145/584369.584373

Published: 24 July 2002 Publication History

Get Access

Abstract

A major concern of most businesses is their ability to meet customers' performance requirements. This paper describes our workload-based approach to performance testing, and includes a case study that demonstrates the application of this approach to a large, industrial software system. For this system, we collected data in the field to determine the current production usage, and then assessed the performance of the system under both current workloads, and those likely to be encountered in the future. This led to the identification of a software bottleneck, which, had it occurred in the field rather than in the test lab, would have likely had significant consequences.

References

[1]

V. Almeida, A. Bestavros, M. Crovella and A. Oliveira. Characterizing reference locality in the WWW. In Proc. IEEE Conf Parallel and Distributed Information Systems (PDIS'96), December 1996.

Digital Library

Google Scholar

[2]

M. Arlitt and T. Jin. Workload characterization of the 1998 world cup web site. Hewlett-Packard Labs, Tech Rep HPL-1999-35R1, 1999, http://www.hpl.hp.com/techreports/1999/HPL-1999-35R1.html.

Google Scholar

[3]

M. Arlitt and C. Williamson. Internet web servers: workload characterization and performance implications. IEEE/ACM Trans Networking, 5(5):631-645, 1997.

Digital Library

Google Scholar

[4]

A. Avritzer and E.J. Weyuker. The automatic generation of load test suites and the assessment of the resulting software, IEEE Trans Software Eng, Sept 1995, pp.705-716.

Digital Library

Google Scholar

[5]

A. Cockcroft and R. Pettit Sun performance and tuning. Sun Microsystems, 2ed, 1998.

Digital Library

Google Scholar

[6]

J.D. Musa. Operational profiles in software reliability engineering. IEEE Software, 10(2):14-32, March 1993.

Digital Library

Google Scholar

[7]

D. Menasce, V. Almeida, R. Fonseca and M. Mendes. A methodology for workload characterization of e-commerce sites. In Proc first ACM conference on Electronic Commerce, 1999, Denver.

Digital Library

Google Scholar

[8]

D. Menasce and V. Almeida. Capacity planning for web services: metrics, models and methods. Prentice Hall, 2001.

Digital Library

Google Scholar

[9]

P. Reeser. Performance of web servers in a distributed computing environment. Teletraffic Engineering in the Internet Era, Elsevier, 2001.

Google Scholar

[10]

E. J. Weyuker. Using failure cost information for testing and reliability Assessment. ACM Trans Software Eng and Methodology, 5(2):87-98, April 1996.

Digital Library

Google Scholar

Cited By

View all

Sena PAbelem ADán GMenasché D(2024)Management of Caching Policies and Redundancy Over Unreliable ChannelsIEEE Transactions on Network and Service Management10.1109/TNSM.2023.333455921:2(1912-1928)Online publication date: Apr-2024
https://doi.org/10.1109/TNSM.2023.3334559
Castaño Díaz C(2023)Designing Games for Stealth AssessmentGames as Stealth Assessments10.4018/979-8-3693-0568-3.ch007(154-190)Online publication date: 29-Sep-2023
https://doi.org/10.4018/979-8-3693-0568-3.ch007
Tomak JGorlatch S(2022)RAST: Evaluating Performance of a Legacy System Using Regression Analysis and Simulation2022 30th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS56607.2022.00015(49-56)Online publication date: Oct-2022
https://doi.org/10.1109/MASCOTS56607.2022.00015
Show More Cited By

Recommendations

Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study

An approach to software performance testing is discussed. A case study describing the experience of using this approach for testing the performance of a system used as a gateway in a large industrial client/server transaction processing application is ...
Developing software performance with the performance refinement and evolution model
WOSP '07: Proceedings of the 6th international workshop on Software and performance

Performance is an important attribute of a software system. To develop a software system of acceptable performance, the team needs to specify precise performance requirements, design appropriate test cases, and use suitable techniques to analyze the ...
Teaching software testing concepts using a mutation testing game
ICSE-SEET '17: Proceedings of the 39th International Conference on Software Engineering: Software Engineering and Education Track

Software testing is a core aspect of software development, but testing programs systematically is not always a core aspect of software engineering education. As a result, software developers often treat testing as a liability, and overall software ...

Comments

Information & Contributors

Information

Published In

WOSP '02: Proceedings of the 3rd international workshop on Software and performance

July 2002

318 pages

ISBN:1581135637

DOI:10.1145/584369

Program Chairs:
Simonetta Balsamo
Universita Ca' Foscari di Venezia
,
Paola Inverardi
Universita di L' Aquila
,
Bran Selic
Rational Software Inc.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 July 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

WOSP02

Sponsor:

WOSP02: Workshop on Software and Performance ( co-located with ISSTA 2002 )

July 24 - 26, 2002

Rome, Italy

Acceptance Rates

WOSP '02 Paper Acceptance Rate 35 of 64 submissions, 55%;

Overall Acceptance Rate 149 of 241 submissions, 62%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

46
Total Citations
View Citations
1,687
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Sena PAbelem ADán GMenasché D(2024)Management of Caching Policies and Redundancy Over Unreliable ChannelsIEEE Transactions on Network and Service Management10.1109/TNSM.2023.333455921:2(1912-1928)Online publication date: Apr-2024
https://doi.org/10.1109/TNSM.2023.3334559
Castaño Díaz C(2023)Designing Games for Stealth AssessmentGames as Stealth Assessments10.4018/979-8-3693-0568-3.ch007(154-190)Online publication date: 29-Sep-2023
https://doi.org/10.4018/979-8-3693-0568-3.ch007
Tomak JGorlatch S(2022)RAST: Evaluating Performance of a Legacy System Using Regression Analysis and Simulation2022 30th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS56607.2022.00015(49-56)Online publication date: Oct-2022
https://doi.org/10.1109/MASCOTS56607.2022.00015
Tomak JGorlatch S(2021)Measuring Performance of Fault Management in a Legacy System: An Alarm System StudyModelling, Analysis, and Simulation of Computer and Telecommunication Systems10.1007/978-3-030-68110-4_9(129-146)Online publication date: 29-Jan-2021
https://doi.org/10.1007/978-3-030-68110-4_9
ÜNLÜ OBÖLÜK P(2020)WORKLOAD CHARACTERIZATION OF TELECOM SOFTWARETelekom Yazılımı İş Yükü KarakterizasyonuKonya Journal of Engineering Sciences10.36306/konjes.6987088:1(151-163)Online publication date: 5-Mar-2020
https://doi.org/10.36306/konjes.698708
Al-tekreeti MNaik KAbdrabou AZaman MSrivastava P(2019)A Methodology for Generating Tests for Evaluating User-Centric Performance of Mobile Streaming ApplicationsModel-Driven Engineering and Software Development10.1007/978-3-030-11030-7_18(406-429)Online publication date: 1-Feb-2019
https://doi.org/10.1007/978-3-030-11030-7_18
Al‐tekreeti MAbdrabou ANaik K(2019)An end‐user‐centric test generation methodology for performance evaluation of mobile networked applicationsSoftware Testing, Verification and Reliability10.1002/stvr.171329:6-7Online publication date: 7-Oct-2019
https://doi.org/10.1002/stvr.1713
Noor JHossain MAlam MUddin AChellappan SAl Islam A(2018)svLoad: An Automated Test-Driven Architecture for Load Testing in Cloud Systems2018 IEEE Global Communications Conference (GLOBECOM)10.1109/GLOCOM.2018.8647493(1-7)Online publication date: Dec-2018
https://doi.org/10.1109/GLOCOM.2018.8647493
Bahana RGaol FWiguna THendric SSoewito BNugroho EDirgantoro BAbdurachman E(2018)Performance test for prototype game for children with adhdJournal of Physics: Conference Series10.1088/1742-6596/978/1/012004978(012004)Online publication date: 15-Mar-2018
https://doi.org/10.1088/1742-6596/978/1/012004
Yu TPradel M(2018)Pinpointing and repairing performance bottlenecks in concurrent programsEmpirical Software Engineering10.1007/s10664-017-9578-123:5(3034-3071)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1007/s10664-017-9578-1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study

Developing software performance with the performance refinement and evolution model

Teaching software testing concepts using a mutation testing game