Article

A structured approach to instrumentation system development and evaluation

Authors:

Diane T. RoverAuthors Info & Claims

Supercomputing '95: Proceedings of the 1995 ACM/IEEE conference on Supercomputing

Pages 30 - es

https://doi.org/10.1145/224170.224271

Published: 08 December 1995 Publication History

Abstract

Software instrumentation is a widely used technique for parallel program performance evaluation, debugging, steering, and visualization. With increasing sophistication of parallel tool development technologies and broadening of application areas where these tools are being used, runtime data collection and management activities are growing in importance; we use the term instrumentation system (IS) to refer to components that support these activities in state-of-the-art parallel tool environments. An IS consists of Local Instrumentation Servers, an Instrumentation System Manager, and a Transfer Protocol. The overheads and perturbation effects attributed to an IS must be accounted for to ensure correct and efficient representation of program behavior, especially for on-line and real-time environments. Moreover, an IS is a key facilitator of integration of tools in an environment. In this paper, we define the primary components of an IS and their roles in an integrated environment, and classify ISs according to selected features. We introduce a structured approach to plan, design, model, evaluate, implement, and validate an IS. The approach provides a means to formally address domain-specific requirements. The modeling and evaluation processes are illustrated in the context of three distinctive IS case studies for PICL, Paradyn, and Vista. Valuable feedback on performance effects of IS parameters and policies can assist developers in making design decisions early in the software development cycle. Additionally, use of structured software engineering methods can support the mapping of an abstract IS model to an implementation of the IS.

References

[1]

Bhatt, Devesh, Rakesh Jha, Todd Steeves, Rashmi Bhatt, and David Wills, "SPI: An Instrumentation Development Environment for Parallel/Distributed Systems," to Appear in the Proc. of Int. Parallel Processing Symposium, April 1995.

Digital Library

[2]

Brown D., S. Hackstadt, A. Malony, B. Mohr, "Program Analysis Environments for Parallel Language Systems: The TAU Environment," Proc. of the Second Workshop on Environments and Tools For Parallel Scientific Computing, Townsend, Tennessee, May 1994, pp. 162--171.

[3]

Cheng, Doreen Y., "A Survey of Parallel Programming Languages and Tools," Report RND-93-005, NASA Ames Research Center, March 1993.

[4]

Geist, G., M. Heath, B. Peyton, and P. Worley, "A User's Guide to PICL", Technical Report ORNL/TM-11616, Oak Ridge National Laboratory, March 1991.

[5]

Gelenbe, E., G. Pujolle, and J. C. C. Nelson, Introduction to Queuing Networks, John Wiley, 1987.

Digital Library

[6]

Gu, Weiming, Greg Eisenhauer, Eileen Kramer, Karsten Schwan, John Stasko, and Jeffrey Vetter, "Falcon: On-line Monitoring and Steering of Large-Scale Parallel Programs," Technical Report GIT--CC--94--21, 1994.

[7]

Hao, Ming C., Alan H. Karp, Abdul Waheed, and Mehdi Jazayeri, "VIZIR: An Integrated Environment for Distributed Program Visualization," Proc. of Int. Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS `95) Tools Fair, Durham, North Carolina, Jan. 1995.

Digital Library

[8]

Heath, Michael T. and Jennifer A. Etheridge, "Visualizing the Performance of Parallel Programs," IEEE Software, 8(5), September 1991, pp. 29--39.

Digital Library

[9]

Hollingsworth, J. K. and B. P. Miller, "Dynamic Control of Performance Monitoring on Large Scale Parallel Systems," Proc. of Int. Con. on Supercomputing, Tokyo, Japan, July 19--23, 1993.

Digital Library

[10]

Hollingsworth, J. K. and B. P. Miller, "An Adaptive Cost Model for Parallel Program Instrumentation," Technical Report, Oct. 1994.

[11]

Jain, Raj, The Art of Computer Systems Performance Analysis---Techniques for Experimental Design, Measurement, Simulation, and Modeling, John Wiley & Sons, Inc., 1991.

[12]

Kilpatrick, Carol and Karsten Schwan, "ChaosMON---Application-Specific Monitoring and Display of Performance Information for Parallel and Distributed Systems," Proceedings of the ACM/ONR Workshop on Parallel and Distributed Debugging, Santa Cruz, California, May 20--21, 1991.

Digital Library

[13]

Kleinrock, Leonard and Willard Korfhage, "Collecting Unused Processing Capacity: An Analysis of Transient Distributed Systems," IEEE Transactions on Parallel and Distributed Systems, 4(4), May 1993, pp. 535--546.

Digital Library

[14]

Law, Averill M. and W. D. Kelton, Simulation Modeling and Analysis, McGraw-Hill, Inc., 1991.

Digital Library

[15]

Lieu, Eric, personal communications, Hewlett-Packard Labs, Palo Alto, California, June 1994.

[16]

Malony, A. D., D. A. Reed, and H. A. G. Wijshoff, "Performance Measurement Intrusion and Perturbation Analysis," IEEE Transactions on Parallel and Distributed Systems, 3(4), July 1992.

Digital Library

[17]

Malony, A., B. Mohr, P. Beckman, D. Gannon, S. Yang, F. Bodin, and S. Kesavan, "Implementing a Parallel C++ Runtime System for Scalable Parallel Systems," Proceedings of Supercomputing `93, Portland, Oregon, November 15--19, 1993.

Digital Library

[18]

Malony, A. D., "Measurement and Monitoring of Parallel Programs," Tutorial, Sigmetrics `1994, Nashville, Tennessee, May 16--20, 1994.

[19]

Miller, Barton P., Jonathan M. Cargille, R. Bruce Irvin, Krishna Kunchithapadam, Mark D. Callaghan, Jeffrey K. Hollingsworth, Karen L. Karavanic, and Tia Newhall, "The Paradyn Parallel Performance Measurement Tools," Technical Report, 1994.

[20]

Nutt, Gary J. and Adam J. Griff, "Extensible Parallel Program Performance Visualization," Proc. of Int. Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS `95), Durham, North Carolina, Jan. 1995.

Digital Library

[21]

Ogle, David M., Karsten Schwan, and Richard Snodgrass, "Application-Dependent Dynamic Monitoring of Distributed and Parallel Systems," IEEE Transactions on Parallel and Distributed Systems, 4(7), July 1993, pp. 762--778.

Digital Library

[22]

Reed, Daniel A., Ruth A. Aydt, Tara M. Madhyastha, Roger J. Noe, Keith A. Shields, Bradley W. Schwartz, "The Pablo Performance Analysis Environment," Dept. of Comp. Sci., Univ. of Ill., 1992.

[23]

Resnick, Sidney I., Adventures in Stochastic Processes, Birkhauser, 1992.

Digital Library

[24]

Ries, Bernhard, R. Anderson, D. Breazeal, K. Callaghan, E. Richards, and W. Smith, "The Paragon Performance Monitoring Environment," Proceedings of Supercomputing `93, Portland, Oregon, Nov. 15--19, 1993.

Digital Library

[25]

Rover, Diane T., "Vista: Visualization and Instrumentation of Scalable Multicomputer Applications," Project Summary, IEEE Parallel and Distributed Technology, 1(3), August 1993, pp. 83.

[26]

Rover, Diane T., "Performance Evaluation: Integrating Techniques and Tools into Environments and Frameworks," Roundtable, Supercomputing `94, Washington DC, November 14--18, 1994.

Digital Library

[27]

Simmons, M., and R. Koskela, editors, Performance Instrumentation and Visualization, ACM & Addison-Wesley, 1990.

[28]

Waheed, A., B. Kronmuller, Roomi Sinha, and D. T. Rover, "A Toolkit for Advanced Performance Analysis," Proc. of Int. Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS `94) Tools Fair, Durham, North Carolina, Jan. 31-- Feb. 2, 1994.

Digital Library

[29]

Waheed, A., Vincent Melfi, and Diane T. Rover, "A Model for Instrumentation System Management in Concurrent Systems," Proceedings of the Twenty Eights Hawaii International Conference on System Sciences, Maui, Hawaii, Jan. 3-6, 1995.

Digital Library

[30]

Workshop on Debugging and Performance Tuning of Parallel Computing Systems, Chatham, Mass., Oct. 3-5, 1994.

[31]

Yan, Jerry C. and S. Listgarten, "Intrusion Compensation for Performance Evaluation of Parallel Programs on a Multicomputer," Proceedings of the Sixth International Conference on Parallel and Distributed systems, Louisville, KY, Oct. 14--16, 1993.

[32]

Yan, Jerry, "Performance Tuning with AIMS---An Automated Instrumentation and Monitoring System for Multicomputers," Proc. of the Twenty-Seventh Hawaii Int. Conf. on System Sciences, Hawaii, January 1994.

Cited By

Edmondson JGokhale ANeema S(2011)Automating testing of service-oriented mobile applications with distributed knowledge and reasoningProceedings of the 2011 IEEE International Conference on Service-Oriented Computing and Applications10.1109/SOCA.2011.6166261(1-4)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1109/SOCA.2011.6166261
Reilly DTaleb-Bendiab A(2002)Dynamic instrumentation for Jini applicationsProceedings of the 3rd international conference on Software engineering and middleware10.5555/1756361.1756376(157-173)Online publication date: 20-May-2002
https://dl.acm.org/doi/10.5555/1756361.1756376
Reilly DTaleb-Bendiab A(2002)Dynamic Instrumentation for Jini ApplicationsSoftware Engineering and Middleware10.1007/3-540-38093-0_11(157-173)Online publication date: 6-Nov-2002
https://doi.org/10.1007/3-540-38093-0_11
Show More Cited By

Index Terms

A structured approach to instrumentation system development and evaluation

Recommendations

Automatic Low Overhead Program Instrumentation with the LOPI Framework
INTERACT '05: Proceedings of the 9th Annual Workshop on Interaction between Compilers and Computer Architectures

Program instrumentation is an important technique for a different tasks such as performance measurements, debugging, and coverage analysis. Instrumentation, however, poses two important requirements to be useful: it must be easy to apply and it should ...
Dynamic Instrumentation of Large-Scale MPI and OpenMP Applications
IPDPS '03: Proceedings of the 17th International Symposium on Parallel and Distributed Processing

In recent years, software infrastructures for the run-time instrumentation of programs have begun to emerge. This paper presents and evaluates prototypes of dynamic instrumentation and dynamic control of instrumentation for parallel mixed MPI/OpenMP ...
A unified system-framework engine for industrial instrumentation systems

To create high-quality products and improve the productivity of iron-and steel-making processes, several types of instrumentation system are used in the process–operation lines. However, current instrumentation systems have fundamental problems with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Supercomputing '95: Proceedings of the 1995 ACM/IEEE conference on Supercomputing

December 1995

875 pages

ISBN:0897918169

DOI:10.1145/224170

Chairman:
Sid Karin
San Diego Supercomputer Center, San Diego, CA

Copyright © 1995 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 December 1995

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SC '95

Sponsor:

SIGARCH
IEEE-CS

SC '95: International Conference for High Performance Computing, Networking, Storage and Analysis

December 4 - 8, 1995

California, San Diego, USA

Acceptance Rates

Supercomputing '95 Paper Acceptance Rate 69 of 241 submissions, 29%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
408
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Edmondson JGokhale ANeema S(2011)Automating testing of service-oriented mobile applications with distributed knowledge and reasoningProceedings of the 2011 IEEE International Conference on Service-Oriented Computing and Applications10.1109/SOCA.2011.6166261(1-4)Online publication date: 12-Dec-2011
https://dl.acm.org/doi/10.1109/SOCA.2011.6166261
Reilly DTaleb-Bendiab A(2002)Dynamic instrumentation for Jini applicationsProceedings of the 3rd international conference on Software engineering and middleware10.5555/1756361.1756376(157-173)Online publication date: 20-May-2002
https://dl.acm.org/doi/10.5555/1756361.1756376
Reilly DTaleb-Bendiab A(2002)Dynamic Instrumentation for Jini ApplicationsSoftware Engineering and Middleware10.1007/3-540-38093-0_11(157-173)Online publication date: 6-Nov-2002
https://doi.org/10.1007/3-540-38093-0_11
Waheed ARover DHollingsworth J(1998)Modeling and Evaluating Design Alternatives for an On-Line Instrumentation SystemIEEE Transactions on Software Engineering10.1109/32.68940224:6(451-470)Online publication date: 1-Jun-1998
https://dl.acm.org/doi/10.1109/32.689402
Walker EJianjian S(1997)An instrumentation system for post-mortem visualization of distributed applicationsProceedings of the High-Performance Computing on the Information Superhighway, HPC-Asia '9710.5555/523549.822873Online publication date: 28-Apr-1997
https://dl.acm.org/doi/10.5555/523549.822873
Walker ESong Jianjian (1997)An instrumentation system for post-mortem visualization of distributed applicationsProceedings High Performance Computing on the Information Superhighway. HPC Asia '9710.1109/HPC.1997.592231(684-687)Online publication date: 1997
https://doi.org/10.1109/HPC.1997.592231
Waheed ARover D(1996)Performance Evaluation of an Integrated Instrumentation SystemProceedings of the 4th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems10.5555/823448.823550Online publication date: 1-Feb-1996
https://dl.acm.org/doi/10.5555/823448.823550
Waheed ARover DMutka MBakic APierce D(1996)VistaProceedings of the 4th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems10.5555/823448.823542Online publication date: 1-Feb-1996
https://dl.acm.org/doi/10.5555/823448.823542
Waheed ARover DHollingsworth JClayton B(1996)Modeling, evaluation, and testing of paradyn instrumentation systemProceedings of the 1996 ACM/IEEE conference on Supercomputing10.1145/369028.369065(18-es)Online publication date: 17-Nov-1996
https://dl.acm.org/doi/10.1145/369028.369065
Haake BSchauser KScheiman CClayton B(1996)Profiling a parallel language based on fine-grained communicationProceedings of the 1996 ACM/IEEE conference on Supercomputing10.1145/369028.369063(17-es)Online publication date: 17-Nov-1996
https://dl.acm.org/doi/10.1145/369028.369063
Show More Cited By

View Options

View options

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents