Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Specification techniques for Markov reward models

  • Published:
Discrete Event Dynamic Systems Aims and scope Submit manuscript

Abstract

Markov reward models (MRMs) are commonly used for the performance, dependability, and performability analysis of computer and communication systems. Many papers have addressed solution techniques for MRMs. Far less attention has been paid to the specification of MRMs and the subsequent derivation of the underlying MRM. In this paper we only briefly address the mathematical aspects of MRMs. Instead, emphasis is put on specification techniques. In an application independent way, we distinguish seven classes of specification techniques: stochastic Petri nets, queuing networks, fault trees, production rule systems, communicating processes, specialized languages, and hybrid techniques. For these seven classes, we discuss the main principles, give examples and discuss software tools that support the use of these techniques. An overview like this has not been presented in the literature before. Finally, the paper addresses the generation of the underlying MRM from the high-level specification, and indicates important future research areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ajmone Marsan, M., Conte, G., and Balbo, G. 1984. A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems.ACM Trans. Comput. Systems, 2(2), pp. 93–122.

    Article  Google Scholar 

  • Ammar, H.H., Islam, S.M.R., and Deng, S. 1989. Performability analysis of parallel and distributed algorithms.Proc. Third Int. Workshop Petri Nets and Performance Models, IEEE Computer Society Press, pp. 240–248.

  • Baskett, F., Muntz, R.R., Chandy, K.M., and Palacios, F.G. 1975. Open, closed and mixed networks of queues with different classes of customers.J. ACM, 22(2), pp. 248–260.

    Article  MATH  MathSciNet  Google Scholar 

  • Bavuso, S.J., Bechta Dugan, J., Trivedi, K.S., Rothmann, E.M., and Smith, W.E. 1987. Analysis of typical faulttolerant architectures using HARP.IEEE Trans. Reliability, 36(2), pp. 176–185.

    Article  Google Scholar 

  • Beilner, H. Mäter, J., and Weissenberg, N. 1989. Towards a performance modeling environment: news on HIT.Modeling Techniques and Tools for Computer Performance Evaluation (D. Potier and R. Puigjaner, eds.). New York: Plenum Press, pp. 57–75.

    Google Scholar 

  • Berson, S., de Souza e Silva, E., and Muntz, R.R. 1987. An object oriented methodology for the specification of Markov models.UCLA Technical Report CSD-870030.

  • Bobbio, A. 1989. Petri nets generating Markov reward models for performance/reliability analysis of degradable systems.Modeling Techniques and Tools for Computer Performance Evaluation (D. Potier and R. Puigjaner, eds.). New York: Plenum Press, pp. 353–365.

    Google Scholar 

  • Buchholz, P. 1991.Die strukturierte Analyse Markoffscher Modelle, Informatik Fachberichte 282, Springer-Verlag.

  • Butler, R.W. 1986. The SURE reliability analysis program. NASA Technical Memorandum 87593.

  • Carrasco, J.A. 1986. Modelation y Evaluacion de la Tolerancia a Fallos de Sistemas Distribuidos con Capacidad de Reconfiguracion. Ph.D. thesis, University of Catalunya, Spain.

    Google Scholar 

  • Carrasco, J.A., and Figueras, J. 1986. Metfac: design and implementation of a software tool for modeling and evaluation of complex fault-tolerant computing systems.Proc. FTCS 16, IEEE Computer Society Press, pp. 424–429.

  • Chiola, G. 1987. A graphical Petri net tool for performance analysis.Modeling Techniques and Performance Evaluation (S. Fdida and G. Pujolle, eds.). Amsterdam: North-Holland, pp. 323–333.

    Google Scholar 

  • Ciardo, G., Blakemore, A., Chimento, P.F.J., Muppala, J.K., and Trivedi, K.S. 1992. Automated Generation and analysis of Markov reward models using stochastic reward nets.Linear Algebra, Markov Chains, and Queuing Models (C. Meyer and R.J. Plemmons, eds.). Vol. 48 ofIMA Volumes in Mathematics and Its Applications, Springer-Verlag.

  • Ciardo, G., Muppala, J., and Trivedi, K.S. 1989. SPNP: stochastic Petri net package.Proc. Third Workshop Petri Nets and Performance Models, IEEE Computer Society Press, pp. 142–151.

  • Ciardo, G., Muppala, J.K., and Trivedi, K.S. 1991. “On the solution of GSPN reward models.Perform. Eval. 12(4), pp. 237–254.

    Article  MATH  Google Scholar 

  • Ciardo, G., and Trivedi, K.S. 1993. A decomposition approach for stochastic Petri net models. Perform. Eval. 18(1), pp. 37–59.

    Article  MATH  MathSciNet  Google Scholar 

  • Conway, A.E., and Goyal, A. 1987. Monte Carlo simulation of computer systems availability/reliability models.Proc. FTCS 17, IEEE Computer Society Press, pp. 230–235.

  • Couvillion, J.A., Freire, R., Johnson, R., Obal, W.D. II, Qureshi, A., Rai, M., Sanders, W.H., and Tvedt, J.E. 1991. Performability modeling with UltraSAN.IEEE Software, September, pp. 69–80.

  • Cumani, A. 1985. ESP—a package for the evaluation of stochastic Petri nets with phase-type distributed transition times.Proc. Int. Workshop Timed Petri Nets, IEEE Computer Society Press, pp. 144–151.

  • Diaz, M., Ansart, P., Courtait, J.-P., Azema, P., and Chari, V. (eds.) 1989.The Formal Description Technique ESTELLE. Amsterdam: North-Holland.

    MATH  Google Scholar 

  • Donatiello, L., and Iyer, B.R. 1987. Analysis of a composite performance reliability measure for fault-tolerant systems.J. ACM, 34(1), pp. 179–199.

    Article  MATH  MathSciNet  Google Scholar 

  • van Eijk, P.H.J., Vissers, C.A., and Diaz, M. (eds.). 1989.The Formal Description Technique LOTOS. Amsterdam: North-Holland.

    MATH  Google Scholar 

  • Geist, R., and Trivedi, K.S. 1990. Reliability estimation of fault-tolerant systems: tools and techniques.IEEE Comput. 23(7), pp. 52–61.

    Google Scholar 

  • Goyal, A., Carter, W.C., de Souza e Silva, E., Lavenberg, S.S., and Trivedi, K.S. 1986. The system availability estimator.Proc. FTCS 16, IEEE Computer Society Press, pp. 84–89.

  • Goyal, A., and Lavenberg, S.S. 1987. Modeling and analysis of computer system availability.IBM J. Res. Dev. 31(6), pp. 651–664.

    Article  Google Scholar 

  • Goyal, A., Lavenberg, S.S., and Trivedi, K.S. 1987. Probabilistic modeling of computer system availabilityAnn. Oper. Res. 8, pp. 285–306.

    Article  Google Scholar 

  • Goyal, A., Shahabuddin, P., Heidelberger, P., Nicola, V.F., and Glynn, P.W. 1992. A unified framework for simulating Markovian models of highly dependable systems.IEEE Trans. Comput. 41(1), pp. 36–51.

    Article  Google Scholar 

  • Goyal, A., and Tantawi, A.N. 1987. Evaluation of performability for degradable computer systems.IEEE Trans. Comput., 36(6), pp. 738–744.

    MATH  Google Scholar 

  • Gross, D., and Miller, D.R. 1984. The randomization technique as a modeling tool and solution procedure for transient Markov processes.Oper. Res., 32(2), pp. 343–361.

    Article  MATH  MathSciNet  Google Scholar 

  • Haverkort, B.R. 1993. Approximate performability and dependability modeling using generalized stochastic Petri nets. Perform. Eval. 18(1), pp. 61–78.

    Article  MATH  Google Scholar 

  • Haverkort, B.R. 1990. Performability modeling tools, evaluation techniques, and applications. Ph.D. thesis, University of Twente.

  • Haverkort, B.R., and Niemegeers, I.G., 1991. Performability modeling tools: a survey.Mem. Inform. 91–82, University of Twente.

  • Haverkort, B.R., and Niemegeers, I.G. 1989. Using dynamic queuing networks as a tool for specifying performability models.ACM Performan. Eval. Rev. 17(1), p. 225.

    Google Scholar 

  • Haverkort, B.R., Niemegeers, I.G., and Veldhuyzen van Zanten, P. 1992. DyQNtool—a performability modeling tool based on the dynamic queuing network concept.Computer Performance Evaluation: Modeling Techniques and Tools (G. Balbo and G. Serazzi, eds.). Amsterdam: North-Holland, pp. 181–195.

    Google Scholar 

  • Heck, E., Hogrefe, D., and Müller-Clostermann, B. 1991. Hierarchical performance evaluation based on form specified communication protocols.IEEE Trans. Comput., 40(4), pp. 500–513.

    Article  Google Scholar 

  • Hoare, C.A.R. 1985.Communicating Sequential Processes. Englewood Cliffs: Prentice-Hall.

    MATH  Google Scholar 

  • Howard, R.A. 1971.Dynamic Probabilisitc Systems. Vol. II. Semi-Markov and Decision Processes. New York: Wiley.

    Google Scholar 

  • Ibe, O.C. and Trivedi, K.S. 1990. Stochastic Petri net models of polling systems.IEEE J. Selected Areas Comm., 8(9), pp. 1649–1657.

    Article  Google Scholar 

  • Johnson, S.C., and Butler, R.W. 1988. Automated generation of reliability models.Proc. 1988 Ann. Reliability and Maintainability Symp., pp. 17–22.

  • Johnson, A.M., Jr., and Malek, M. 1988. Survey of software tools for evaluating reliability, availability, and serviceability.ACM Comput. Surveys, 20(4), pp. 227–269.

    Article  Google Scholar 

  • Klas, G.I., Seidel, U.G. 1992. A subsystem identification algorithm for the approximate solution of large GSPN models.Computer Performance Evaluation: Modeling Techniques and Tools (G. Balbo and G. Serazzi, eds.). Amsterdam: North-Holland, pp. 269–286.

    Google Scholar 

  • Krieger, U., Müller-Clostermann and Sczittnick, M. 1980. Modeling and analysis of communication systems based on computational methods for Markov chains.IEEE J. Areas Comm., 8(9), pp. 1630–1648.

    Article  Google Scholar 

  • Kulkarni, V.G., Nicola, V.F., and Trivedi, K.S. 1987. The completion time of a job on multimode systems.Adv. Appl. Probab., 19, pp. 932–954.

    Article  MATH  MathSciNet  Google Scholar 

  • Kulkarni, V.G., Nicola, V.F., and Trivedi K.S. 1990 Effects of checkpointing and queuing on program performance.Stoch. Models, 6(4), pp. 615–648.

    MATH  MathSciNet  Google Scholar 

  • Laprie, J.C. 1985. Dependable computing and fault-tolerance: concepts and terminology.Proc. FTCS 15, IEEE Computer Society Press, pp. 2–7.

  • Lazowska, E.D., Zahorjan, J., Graham, G.S., and Sevcik, K.C. 1984.Quantitative System Performance: Computer System Analysis Using Queuing Network Models. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Lepold, R. 1991a. Tomspin: Benutzerhandbuch. Internal report Siemens AG.

  • Lepold, R. 1991b. PENPET: a new approach to performability modeling using stochastic Petri nets.Proc. First Int. Workshop Performability Modeling of Computer and Communication Systems. (B.R. Haverkort, I.G. Niemegeers, and N.M. van Dijk, eds.). University of Twente, pp. 3–17.

  • Lepold, R. 1991c. Performability evaluation of a fault-tolerant computer system using stochastic Petri nets.Proc. Fifth Int. Conf. Fault-Tolerant Computing Systems, Nürnberg: Springer-Verlag.

    Google Scholar 

  • Li, V.O.K., and Silvester, J.A. 1984. Performance analysis of networks with unreliable components.IEEE Trans. Comm. 32(10), pp. 1105–1110.

    Article  Google Scholar 

  • Lindemann, C., and German, R., 1992. DSPNexpress: a software package for efficiently solving deterministic and stochastic Petri nets.Performance Tools 1992. (R. Pooley, and J. Hillston, eds.). Edinburgh University Press.

  • Meyer, J.F. 1980. On evaluating the performability of degradable computer systems.IEEE Trans. Comput. 29(8), pp. 720–731.

    MATH  Google Scholar 

  • Meyer, J.F. 1982. Closed-form solutions of performability.IEEE Trans. Comput. 31(7), pp. 648–657.

    Google Scholar 

  • Meyer, J.F. 1992. Performability: a retrospective and some pointers to the future.Perform. Eval., 14(3&4), pp. 139–156.

    Article  MATH  Google Scholar 

  • Meyer, J.F., Movaghar, A., and Sanders, W.H. 1985. Stochastic activity networks: structure, behavior, and application.Proc. Int. Workshop Timed Petri Nets, IEEE Computer Society Press, pp. 106–115.

  • Molloy, M.K. 1982. Performance analysis using stochastic Petri nets.IEEE Trans. Comput., 31(9), pp. 913–917.

    Google Scholar 

  • Mulazzani, M., and Trivedi, K.S. 1986. Dependability prediction: comparison of tools and techniques.Proc. IFAC SAFECOMP, pp. 171–178.

  • Müller-Clostermann, B. 1985. NUMAS—a tool for the numerical analysis of computer systems.Modeling Techniques and Tools for Computer Performance Analysis (D. Potier, ed.). Amsterdam: North-Holland, pp. 141–154.

    Google Scholar 

  • Muntz, R.R., de Souza e Silva, E., and Goyal, A. 1989. Bounding availability of repairable computer systems.IEEE Trans. Comput., 38(12), pp. 1714–1723.

    Article  MathSciNet  Google Scholar 

  • Muppala, J.K., Sathaye, A., Howe, R., and Trivedi, K.S. Forthcoming. Dependability modeling of a heterogeneous VAXcluster system using stochastic reward nets.Hardware and Software Fault Tolerance in Parallel Computing Systems (D. Averesky, ed.) London: Ellis Horwood.

  • Muppula, J.K., and Trivedi, K.S. 1992. Composite performance and availability analysis using a hierarchy of stochastic reward nets.Computer Performance Evaluation: Modeling Techniques and Tools (G. Balbo and G. Serazzi, eds.). Amsterdam: North-Holland, pp. 335–349.

    Google Scholar 

  • Neuts, M.F. 1981.Matrix Geometric Solutions in Stochastic Models—An Algorithmic Approach. Baltimore: The Johns Hopkins University Press.

    MATH  Google Scholar 

  • Neuts, M.F. 1989.Structured Matrices of M/G/1 Type and Their Applications. New York: Marcel Dekker.

    MATH  Google Scholar 

  • Page Jr., T.W., Berson, S.E., Cheng, W.C., and Muntz, R.R. 1989. An object-oriented modeling environment.ACM Sigplan Notices, 24(10), pp. 287–296.

    Article  Google Scholar 

  • Pattipati, K.R., Li, Y., and Blom, H.A.P. 1993. A unified framework for the performability evaluation of fault-tolerant computer systems. IEEE Trans. Comp., 42(3), pp. 312–326.

    Article  Google Scholar 

  • Plateau, B., Fourneau, J.-M., Lee, K.-H., 1990. PEPS: a package for solving complex Markov models of parallel systems.Modeling Techniques and Tools for Computer Performance Evaluation (E. Potier and R. Puigjaner eds.). pp. 291–305.

  • Potier, D. and Veran, M. 1986. The Markovian solver of QNAP2 and examples.Computer Networking and Performance Evaluation (T. Hasegawa, H. Takagi, and Y. Takahashi), pp. 259–279.

  • Reibman, A.L., Smith, R., and Trivedi, K.S. 1989. Markov and Markov reward models transient analysis: an overview of numerical approaches.Europ. J. Oper. Res. 40, pp. 257–267.

    Article  MATH  MathSciNet  Google Scholar 

  • Reibman, A.L., and Trivedi, K.S., 1988. Numerical transient analysis of Markov models.Comput. Oper. Res., 15(1), pp. 19–36.

    Article  MATH  Google Scholar 

  • Reibman, A.L., and Trivedi, K.S. 1989. Transient analysis of cumulative measures of Markov model behavior.Stoch. Models, 5(4), pp. 683–710.

    MATH  MathSciNet  Google Scholar 

  • Sahner, R.A., and Trivedi, K.S. 1987. Reliability modeling using SHARPE.IEEE Trans. Reliability 36(2), pp. 186–193.

    Google Scholar 

  • Sahner, R.A., and Trivedi, K.S. 1993. A software tool for learning about stochastic models.IEEE Trans. Educ., 36(1), pp. 56–61.

    Article  Google Scholar 

  • Sanders, W.H. 1988.Construction and solution of performability models based on stochastic activity networks. Ph.D. dissertation, University of Michigan.

  • Sanders, W.H., and Meyer, J.F. 1987. Performability evaluation of distributed systems using stochastic activity networks.Proc. 1987 Int. workshop Petri Nets and Performance Models, IEEE Computer Society Press, pp. 111–120.

  • Sanders, W.H., and Meyer, J.F. 1991. Reduced base model construction for stochastic activity network.IEEE J. Selected Areas Comm., 9(1), pp. 25–36.

    Article  Google Scholar 

  • Sczittnick, M., 1987.Techniken zur funktionalen und quantitativen Analyse von Markoffschen Rechensystemmodellen. M.Sc. thesis, University of Dortmund.

  • Sczittnick, M., and Müller-Clostermann, B. 1990. MACOM—a tool for the Markovian analysis of communication systems.Proc. Fourth Int. Conf. Data Communication Systems and Their Performance (R. Puigjaner, ed.), pp. 456–470.

  • Smith, R.M., Trivedi, K.S., and Ramesh, A.V. 1988. Performability analysis: measures, an algorithm and a case study.IEEE Trans. Comput. 37(4), pp. 406–417.

    Article  Google Scholar 

  • de Souza e Silva, E., and Gail, H.R. 1986. Calculating cumulative operational time distributions of repairable computer systems.IEEE Trans. Comput., 35(4), pp. 322–332.

    Google Scholar 

  • de Souza e Silva, E., and Gail, H.R., 1989. Calculating availability and performability measures of repairable computer systems using randomization.J. ACM 36(1), pp. 171–193.

    Article  MATH  Google Scholar 

  • de Souza e Silva, E., and Gail, H.R. 1992. Performability analysis of computer systems; from model specification to solution.Perform. Eval. 14(3&4), pp. 157–196.

    Article  MATH  Google Scholar 

  • de Souza e Silva, E., and Ochoa, P.M. 1992. State space exploration in Markov models.ACM Perform. Eval. Rev., 20(1), pp. 152–166.

    Google Scholar 

  • Stewart, W.J. 1991. MARCA: Markov chain analyzer.Numerical Solution of Markov Chains (W.J. Steward, ed.). New York: Marcel Dekker.

    Google Scholar 

  • Stewart, W.J., Goyal, A. 1985. Matrix methods in large dependability models.IBM Res. Report RC 11485.

  • Trivedi, K.S. 1982.Probability & Statistics with Reliability, Queuing, and Computer Science Applications. Englewood Cliffs, Prentice-Hall.

    Google Scholar 

  • Trivedi, K.S., Muppala J.K., Woolet, S.P., and Haverkort, B.R. 1992. Composite performance and dependability analysis.Perform. Eval., 14(3&4), pp. 197–215.

    Article  MATH  Google Scholar 

  • Veran, M., and Potier, D. 1985. A portable environment for queuing system modeling.Modeling Techniques and Tools for Computer Performance Evaluation (D. Potier, ed.) Amsterdam: North-Holland, pp. 25–63.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tele-Informatics and Open Systems, University of Twente, 7500 AE, Enschede, the Netherlands

    Boudewijn R. Haverkort

  2. Department of Electrical Engineering, Duke University, 27708-0291, Durham, NC

    Kishor S. Trivedi

Authors
  1. Boudewijn R. Haverkort
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kishor S. Trivedi
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This work was supported in part by the Naval Surface Warfare Center under contract N60921-92-C-0161 and by the National Science Foundation under grant CCR-9108114.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haverkort, B.R., Trivedi, K.S. Specification techniques for Markov reward models. Discrete Event Dyn Syst 3, 219–247 (1993). https://doi.org/10.1007/BF01439850

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01439850

Key Words

Search

Navigation