research-article

Probabilistic-WCET Reliability: On the experimental validation of EVT hypotheses

Authors:

Federico Reghenzani,

Giuseppe Massari,

William Fornaciari,

Andrea GalimbertiAuthors Info & Claims

COINS '19: Proceedings of the International Conference on Omni-Layer Intelligent Systems

Pages 229 - 234

https://doi.org/10.1145/3312614.3312660

Published: 05 May 2019 Publication History

Abstract

The interest in probabilistic real-time is increasing, in response to the lack of traditional static WCET analysis methods for applications running on complex systems, like multi/many-cores and COTS platforms. However, the probabilistic theory is still immature and, furthermore, it requires strong guarantees on the timing traces, in order to provide safe probabilistic-WCET estimations. These requirements can be verified with appropriate statistical tests, as described in this paper, and tested with synthetic and realistic sources, to assess their ability to detect unreliable results. In this work, we identified also the challenges and the problems of using statistical test based procedures for probabilistic real-time computing.

References

[1]

J. Abella, D. Hardy, I. Puaut, E. Qui nones, and F. J. Cazorla. 2014. On the Comparison of Deterministic and Probabilistic WCET Estimation Techniques. In 2014 26th Euromicro Conference on Real-Time Systems. IEEE, 266--275.

Digital Library

[2]

Jaume Abella, Maria Padilla, Joan Del Castillo, and Francisco J. Cazorla. 2017. Measurement-Based Worst-Case Execution Time Estimation Using the Coefficient of Variation. ACM Trans. Des. Autom. Electron. Syst. 22, 4, Article 72 (June 2017), 29 pages.

Digital Library

[3]

R. Bender and S. Lange. 2001. Adjusting for multiple testing - when and how? Journal of Clinical Epidemiology 54, 4 (April 2001), 343--349.

[4]

G. Bernat, A. Colin, and S. M. Petters. 2002. WCET analysis of probabilistic hard real-time systems. In 23rd IEEE Real-Time Systems Symposium, 2002. RTSS 2002. IEEE, 279--288.

Digital Library

[5]

W. A. Broock, J. A. Scheinkman, W. D. Dechert, and B. LeBaron. 1996. A test for independence based on the correlation dimension. Econometric Reviews 15, 3 (1996), 197--235.

[6]

A. Burns and S. Edgar. 2000. Predicting computation time for advanced processor architectures. In Proceedings 12th Euromicro Conference on Real-Time Systems. Euromicro RTS 2000. 89--96.

Digital Library

[7]

Enrique Castillo, Ali S Hadi, Narayanaswamy Balakrishnan, and José-Mariá Sarabia. 2005. Extreme value and related models with applications in engineering and science. Wiley Hoboken, NJ.

[8]

F.J. Cazorla, E. Quiñones, T. Vardanega, L. Cucu, B. Triquet, G. Bernat, E. Berger, J. Abella, F. Wartel, M. Houston, L. Santinelli, L. Kosmidis, C. Lo, and D. Maxim. 2013. PROARTIS: Probabilistically Analyzable Real-Time Systems. ACM Trans. Embed. Comput. Syst. 12, 2s, Article 94 (May 2013), 26 pages.

Digital Library

[9]

Michel Couillard and Matt Davison. 2005. A comment on measuring the Hurst exponent of financial time series. Physica A: Statistical Mechanics and its Applications 348 (2005), 404--418.

[10]

R. A. Fisher and L. H. C. Tippett. 1928. Limiting forms of the frequency distribution of the largest or smallest member of a sample. Mathematical Proceedings of the Cambridge Philosophical Society 24, 2 (1928), 180--190.

[11]

S.Jiménez Gil, I. Bate, G. Lima, L. Santinelli, A. Gogonel, and L. Cucu-Grosjean. 2017. Open Challenges for Probabilistic Measurement-Based Worst-Case Execution Time. IEEE Embedded Systems Letters 9, 3 (Sept 2017), 69--72.

[12]

Jan Gustafsson, Adam Betts, Andreas Ermedahl, and Björn Lisper. 2010. The Mälardalen WCET Benchmarks -- Past, Present and Future. In 10th International Workshop on Worst-Case Execution Time Analysis, WCET 2010, July 6, 2010, Brussels, Belgium, Björn Lisper (Ed.). OCG, Brussels, Belgium, 137--147.

[13]

H. E. HURST. 1951. Long term storage capacity of reservoirs. ASCE Transactions 116, 776 (1951), 770--808. https://ci.nii.ac.jp/naid/10011004012/en/

[14]

Frank J. Massey Jr. 1951. The Kolmogorov-Smirnov Test for Goodness of Fit. J. Amer. Statist. Assoc. 46, 253 (1951), 68--78.

[15]

R. Kirner and P. Puschner. 2008. Obstacles inWorst-Case Execution Time Analysis. In 2008 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC). 333--339.

Digital Library

[16]

L. Kosmidis, C. Curtsinger, E. QuiÃśones, J. Abella, E. Berger, and F.J. Cazorla. 2013. Probabilistic timing analysis on conventional cache designs. In 2013 Design, Automation Test in Europe Conference Exhibition (DATE). 603--606.

Digital Library

[17]

O. Kotaba, J. Nowotsch, M. Paulitsch, S. M. Petters, and H. Theiling. 2013. Multicore in real-time systems--temporal isolation challenges due to shared resources. In Workshop on Industry-Driven Approaches for Cost-effective Certification of Safety-Critical, Mixed-Criticality Systems. Grenoble, France, 6.

[18]

Denis Kwiatkowski, Peter C.B. Phillips, Peter Schmidt, and Yongcheol Shin. 1992. Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root? Journal of Econometrics 54, 1 (1992), 159--178.

[19]

M. R. Leadbetter and Holger Rootzen. 1988. Extremal Theory for Stochastic Processes. Ann. Probab. 16, 2 (04 1988), 431--478.

[20]

G. Lima and I. Bate. 2017. Valid Application of EVT in Timing Analysis by Randomising Execution Time Measurements. In 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). 187--198.

[21]

G. Lima, D. Dias, and E. Barros. 2016. Extreme Value Theory for Estimating Task Execution Time Bounds: A Careful Look. In 2016 28th Euromicro Conference on Real-Time Systems (ECRTS). 200--211.

[22]

Luiz Renato Lima and Breno Neri. 2013. A Test for Strict Stationarity. In Uncertainty Analysis in Econometrics with Applications, Van-Nam Huynh, Vladik Kreinovich, Songsak Sriboonchitta, and Komsan Suriya (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 17--30.

[23]

Bo Qian and Khaled Rasheed. 2004. Hurst exponent and financial market predictability. In Proceedings of the Second IASTED International Conference on Financial Engineering and Applications.

[24]

Petar Radojković, Sylvain Girbal, Arnaud Grasset, Eduardo Quiñones, Sami Yehia, and Francisco J. Cazorla. 2012. On the Evaluation of the Impact of Shared Resources in Multithreaded COTS Processors in Time-critical Environments. ACM Trans. Archit. Code Optim. 8, 4, Article 34 (Jan. 2012), 25 pages.

Digital Library

[25]

F. Reghenzani, G. Massari, and W. Fornaciari. 2017. Mixed Time-Criticality Process Interferences Characterization on a Multicore Linux System. In 2017 Euromicro Conference on Digital System Design (DSD). IEEE, Wien, 427--434.

[26]

F. Reghenzani, G. Massari, and W. Fornaciari. 2018. chronovise: Measurement-Based Probabilistic Timing Analysis framework. Journal of Open Source Software 3, 28 (2018), 711.

[27]

F. Reghenzani, G. Massari, and W. Fornaciari. 2018. The Misconception of Exponential Tail Upper-Bounding in Probabilistic Real-Time. IEEE Embedded Systems Letters (2018), 1--1.

[28]

Jan Reineke. 2014. Randomized Caches Considered Harmful in Hard Real-Time Systems. Leibniz Transactions on Embedded Systems 1, 1 (2014), 03:1--03:13.

[29]

R.D. Reiss and M. Thomas. 2007. Statistical Analysis of Extreme Values: with Applications to Insurance, Finance, Hydrology and Other Fields. Birkhäuser Basel. https://books.google.it/books?id=I-g-I_I2OZIC

[30]

L. Santinelli, F. Guet, and J. Morio. 2017. Revising Measurement-Based Probabilistic Timing Analysis. In 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). 199--208.

[31]

L. Santinelli, F. Guet, and J. Morio. 2017. Revising Measurement-Based Probabilistic Timing Analysis. In 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). 199--208.

[32]

Luca Santinelli, Jérôme Morio, Guillaume Dufour, and Damien Jacquemart. 2014. On the Sustainability of the Extreme Value Theory for WCET Estimation. In 14th International Workshop on Worst-Case Execution Time Analysis (OpenAccess Series in Informatics (OASIcs)), Vol. 39. 21--30.

[33]

M. A. Stephens. 1974. EDF Statistics for Goodness of Fit and Some Comparisons. J. Amer. Statist. Assoc. 69, 347 (1974), 730--737.

[34]

Federico Terraneo. 2008. Miosix embedded OS. http://miosix.org.

Cited By

Christmann AHapka RErnst R(2023)Formal Analysis of Timing Diversity for Autonomous Systems2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137030(1-6)Online publication date: Apr-2023
https://doi.org/10.23919/DATE56975.2023.10137030
Hapka RChristmann AErnst RKuzolap AHecker PRockschies MHalle MThielecke F(2023)Safe Usage of Multi-Cores in Neural Network Avionics Applications2023 IEEE/AIAA 42nd Digital Avionics Systems Conference (DASC)10.1109/DASC58513.2023.10311131(1-9)Online publication date: 1-Oct-2023
https://doi.org/10.1109/DASC58513.2023.10311131
Reghenzani F(2022)Beyond the Traditional Analyses and Resource Management in Real-Time SystemsSpecial Topics in Information Technology10.1007/978-3-030-85918-3_6(67-77)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-85918-3_6
Show More Cited By

Index Terms

Probabilistic-WCET Reliability: On the experimental validation of EVT hypotheses
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems
  2. Real-time systems

Recommendations

Estimating Probabilistic Safe WCET Ranges of Real-Time Systems at Design Stages
Estimating worst-case execution time (WCET) is an important activity at early design stages of real-time systems. Based on WCET estimates, engineers make design and implementation decisions to ensure that task executions always complete before their ...
Dealing with Uncertainty in pWCET Estimations
Special Issue on LCETES, Part 1, Real-Time, Critical Systems, and Approximation

The problem of estimating a tight and safe Worst-Case Execution Time (WCET), needed for certification in safety-critical environment, is a challenging problem for modern embedded systems. A possible solution proposed in past years is to exploit ...
On the Reliability and Tightness of GP and Exponential Models for Probabilistic WCET Estimation

As computer architectures evolve, guaranteeing that Real-Time Systems’ (RTSs’) timing requirements are met through Worst Case Execution Time (WCET) upper bounds becomes increasingly difficult. Techniques such as Measurement-Based Probabilistic Timing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

COINS '19: Proceedings of the International Conference on Omni-Layer Intelligent Systems

May 2019

241 pages

ISBN:9781450366403

DOI:10.1145/3312614

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020 Framework Programme

Conference

COINS '19

COINS '19: INTERNATIONAL CONFERENCE ON OMNI-LAYER INTELLIGENT SYSTEMS

May 5 - 7, 2019

Crete, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
169
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Christmann AHapka RErnst R(2023)Formal Analysis of Timing Diversity for Autonomous Systems2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137030(1-6)Online publication date: Apr-2023
https://doi.org/10.23919/DATE56975.2023.10137030
Hapka RChristmann AErnst RKuzolap AHecker PRockschies MHalle MThielecke F(2023)Safe Usage of Multi-Cores in Neural Network Avionics Applications2023 IEEE/AIAA 42nd Digital Avionics Systems Conference (DASC)10.1109/DASC58513.2023.10311131(1-9)Online publication date: 1-Oct-2023
https://doi.org/10.1109/DASC58513.2023.10311131
Reghenzani F(2022)Beyond the Traditional Analyses and Resource Management in Real-Time SystemsSpecial Topics in Information Technology10.1007/978-3-030-85918-3_6(67-77)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-85918-3_6
Amalou APuaut IMuller G(2021)WE-HML: hybrid WCET estimation using machine learning for architectures with caches2021 IEEE 27th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)10.1109/RTCSA52859.2021.00011(31-40)Online publication date: Aug-2021
https://doi.org/10.1109/RTCSA52859.2021.00011
Arcaro LSilva Kde Oliveira RAlmeida L(2020)Reliability Test based on a Binomial Experiment for Probabilistic Worst-Case Execution Times2020 IEEE Real-Time Systems Symposium (RTSS)10.1109/RTSS49844.2020.00016(51-62)Online publication date: Dec-2020
https://doi.org/10.1109/RTSS49844.2020.00016
Reghenzani FMassari GFornaciari W(2020)Timing Predictability in High-Performance Computing with Probabilistic Real-TimeIEEE Access10.1109/ACCESS.2020.3038559(1-1)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3038559
Agosta GFornaciari WAtienza DCanal RCilardo AFlich Cardo JHernandez Luz CKulczewski MMassari GTornero Gavilá RZapater M(2020)The RECIPE approach to challenges in deeply heterogeneous high performance systemsMicroprocessors & Microsystems10.1016/j.micpro.2020.10318577:COnline publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1016/j.micpro.2020.103185
Reghenzani FSantinelli LFornaciari W(2019)Why statistical power matters for probabilistic real-timeProceedings of the International Conference on Embedded Software Companion10.1145/3349568.3351555(1-2)Online publication date: 13-Oct-2019
https://dl.acm.org/doi/10.1145/3349568.3351555
Reghenzani FMassari GFornaciari W(2019)A Probabilistic Approach to Energy-Constrained Mixed-Criticality Systems2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)10.1109/ISLPED.2019.8824991(1-6)Online publication date: Jul-2019
https://doi.org/10.1109/ISLPED.2019.8824991
Agosta GFornaciari WAtienza DCanal RCilardo AFlich JHernandez Luz CKulczewski MMassari GTornero Gavila RZapater Sancho M(2019)Challenges in Deeply Heterogeneous High Performance Systems2019 22nd Euromicro Conference on Digital System Design (DSD)10.1109/DSD.2019.00068(428-435)Online publication date: Aug-2019
https://doi.org/10.1109/DSD.2019.00068
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten