Classification of clinical alarms is at the heart of prioritization, suppression, integration, po... more Classification of clinical alarms is at the heart of prioritization, suppression, integration, postponement, and other methods of mitigating alarm fatigue. Since these methods directly affect clinical care, alarm classifiers, such as intelligent suppression systems, need to be evaluated in terms of their sensitivity and specificity, which is typically calculated on a labeled dataset of alarms. Unfortunately, the collection and particularly labeling of such datasets requires substantial effort and time, thus deterring hospitals from investigating mitigations of alarm fatigue. This paper develops a lightweight method for evaluating alarm classifiers without perfect alarm labels. The method relies on probabilistic labels obtained from data programming — a labeling paradigm based on combining noisy and cheap-to-obtain labeling heuristics. Based on these labels, the method produces confidence bounds for the sensitivity/specificity values from a hypothetical evaluation with manual labelin...
This archive contains the models and views for a case study of integrating power and planning mod... more This archive contains the models and views for a case study of integrating power and planning models in a power-aware mobile robot. It also contains an implementation of the Integration Property Language as a plugin for the OSATE2/Eclipse environment.<br>
Closed-loop verification of cyber-physical systems with neural network controllers offers strong ... more Closed-loop verification of cyber-physical systems with neural network controllers offers strong safety guarantees under certain assumptions. It is, however, difficult to determinewhether these guarantees apply at run time because verification assumptions may be violated. To predict safety violations in a verified system, we propose a three-step framework for monitoring the confidence in verification assumptions. First, we represent the sufficient condition for verified safety with a propositional logical formula over assumptions. Second, we build calibrated confidence monitors that evaluate the probability that each assumption holds. Third, we obtain the confidence in the verification guarantees by composing the assumption monitors using a composition function suitable for the logical formula. Our framework provides theoretical bounds on the calibration and conservatism of compositional monitors. In two case studies, we demonstrate that the composedmonitors improve over their const...
Ivan Ruchkin https://www.seas.upenn.edu/~iruchkin Our society is undergoing a profound change: co... more Ivan Ruchkin https://www.seas.upenn.edu/~iruchkin Our society is undergoing a profound change: connected computing systems are being deployed ubiquitously in our physical surroundings. A rapidly developing class of such systems, termed Cyber-Physical Systems (CPS, sometimes referred to as Robotic Systems), is illustrated in Figure 1. Enabled by advances in hardware, sensing, and artificial intelligence, these systems promise enormous economic returns with autonomous operation across a multitude of industries. The widespread use of these systems is critically dependent on their safety and trustworthiness, as well as their cost-effective engineering on a large scale. This engineering needs to overcome three complexity factors: Physicality: continual close interaction with the physical world Autonomy: purposeful behavior without direct human supervision Heterogeneity: mixture of component types and engineering techniques
Autonomous systems with machine learning-based perception can exhibit unpredictable behaviors tha... more Autonomous systems with machine learning-based perception can exhibit unpredictable behaviors that are difficult to quantify, let alone verify. Such behaviors are convenient to capture in probabilistic models, but probabilistic model checking of such models is difficult to scale — largely due to the non-determinism added to models as a prerequisite for provable conservatism. Statistical model checking (SMC) has been proposed to address the scalability issue. However it requires large amounts of data to account for the aforementioned non-determinism, which in turn limits its scalability. This work introduces a general technique for reduction of non-determinism based on assumptions of “monotonic safety”, which define a partial order between system states in terms of their probabilities of being safe. We exploit these assumptions to remove non-determinism from controller/plant models to drastically speed up probabilistic model checking and statistical model checking while providing pro...
This dataset includes the set of papers retrieved to perform a systematic mapping study (SMS) on ... more This dataset includes the set of papers retrieved to perform a systematic mapping study (SMS) on multi-paradigm modeling (MPM) for cyber-physical systems (CPS). Moreover, it includes several tables that map the literature into several perspectives, notably: used modeling formalisms and processes, part of the CPS addressed by the research, domain of expertise of paper authors, and relevance of the papers at review date. The set of papers is selected over a period ranging from 2006 to 2021, according to publication dates. The selection of the papers and their mapping has been performed by means of a rigorous process based on precise research questions and peer-review. Furthermore, the process has been supported by a web-based survey management application. Both the selection of existing publications and their mappings by means of the included perspectives provide interested readers/researchers with interesting data potentially re-usable for multiple purposes, notably: analyzing the pr...
Stochastic simulations of complex systems often rely on sampling dependent discrete random variab... more Stochastic simulations of complex systems often rely on sampling dependent discrete random variables. Currently, their users are limited in expressing their intention about how these variables are distributed and related to each other over time. This limitation leads the users to program complex and error-prone sampling algorithms. This paper introduces a way to specify, declaratively and precisely, a temporal distribution over discrete variables. Our tool Prospect infers and samples this distribution by solving a system of polynomial equations. The evaluation on three simulation scenarios shows that the declarative specifications are easier to write, 3x more succinct than imperative sampling programs, and are processed correctly by Prospect.
Technical report for activities of WG4 (delivery 4.4) within the context of MPM4CPS IC1404 COST A... more Technical report for activities of WG4 (delivery 4.4) within the context of MPM4CPS IC1404 COST Action (European supported by the EU Framework Programme Horizon 2020), in 2018.
This dataset includes the set of papers retrieved to perform a systematic mapping review (SMR) on... more This dataset includes the set of papers retrieved to perform a systematic mapping review (SMR) on multi-paradigm modelling (MPM) for cyber-physical systems (CPS). Moreover, it includes several tables that map the studies under several perspectives, notably used modelling formalisms and processes, part of the CPS addressed by the research, domain of expertise of paper authors, and relevance of the paper at review date. The set of papers is selected over a period ranging from 2006 to 2017, according to publication dates. The selection of the papers and their mapping has been performed by means of a rigorous process based on precise aspects to be evaluated and peer reviewing. Further, the process has been supported by a web-based survey management application. Both the selection of existing publications and their mappings by means of the included perspectives provide interested readers/researches with interesting data possible re-usable for multiple purposes: analysing the progress of ...
This archive contains hybrid programs (from a different study), their architectural views, analys... more This archive contains hybrid programs (from a different study), their architectural views, analysis data, and the source code of a hybrid program plugin for AcmeStudio.<br>Published as Ivan Ruchkin, Bradley Schmerl, David Garlan.Architectural Abstractions for Hybrid Programs.In Proceedings of the 18th International Symposium on Component-Based Software Engineering (CBSE). Montreal, Canada, 2015.
This archive contains the source code of the ACTIVE tool, and models/data from a case study of an... more This archive contains the source code of the ACTIVE tool, and models/data from a case study of analysis contracts in two domains: thread scheduling, and battery design.<br><br><br>
Designing secure cyber-physical systems (CPS) is a partic-ularly difficult task since security vu... more Designing secure cyber-physical systems (CPS) is a partic-ularly difficult task since security vulnerabilities stem not only from traditional cybersecurity concerns, but also phys-ical ones. Many of the standard methods for CPS design make strong and unverified assumptions about the trust-worthiness of physical devices, such as sensors. When these assumptions are violated, subtle inter-domain vulnerabili-ties are introduced into the system model. In this paper we use formal specification of analysis contracts to expose secu-rity assumptions and guarantees of analyses from reliability, control, and sensor security domains. We show that this specification allows us to determine where these assump-tions are violated, opening the door to malicious attacks. We demonstrate how this approach can help discover and prevent vulnerabilities using a self-driving car example.
Classification of clinical alarms is at the heart of prioritization, suppression, integration, po... more Classification of clinical alarms is at the heart of prioritization, suppression, integration, postponement, and other methods of mitigating alarm fatigue. Since these methods directly affect clinical care, alarm classifiers, such as intelligent suppression systems, need to be evaluated in terms of their sensitivity and specificity, which is typically calculated on a labeled dataset of alarms. Unfortunately, the collection and particularly labeling of such datasets requires substantial effort and time, thus deterring hospitals from investigating mitigations of alarm fatigue. This paper develops a lightweight method for evaluating alarm classifiers without perfect alarm labels. The method relies on probabilistic labels obtained from data programming — a labeling paradigm based on combining noisy and cheap-to-obtain labeling heuristics. Based on these labels, the method produces confidence bounds for the sensitivity/specificity values from a hypothetical evaluation with manual labelin...
This archive contains the models and views for a case study of integrating power and planning mod... more This archive contains the models and views for a case study of integrating power and planning models in a power-aware mobile robot. It also contains an implementation of the Integration Property Language as a plugin for the OSATE2/Eclipse environment.<br>
Closed-loop verification of cyber-physical systems with neural network controllers offers strong ... more Closed-loop verification of cyber-physical systems with neural network controllers offers strong safety guarantees under certain assumptions. It is, however, difficult to determinewhether these guarantees apply at run time because verification assumptions may be violated. To predict safety violations in a verified system, we propose a three-step framework for monitoring the confidence in verification assumptions. First, we represent the sufficient condition for verified safety with a propositional logical formula over assumptions. Second, we build calibrated confidence monitors that evaluate the probability that each assumption holds. Third, we obtain the confidence in the verification guarantees by composing the assumption monitors using a composition function suitable for the logical formula. Our framework provides theoretical bounds on the calibration and conservatism of compositional monitors. In two case studies, we demonstrate that the composedmonitors improve over their const...
Ivan Ruchkin https://www.seas.upenn.edu/~iruchkin Our society is undergoing a profound change: co... more Ivan Ruchkin https://www.seas.upenn.edu/~iruchkin Our society is undergoing a profound change: connected computing systems are being deployed ubiquitously in our physical surroundings. A rapidly developing class of such systems, termed Cyber-Physical Systems (CPS, sometimes referred to as Robotic Systems), is illustrated in Figure 1. Enabled by advances in hardware, sensing, and artificial intelligence, these systems promise enormous economic returns with autonomous operation across a multitude of industries. The widespread use of these systems is critically dependent on their safety and trustworthiness, as well as their cost-effective engineering on a large scale. This engineering needs to overcome three complexity factors: Physicality: continual close interaction with the physical world Autonomy: purposeful behavior without direct human supervision Heterogeneity: mixture of component types and engineering techniques
Autonomous systems with machine learning-based perception can exhibit unpredictable behaviors tha... more Autonomous systems with machine learning-based perception can exhibit unpredictable behaviors that are difficult to quantify, let alone verify. Such behaviors are convenient to capture in probabilistic models, but probabilistic model checking of such models is difficult to scale — largely due to the non-determinism added to models as a prerequisite for provable conservatism. Statistical model checking (SMC) has been proposed to address the scalability issue. However it requires large amounts of data to account for the aforementioned non-determinism, which in turn limits its scalability. This work introduces a general technique for reduction of non-determinism based on assumptions of “monotonic safety”, which define a partial order between system states in terms of their probabilities of being safe. We exploit these assumptions to remove non-determinism from controller/plant models to drastically speed up probabilistic model checking and statistical model checking while providing pro...
This dataset includes the set of papers retrieved to perform a systematic mapping study (SMS) on ... more This dataset includes the set of papers retrieved to perform a systematic mapping study (SMS) on multi-paradigm modeling (MPM) for cyber-physical systems (CPS). Moreover, it includes several tables that map the literature into several perspectives, notably: used modeling formalisms and processes, part of the CPS addressed by the research, domain of expertise of paper authors, and relevance of the papers at review date. The set of papers is selected over a period ranging from 2006 to 2021, according to publication dates. The selection of the papers and their mapping has been performed by means of a rigorous process based on precise research questions and peer-review. Furthermore, the process has been supported by a web-based survey management application. Both the selection of existing publications and their mappings by means of the included perspectives provide interested readers/researchers with interesting data potentially re-usable for multiple purposes, notably: analyzing the pr...
Stochastic simulations of complex systems often rely on sampling dependent discrete random variab... more Stochastic simulations of complex systems often rely on sampling dependent discrete random variables. Currently, their users are limited in expressing their intention about how these variables are distributed and related to each other over time. This limitation leads the users to program complex and error-prone sampling algorithms. This paper introduces a way to specify, declaratively and precisely, a temporal distribution over discrete variables. Our tool Prospect infers and samples this distribution by solving a system of polynomial equations. The evaluation on three simulation scenarios shows that the declarative specifications are easier to write, 3x more succinct than imperative sampling programs, and are processed correctly by Prospect.
Technical report for activities of WG4 (delivery 4.4) within the context of MPM4CPS IC1404 COST A... more Technical report for activities of WG4 (delivery 4.4) within the context of MPM4CPS IC1404 COST Action (European supported by the EU Framework Programme Horizon 2020), in 2018.
This dataset includes the set of papers retrieved to perform a systematic mapping review (SMR) on... more This dataset includes the set of papers retrieved to perform a systematic mapping review (SMR) on multi-paradigm modelling (MPM) for cyber-physical systems (CPS). Moreover, it includes several tables that map the studies under several perspectives, notably used modelling formalisms and processes, part of the CPS addressed by the research, domain of expertise of paper authors, and relevance of the paper at review date. The set of papers is selected over a period ranging from 2006 to 2017, according to publication dates. The selection of the papers and their mapping has been performed by means of a rigorous process based on precise aspects to be evaluated and peer reviewing. Further, the process has been supported by a web-based survey management application. Both the selection of existing publications and their mappings by means of the included perspectives provide interested readers/researches with interesting data possible re-usable for multiple purposes: analysing the progress of ...
This archive contains hybrid programs (from a different study), their architectural views, analys... more This archive contains hybrid programs (from a different study), their architectural views, analysis data, and the source code of a hybrid program plugin for AcmeStudio.<br>Published as Ivan Ruchkin, Bradley Schmerl, David Garlan.Architectural Abstractions for Hybrid Programs.In Proceedings of the 18th International Symposium on Component-Based Software Engineering (CBSE). Montreal, Canada, 2015.
This archive contains the source code of the ACTIVE tool, and models/data from a case study of an... more This archive contains the source code of the ACTIVE tool, and models/data from a case study of analysis contracts in two domains: thread scheduling, and battery design.<br><br><br>
Designing secure cyber-physical systems (CPS) is a partic-ularly difficult task since security vu... more Designing secure cyber-physical systems (CPS) is a partic-ularly difficult task since security vulnerabilities stem not only from traditional cybersecurity concerns, but also phys-ical ones. Many of the standard methods for CPS design make strong and unverified assumptions about the trust-worthiness of physical devices, such as sensors. When these assumptions are violated, subtle inter-domain vulnerabili-ties are introduced into the system model. In this paper we use formal specification of analysis contracts to expose secu-rity assumptions and guarantees of analyses from reliability, control, and sensor security domains. We show that this specification allows us to determine where these assump-tions are violated, opening the door to malicious attacks. We demonstrate how this approach can help discover and prevent vulnerabilities using a self-driving car example.
Uploads
Papers by Ivan Ruchkin