-
Enhancing Data Privacy in Large Language Models through Private Association Editing
Authors:
Davide Venditti,
Elena Sofia Ruzzetti,
Giancarlo A. Xompero,
Cristina Giannone,
Andrea Favalli,
Raniero Romagnoli,
Fabio Massimo Zanzotto
Abstract:
Large Language Models (LLMs) are powerful tools with extensive applications, but their tendency to memorize private information raises significant concerns as private data leakage can easily happen. In this paper, we introduce Private Association Editing (PAE), a novel defense approach for private data leakage. PAE is designed to effectively remove Personally Identifiable Information (PII) without…
▽ More
Large Language Models (LLMs) are powerful tools with extensive applications, but their tendency to memorize private information raises significant concerns as private data leakage can easily happen. In this paper, we introduce Private Association Editing (PAE), a novel defense approach for private data leakage. PAE is designed to effectively remove Personally Identifiable Information (PII) without retraining the model. Our approach consists of a four-step procedure: detecting memorized PII, applying PAE cards to mitigate memorization of private data, verifying resilience to targeted data extraction (TDE) attacks, and ensuring consistency in the post-edit LLMs. The versatility and efficiency of PAE, which allows for batch modifications, significantly enhance data privacy in LLMs. Experimental results demonstrate the effectiveness of PAE in mitigating private data leakage. We believe PAE will serve as a critical tool in the ongoing effort to protect data privacy in LLMs, encouraging the development of safer models for real-world applications.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Building Hybrid B-Spline And Neural Network Operators
Authors:
Raffaele Romagnoli,
Jasmine Ratchford,
Mark H. Klein
Abstract:
Control systems are indispensable for ensuring the safety of cyber-physical systems (CPS), spanning various domains such as automobiles, airplanes, and missiles. Safeguarding CPS necessitates runtime methodologies that continuously monitor safety-critical conditions and respond in a verifiably safe manner. A fundamental aspect of many safety approaches involves predicting the future behavior of sy…
▽ More
Control systems are indispensable for ensuring the safety of cyber-physical systems (CPS), spanning various domains such as automobiles, airplanes, and missiles. Safeguarding CPS necessitates runtime methodologies that continuously monitor safety-critical conditions and respond in a verifiably safe manner. A fundamental aspect of many safety approaches involves predicting the future behavior of systems. However, achieving this requires accurate models that can operate in real time. Motivated by DeepONets, we propose a novel strategy that combines the inductive bias of B-splines with data-driven neural networks to facilitate real-time predictions of CPS behavior. We introduce our hybrid B-spline neural operator, establishing its capability as a universal approximator and providing rigorous bounds on the approximation error. These findings are applicable to a broad class of nonlinear autonomous systems and are validated through experimentation on a controlled 6-degree-of-freedom (DOF) quadrotor with a 12 dimensional state space. Furthermore, we conduct a comparative analysis of different network architectures, specifically fully connected networks (FCNN) and recurrent neural networks (RNN), to elucidate the practical utility and trade-offs associated with each architecture in real-world scenarios.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Multi-Agent Reinforcement Learning with Control-Theoretic Safety Guarantees for Dynamic Network Bridging
Authors:
Raffaele Galliera,
Konstantinos Mitsopoulos,
Niranjan Suri,
Raffaele Romagnoli
Abstract:
Addressing complex cooperative tasks in safety-critical environments poses significant challenges for Multi-Agent Systems, especially under conditions of partial observability. This work introduces a hybrid approach that integrates Multi-Agent Reinforcement Learning with control-theoretic methods to ensure safe and efficient distributed strategies. Our contributions include a novel setpoint update…
▽ More
Addressing complex cooperative tasks in safety-critical environments poses significant challenges for Multi-Agent Systems, especially under conditions of partial observability. This work introduces a hybrid approach that integrates Multi-Agent Reinforcement Learning with control-theoretic methods to ensure safe and efficient distributed strategies. Our contributions include a novel setpoint update algorithm that dynamically adjusts agents' positions to preserve safety conditions without compromising the mission's objectives. Through experimental validation, we demonstrate significant advantages over conventional MARL strategies, achieving comparable task performance with zero safety violations. Our findings indicate that integrating safe control with learning approaches not only enhances safety compliance but also achieves good performance in mission objectives.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
Investigating the Impact of Data Contamination of Large Language Models in Text-to-SQL Translation
Authors:
Federico Ranaldi,
Elena Sofia Ruzzetti,
Dario Onorati,
Leonardo Ranaldi,
Cristina Giannone,
Andrea Favalli,
Raniero Romagnoli,
Fabio Massimo Zanzotto
Abstract:
Understanding textual description to generate code seems to be an achieved capability of instruction-following Large Language Models (LLMs) in zero-shot scenario. However, there is a severe possibility that this translation ability may be influenced by having seen target textual descriptions and the related code. This effect is known as Data Contamination.
In this study, we investigate the impac…
▽ More
Understanding textual description to generate code seems to be an achieved capability of instruction-following Large Language Models (LLMs) in zero-shot scenario. However, there is a severe possibility that this translation ability may be influenced by having seen target textual descriptions and the related code. This effect is known as Data Contamination.
In this study, we investigate the impact of Data Contamination on the performance of GPT-3.5 in the Text-to-SQL code-generating tasks. Hence, we introduce a novel method to detect Data Contamination in GPTs and examine GPT-3.5's Text-to-SQL performances using the known Spider Dataset and our new unfamiliar dataset Termite. Furthermore, we analyze GPT-3.5's efficacy on databases with modified information via an adversarial table disconnection (ATD) approach, complicating Text-to-SQL tasks by removing structural pieces of information from the database. Our results indicate a significant performance drop in GPT-3.5 on the unfamiliar Termite dataset, even with ATD modifications, highlighting the effect of Data Contamination on LLMs in Text-to-SQL translation tasks.
△ Less
Submitted 12 February, 2024;
originally announced February 2024.
-
Reinforcement Learning-based Optimal Control and Software Rejuvenation for Safe and Efficient UAV Navigation
Authors:
Angela Chen,
Konstantinos Mitsopoulos,
Raffaele Romagnoli
Abstract:
Unmanned autonomous vehicles (UAVs) rely on effective path planning and tracking control to accomplish complex tasks in various domains. Reinforcement Learning (RL) methods are becoming increasingly popular in control applications, as they can learn from data and deal with unmodelled dynamics. Cyber-physical systems (CPSs), such as UAVs, integrate sensing, network communication, control, and compu…
▽ More
Unmanned autonomous vehicles (UAVs) rely on effective path planning and tracking control to accomplish complex tasks in various domains. Reinforcement Learning (RL) methods are becoming increasingly popular in control applications, as they can learn from data and deal with unmodelled dynamics. Cyber-physical systems (CPSs), such as UAVs, integrate sensing, network communication, control, and computation to solve challenging problems. In this context, Software Rejuvenation (SR) is a protection mechanism that refreshes the control software to mitigate cyber-attacks, but it can affect the tracking controller's performance due to discrepancies between the control software and the physical system state. Traditional approaches to mitigate this effect are conservative, hindering the overall system performance. In this paper, we propose a novel approach that incorporates Deep Reinforcement Learning (Deep RL) into SR to design a safe and high-performing tracking controller. Our approach optimizes safety and performance, and we demonstrate its effectiveness during UAV simulations. We compare our approach with traditional methods and show that it improves the system's performance while maintaining safety constraints.
△ Less
Submitted 27 August, 2023;
originally announced August 2023.
-
Reducing Attack Opportunities Through Decentralized Event-Triggered Control
Authors:
Paul Griffioen,
Raffaele Romagnoli,
Bruce H. Krogh,
Bruno Sinopoli
Abstract:
Decentralized control systems are widely used in a number of situations and applications. In order for these systems to function properly and achieve their desired goals, information must be propagated between agents, which requires connecting to a network. To reduce opportunities for attacks that may be carried out through the network, we design an event-triggered mechanism for network connection…
▽ More
Decentralized control systems are widely used in a number of situations and applications. In order for these systems to function properly and achieve their desired goals, information must be propagated between agents, which requires connecting to a network. To reduce opportunities for attacks that may be carried out through the network, we design an event-triggered mechanism for network connection and communication that minimizes the amount of time agents must be connected to the network, in turn decreasing communication costs. This mechanism is a function of only local information and ensures stability for the overall system in attack-free scenarios. Our approach distinguishes itself from current decentralized event-triggered control strategies by considering scenarios where agents are not always connected to the network to receive critical information from other agents and by considering scenarios where the communication graph is undirected and connected. An algorithm describing this network connection and communication protocol is provided, and our approach is illustrated via simulation.
△ Less
Submitted 30 July, 2022;
originally announced August 2022.
-
Exploring the consequences of cyber attacks on Powertrain Cyber Physical Systems
Authors:
Dario Stabili,
Raffaele Romagnoli,
Mirco Marchetti,
Bruno Sinopoli,
Michele Colajanni
Abstract:
This paper proposes a novel approach for the study of cyber-attacks against the powertrain of a generic vehicle. The proposed model is composed by a a generic Internal Combustion engine and a speed controller, that communicate through a Controller Area Network (CAN) bus. We consider a threat model composed by three representative attack scenarios designed to modify the output of the model, thus af…
▽ More
This paper proposes a novel approach for the study of cyber-attacks against the powertrain of a generic vehicle. The proposed model is composed by a a generic Internal Combustion engine and a speed controller, that communicate through a Controller Area Network (CAN) bus. We consider a threat model composed by three representative attack scenarios designed to modify the output of the model, thus affecting the rotational speed of the engine. Two attack scenarios target both vehicle sensor systems and CAN communication, while one attack scenario only requires injection of CAN messages. To the best of our knowledge, this is the first attempt of modeling the consequences of realistic cyber attacks against a modern vehicle.
△ Less
Submitted 1 February, 2022;
originally announced February 2022.
-
Every time I fire a conversational designer, the performance of the dialog system goes down
Authors:
Giancarlo A. Xompero,
Michele Mastromattei,
Samir Salman,
Cristina Giannone,
Andrea Favalli,
Raniero Romagnoli,
Fabio Massimo Zanzotto
Abstract:
Incorporating explicit domain knowledge into neural-based task-oriented dialogue systems is an effective way to reduce the need of large sets of annotated dialogues. In this paper, we investigate how the use of explicit domain knowledge of conversational designers affects the performance of neural-based dialogue systems. To support this investigation, we propose the Conversational-Logic-Injection-…
▽ More
Incorporating explicit domain knowledge into neural-based task-oriented dialogue systems is an effective way to reduce the need of large sets of annotated dialogues. In this paper, we investigate how the use of explicit domain knowledge of conversational designers affects the performance of neural-based dialogue systems. To support this investigation, we propose the Conversational-Logic-Injection-in-Neural-Network system (CLINN) where explicit knowledge is coded in semi-logical rules. By using CLINN, we evaluated semi-logical rules produced by a team of differently skilled conversational designers. We experimented with the Restaurant topic of the MultiWOZ dataset. Results show that external knowledge is extremely important for reducing the need of annotated examples for conversational systems. In fact, rules from conversational designers used in CLINN significantly outperform a state-of-the-art neural-based dialogue system.
△ Less
Submitted 27 September, 2021;
originally announced September 2021.
-
Physical Watermarking for Replay Attack Detection in Continuous-time Systems
Authors:
Bahram Yaghooti,
Raffaele Romagnoli,
Bruno Sinopoli
Abstract:
Physical watermarking is a well established technique for replay attack detection in cyber-physical systems (CPSs). Most of the watermarking methods proposed in the literature are designed for discrete-time systems. In general, real physical systems evolve in continuous time. In this paper, we analyze the effect of watermarking on sampled-data continuous-time systems controlled via a Zero-Order Ho…
▽ More
Physical watermarking is a well established technique for replay attack detection in cyber-physical systems (CPSs). Most of the watermarking methods proposed in the literature are designed for discrete-time systems. In general, real physical systems evolve in continuous time. In this paper, we analyze the effect of watermarking on sampled-data continuous-time systems controlled via a Zero-Order Hold. We investigate the effect of sampling on detection performance and we provide a procedure to find a suitable sampling period that ensures detectability and acceptable control performance. Simulations on a quadrotor system are used to illustrate the effectiveness of the theoretical results.
△ Less
Submitted 1 March, 2021;
originally announced March 2021.
-
Almawave-SLU: A new dataset for SLU in Italian
Authors:
Valentina Bellomaria,
Giuseppe Castellucci,
Andrea Favalli,
Raniero Romagnoli
Abstract:
The widespread use of conversational and question answering systems made it necessary to improve the performances of speaker intent detection and understanding of related semantic slots, i.e., Spoken Language Understanding (SLU). Often, these tasks are approached with supervised learning methods, which needs considerable labeled datasets. This paper presents the first Italian dataset for SLU. It i…
▽ More
The widespread use of conversational and question answering systems made it necessary to improve the performances of speaker intent detection and understanding of related semantic slots, i.e., Spoken Language Understanding (SLU). Often, these tasks are approached with supervised learning methods, which needs considerable labeled datasets. This paper presents the first Italian dataset for SLU. It is derived through a semi-automatic procedure and is used as a benchmark of various open source and commercial systems.
△ Less
Submitted 17 July, 2019;
originally announced July 2019.
-
Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model
Authors:
Giuseppe Castellucci,
Valentina Bellomaria,
Andrea Favalli,
Raniero Romagnoli
Abstract:
Intent Detection and Slot Filling are two pillar tasks in Spoken Natural Language Understanding. Common approaches adopt joint Deep Learning architectures in attention-based recurrent frameworks. In this work, we aim at exploiting the success of "recurrence-less" models for these tasks. We introduce Bert-Joint, i.e., a multi-lingual joint text classification and sequence labeling framework. The ex…
▽ More
Intent Detection and Slot Filling are two pillar tasks in Spoken Natural Language Understanding. Common approaches adopt joint Deep Learning architectures in attention-based recurrent frameworks. In this work, we aim at exploiting the success of "recurrence-less" models for these tasks. We introduce Bert-Joint, i.e., a multi-lingual joint text classification and sequence labeling framework. The experimental evaluation over two well-known English benchmarks demonstrates the strong performances that can be obtained with this model, even when few annotated data is available. Moreover, we annotated a new dataset for the Italian language, and we observed similar performances without the need for changing the model.
△ Less
Submitted 5 July, 2019;
originally announced July 2019.
-
Design of Software Rejuvenation for CPS Security Using Invariant Sets
Authors:
Raffaele Romagnoli,
Bruce H. Krogh,
Bruno Sinopoli
Abstract:
Software rejuvenation has been proposed as a strategy to protect cyber-physical systems (CSPs) against unanticipated and undetectable cyber attacks. The basic idea is to refresh the system periodically with a secure and trusted copy of the online software so as to eliminate all effects of malicious modifications to the run-time code and data. Following each software refresh a safety controller ass…
▽ More
Software rejuvenation has been proposed as a strategy to protect cyber-physical systems (CSPs) against unanticipated and undetectable cyber attacks. The basic idea is to refresh the system periodically with a secure and trusted copy of the online software so as to eliminate all effects of malicious modifications to the run-time code and data. Following each software refresh a safety controller assures the CPS is driven to a safe state before returning to the mission control mode when the CPS is again vulnerable attacks. This paper considers software rejuvenation design from a control-theoretic perspective. Invariant sets for the Lyapunov function for the safety controller are used to derive bounds on the time that the CPS can operate in mission control mode before the software must be refreshed and the maximum time the safety controller will require to bring the CPS to a safe operating state. With these results it can be guaranteed that the CPS will remain safe under cyber attacks against the run-time system and will be able to execute missions successfully if the attacks are not persistent. The general approach is illustrated using simulation of the nonlinear dynamics of a quadrotor system. The concluding section discusses directions for further research.
△ Less
Submitted 24 October, 2018;
originally announced October 2018.
-
Software Rejuvenation for Secure Tracking Control
Authors:
Raffaele Romagnoli,
Bruce H. Krogh,
Dionisio de Niz,
Bruno Sinopoli
Abstract:
Software rejuvenation protects cyber-physical systems (CSPs) against cyber attacks on the run-time code by periodically refreshing the system with an uncorrupted software image. The system is vulnerable to attacks when it is communicating with other agents. Security is guaranteed during the software refresh and re-initialization by turning off all communication. Although the effectiveness of softw…
▽ More
Software rejuvenation protects cyber-physical systems (CSPs) against cyber attacks on the run-time code by periodically refreshing the system with an uncorrupted software image. The system is vulnerable to attacks when it is communicating with other agents. Security is guaranteed during the software refresh and re-initialization by turning off all communication. Although the effectiveness of software rejuvenation has been demonstrated for some simple systems, many problems need to be addressed to make it viable for real applications. This paper expands the scope of CPS applications for which software rejuvenation can be implemented by introducing architectural and algorithmic features to support trajectory tracking. Following each software refresh, while communication is still off, a safety controller is executed to assure the system state is within a sufficiently small neighborhood of the current point on the reference trajectory. Communication is then re-established and the reference trajectory tracking control is resumed. A protected, verified hypervisor manages the software rejuvenation sequence and delivers trusted reference trajectory points, which may be received from untrusted communication, but are verified using an authentication process. We present the approach to designing the tracking and safety controllers and timing parameters and demonstrate the secure tracking control for a 6 DOF quadrotor using the PX4 jMAVSim quadrotor simulator. The concluding section discusses directions for further research.
△ Less
Submitted 24 October, 2018;
originally announced October 2018.
-
Steady-state input calculation for achieving a desired steady-state output of a linear systems
Authors:
Raffaele Romagnoli,
Emanuele Garone
Abstract:
In this note we provide an algorithm for the computation of the steady-state input able to achieve the steady-state output tracking of any desired output signal representable as a rational transfer function.
In this note we provide an algorithm for the computation of the steady-state input able to achieve the steady-state output tracking of any desired output signal representable as a rational transfer function.
△ Less
Submitted 7 March, 2018;
originally announced April 2018.
