-
Relational Q-Functionals: Multi-Agent Learning to Recover from Unforeseen Robot Malfunctions in Continuous Action Domains
Authors:
Yasin Findik,
Paul Robinette,
Kshitij Jerath,
Reza Azadeh
Abstract:
Cooperative multi-agent learning methods are essential in developing effective cooperation strategies in multi-agent domains. In robotics, these methods extend beyond multi-robot scenarios to single-robot systems, where they enable coordination among different robot modules (e.g., robot legs or joints). However, current methods often struggle to quickly adapt to unforeseen failures, such as a malf…
▽ More
Cooperative multi-agent learning methods are essential in developing effective cooperation strategies in multi-agent domains. In robotics, these methods extend beyond multi-robot scenarios to single-robot systems, where they enable coordination among different robot modules (e.g., robot legs or joints). However, current methods often struggle to quickly adapt to unforeseen failures, such as a malfunctioning robot leg, especially after the algorithm has converged to a strategy. To overcome this, we introduce the Relational Q-Functionals (RQF) framework. RQF leverages a relational network, representing agents' relationships, to enhance adaptability, providing resilience against malfunction(s). Our algorithm also efficiently handles continuous state-action domains, making it adept for robotic learning tasks. Our empirical results show that RQF enables agents to use these relationships effectively to facilitate cooperation and recover from an unexpected malfunction in single-robot systems with multiple interacting modules. Thus, our approach offers promising applications in multi-agent systems, particularly in scenarios with unforeseen malfunctions.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
-
Maximizing Information Gain in Privacy-Aware Active Learning of Email Anomalies
Authors:
Mu-Huan Miles Chung,
Sharon Li,
Jaturong Kongmanee,
Lu Wang,
Yuhong Yang,
Calvin Giang,
Khilan Jerath,
Abhay Raman,
David Lie,
Mark Chignell
Abstract:
Redacted emails satisfy most privacy requirements but they make it more difficult to detect anomalous emails that may be indicative of data exfiltration. In this paper we develop an enhanced method of Active Learning using an information gain maximizing heuristic, and we evaluate its effectiveness in a real world setting where only redacted versions of email could be labeled by human analysts due…
▽ More
Redacted emails satisfy most privacy requirements but they make it more difficult to detect anomalous emails that may be indicative of data exfiltration. In this paper we develop an enhanced method of Active Learning using an information gain maximizing heuristic, and we evaluate its effectiveness in a real world setting where only redacted versions of email could be labeled by human analysts due to privacy concerns. In the first case study we examined how Active Learning should be carried out. We found that model performance was best when a single highly skilled (in terms of the labelling task) analyst provided the labels. In the second case study we used confidence ratings to estimate the labeling uncertainty of analysts and then prioritized instances for labeling based on the expected information gain (the difference between model uncertainty and analyst uncertainty) that would be provided by labelling each instance. We found that the information maximization gain heuristic improved model performance over existing sampling methods for Active Learning. Based on the results obtained, we recommend that analysts should be screened, and possibly trained, prior to implementation of Active Learning in cybersecurity applications. We also recommend that the information gain maximizing sample method (based on expert confidence) should be used in early stages of Active Learning, providing that well-calibrated confidence can be obtained. We also note that the expertise of analysts should be assessed prior to Active Learning, as we found that analysts with lower labelling skill had poorly calibrated (over-) confidence in their labels.
△ Less
Submitted 12 May, 2024;
originally announced May 2024.
-
Energy-Guided Data Sampling for Traffic Prediction with Mini Training Datasets
Authors:
Zhaohui Yang,
Kshitij Jerath
Abstract:
Recent endeavors aimed at forecasting future traffic flow states through deep learning encounter various challenges and yield diverse outcomes. A notable obstacle arises from the substantial data requirements of deep learning models, a resource often scarce in traffic flow systems. Despite the abundance of domain knowledge concerning traffic flow dynamics, prevailing deep learning methodologies fr…
▽ More
Recent endeavors aimed at forecasting future traffic flow states through deep learning encounter various challenges and yield diverse outcomes. A notable obstacle arises from the substantial data requirements of deep learning models, a resource often scarce in traffic flow systems. Despite the abundance of domain knowledge concerning traffic flow dynamics, prevailing deep learning methodologies frequently fail to fully exploit it. To address these issues, we propose an innovative solution that merges Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) architecture to enhance the prediction of traffic flow dynamics. A key revelation of our research is the feasibility of sampling training data for large traffic systems from simulations conducted on smaller traffic systems. This insight suggests the potential for referencing a macroscopic-level distribution to inform the sampling of microscopic data. Such sampling is facilitated by the observed scale invariance in the normalized energy distribution of the statistical mechanics model, thereby streamlining the data generation process for large-scale traffic systems. Our simulations demonstrate promising agreement between predicted and actual traffic flow dynamics, underscoring the efficacy of our proposed approach.
△ Less
Submitted 1 April, 2024; v1 submitted 27 March, 2024;
originally announced March 2024.
-
Multi-scale Traffic Flow Modeling: A Renormalization Group Approach
Authors:
Zhaohui Yang,
Kshitij Jerath
Abstract:
Traffic flow modeling is typically performed at one of three different scales (microscopic, mesoscopic, or macroscopic), each with distinct modeling approaches. Recent works that attempt to merge models at different scales have yielded some success, but there still exists a need for a single modeling framework that can seamlessly model traffic flow across several spatiotemporal scales. The present…
▽ More
Traffic flow modeling is typically performed at one of three different scales (microscopic, mesoscopic, or macroscopic), each with distinct modeling approaches. Recent works that attempt to merge models at different scales have yielded some success, but there still exists a need for a single modeling framework that can seamlessly model traffic flow across several spatiotemporal scales. The presented work utilizes a renormalization group (RG) theoretic approach, building upon our prior research on statistical mechanics-inspired traffic flow modeling. Specifically, we use an Ising model-inspired cellular automata model to represent traffic flow dynamics. RG transformations are applied to this model to obtain coarse-grained parameters (interaction and field coefficients) to simulate traffic at coarser spatiotemporal scales and different vehicular densities. We measure the accuracy of the coarse-grained traffic flow simulation using a pixel-based image correlation metric and find good correlation between the dynamics at different scales. Importantly, emergent traffic dynamics such as backward moving congestion waves are retained at coarser scales with this approach. The presented work has the potential to spur the development of a unified traffic flow modeling framework for transportation analysis across varied spatiotemporal scales, while retaining an analytical relationship between the model parameters at these scales.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
Iterative Forgetting: Online Data Stream Regression Using Database-Inspired Adaptive Granulation
Authors:
Niket Kathiriya,
Hossein Haeri,
Cindy Chen,
Kshitij Jerath
Abstract:
Many modern systems, such as financial, transportation, and telecommunications systems, are time-sensitive in the sense that they demand low-latency predictions for real-time decision-making. Such systems often have to contend with continuous unbounded data streams as well as concept drift, which are challenging requirements that traditional regression techniques are unable to cater to. There exis…
▽ More
Many modern systems, such as financial, transportation, and telecommunications systems, are time-sensitive in the sense that they demand low-latency predictions for real-time decision-making. Such systems often have to contend with continuous unbounded data streams as well as concept drift, which are challenging requirements that traditional regression techniques are unable to cater to. There exists a need to create novel data stream regression methods that can handle these scenarios. We present a database-inspired datastream regression model that (a) uses inspiration from R*-trees to create granules from incoming datastreams such that relevant information is retained, (b) iteratively forgets granules whose information is deemed to be outdated, thus maintaining a list of only recent, relevant granules, and (c) uses the recent data and granules to provide low-latency predictions. The R*-tree-inspired approach also makes the algorithm amenable to integration with database systems. Our experiments demonstrate that the ability of this method to discard data produces a significant order-of-magnitude improvement in latency and training time when evaluated against the most accurate state-of-the-art algorithms, while the R*-tree-inspired granulation technique provides competitively accurate predictions
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Consumers' Perceived Privacy Violations in Online Advertising
Authors:
Kinshuk Jerath,
Klaus M. Miller
Abstract:
In response to privacy concerns about collecting and using personal data, the online advertising industry has been developing privacy-enhancing technologies (PETs), e.g., under Google's Privacy Sandbox initiative. In this research, we use the dual-privacy framework, which postulates that consumers have intrinsic and instrumental preferences for privacy, to understand consumers' perceived privacy v…
▽ More
In response to privacy concerns about collecting and using personal data, the online advertising industry has been developing privacy-enhancing technologies (PETs), e.g., under Google's Privacy Sandbox initiative. In this research, we use the dual-privacy framework, which postulates that consumers have intrinsic and instrumental preferences for privacy, to understand consumers' perceived privacy violations (PPVs) for current and proposed online advertising practices. The key idea is that different practices differ in whether individual data leaves the consumer's machine or not and in how they track and target consumers; these affect, respectively, the intrinsic and instrumental components of privacy preferences differently, leading to different PPVs for different practices. We conducted online studies focused on consumers in the United States to elicit PPVs for various advertising practices. Our findings confirm the intuition that tracking and targeting consumers under the industry status quo of behavioral targeting leads to high PPV. New technologies or proposals that ensure that data are kept on the consumer's machine lower PPV relative to behavioral targeting but, importantly, this decrease is small. Furthermore, group-level targeting does not differ significantly from individual-level targeting in reducing PPV. Under contextual targeting, where there is no tracking, PPV is significantly reduced. Interestingly, with respect to PPV, consumers are indifferent between seeing untargeted ads and no ads when they are not being tracked. We find that consumer perceptions of privacy violations under different tracking and targeting practices may differ from what technical definitions suggest. Therefore, rather than relying solely on technical perspectives, a consumer-centric approach to privacy is needed, based on, for instance, the dual-privacy framework.
△ Less
Submitted 28 May, 2024; v1 submitted 6 March, 2024;
originally announced March 2024.
-
Human-guided Swarms: Impedance Control-inspired Influence in Virtual Reality Environments
Authors:
Spencer Barclay,
Kshitij Jerath
Abstract:
Prior works in human-swarm interaction (HSI) have sought to guide swarm behavior towards established objectives, but may be unable to handle specific scenarios that require finer human supervision, variable autonomy, or application to large-scale swarms. In this paper, we present an approach that enables human supervisors to tune the level of swarm control, and guide a large swarm using an assisti…
▽ More
Prior works in human-swarm interaction (HSI) have sought to guide swarm behavior towards established objectives, but may be unable to handle specific scenarios that require finer human supervision, variable autonomy, or application to large-scale swarms. In this paper, we present an approach that enables human supervisors to tune the level of swarm control, and guide a large swarm using an assistive control mechanism that does not significantly restrict emergent swarm behaviors. We develop this approach in a virtual reality (VR) environment, using the HTC Vive and Unreal Engine 4 with AirSim plugin. The novel combination of an impedance control-inspired influence mechanism and a VR test bed enables and facilitates the rapid design and test iterations to examine trade-offs between swarming behavior and macroscopic-scale human influence, while circumventing flight duration limitations associated with battery-powered small unmanned aerial system (sUAS) systems. The impedance control-inspired mechanism was tested by a human supervisor to guide a virtual swarm consisting of 16 sUAS agents. Each test involved moving the swarm's center of mass through narrow canyons, which were not feasible for a swarm to traverse autonomously. Results demonstrate that integration of the influence mechanism enabled the successful manipulation of the macro-scale behavior of the swarm towards task completion, while maintaining the innate swarming behavior.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
Impact of Relational Networks in Multi-Agent Learning: A Value-Based Factorization View
Authors:
Yasin Findik,
Paul Robinette,
Kshitij Jerath,
S. Reza Ahmadzadeh
Abstract:
Effective coordination and cooperation among agents are crucial for accomplishing individual or shared objectives in multi-agent systems. In many real-world multi-agent systems, agents possess varying abilities and constraints, making it necessary to prioritize agents based on their specific properties to ensure successful coordination and cooperation within the team. However, most existing cooper…
▽ More
Effective coordination and cooperation among agents are crucial for accomplishing individual or shared objectives in multi-agent systems. In many real-world multi-agent systems, agents possess varying abilities and constraints, making it necessary to prioritize agents based on their specific properties to ensure successful coordination and cooperation within the team. However, most existing cooperative multi-agent algorithms do not take into account these individual differences, and lack an effective mechanism to guide coordination strategies. We propose a novel multi-agent learning approach that incorporates relationship awareness into value-based factorization methods. Given a relational network, our approach utilizes inter-agents relationships to discover new team behaviors by prioritizing certain agents over other, accounting for differences between them in cooperative tasks. We evaluated the effectiveness of our proposed approach by conducting fifteen experiments in two different environments. The results demonstrate that our proposed algorithm can influence and shape team behavior, guide cooperation strategies, and expedite agent learning. Therefore, our approach shows promise for use in multi-agent systems, especially when agents have diverse properties.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
Influence of Team Interactions on Multi-Robot Cooperation: A Relational Network Perspective
Authors:
Yasin Findik,
Hamid Osooli,
Paul Robinette,
Kshitij Jerath,
S. Reza Ahmadzadeh
Abstract:
Relational networks within a team play a critical role in the performance of many real-world multi-robot systems. To successfully accomplish tasks that require cooperation and coordination, different agents (e.g., robots) necessitate different priorities based on their positioning within the team. Yet, many of the existing multi-robot cooperation algorithms regard agents as interchangeable and lac…
▽ More
Relational networks within a team play a critical role in the performance of many real-world multi-robot systems. To successfully accomplish tasks that require cooperation and coordination, different agents (e.g., robots) necessitate different priorities based on their positioning within the team. Yet, many of the existing multi-robot cooperation algorithms regard agents as interchangeable and lack a mechanism to guide the type of cooperation strategy the agents should exhibit. To account for the team structure in cooperative tasks, we propose a novel algorithm that uses a relational network comprising inter-agent relationships to prioritize certain agents over others. Through appropriate design of the team's relational network, we can guide the cooperation strategy, resulting in the emergence of new behaviors that accomplish the specified task. We conducted six experiments in a multi-robot setting with a cooperative task. Our results demonstrate that the proposed method can effectively influence the type of solution that the algorithm converges to by specifying the relationships between the agents, making it a promising approach for tasks that require cooperation among agents with a specified team structure.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
Collaborative Adaptation: Learning to Recover from Unforeseen Malfunctions in Multi-Robot Teams
Authors:
Yasin Findik,
Paul Robinette,
Kshitij Jerath,
S. Reza Ahmadzadeh
Abstract:
Cooperative multi-agent reinforcement learning (MARL) approaches tackle the challenge of finding effective multi-agent cooperation strategies for accomplishing individual or shared objectives in multi-agent teams. In real-world scenarios, however, agents may encounter unforeseen failures due to constraints like battery depletion or mechanical issues. Existing state-of-the-art methods in MARL often…
▽ More
Cooperative multi-agent reinforcement learning (MARL) approaches tackle the challenge of finding effective multi-agent cooperation strategies for accomplishing individual or shared objectives in multi-agent teams. In real-world scenarios, however, agents may encounter unforeseen failures due to constraints like battery depletion or mechanical issues. Existing state-of-the-art methods in MARL often recover slowly -- if at all -- from such malfunctions once agents have already converged on a cooperation strategy. To address this gap, we present the Collaborative Adaptation (CA) framework. CA introduces a mechanism that guides collaboration and accelerates adaptation from unforeseen failures by leveraging inter-agent relationships. Our findings demonstrate that CA enables agents to act on the knowledge of inter-agent relations, recovering from unforeseen agent failures and selecting appropriate cooperative strategies.
△ Less
Submitted 19 October, 2023;
originally announced October 2023.
-
A Multi-Robot Task Assignment Framework for Search and Rescue with Heterogeneous Teams
Authors:
Hamid Osooli,
Paul Robinette,
Kshitij Jerath,
S. Reza Ahmadzadeh
Abstract:
In post-disaster scenarios, efficient search and rescue operations involve collaborative efforts between robots and humans. Existing planning approaches focus on specific aspects but overlook crucial elements like information gathering, task assignment, and planning. Furthermore, previous methods considering robot capabilities and victim requirements suffer from time complexity due to repetitive p…
▽ More
In post-disaster scenarios, efficient search and rescue operations involve collaborative efforts between robots and humans. Existing planning approaches focus on specific aspects but overlook crucial elements like information gathering, task assignment, and planning. Furthermore, previous methods considering robot capabilities and victim requirements suffer from time complexity due to repetitive planning steps. To overcome these challenges, we introduce a comprehensive framework__the Multi-Stage Multi-Robot Task Assignment. This framework integrates scouting, task assignment, and path-planning stages, optimizing task allocation based on robot capabilities, victim requirements, and past robot performance. Our iterative approach ensures objective fulfillment within problem constraints. Evaluation across four maps, comparing with a state-of-the-art baseline, demonstrates our algorithm's superiority with a remarkable 97 percent performance increase. Our code is open-sourced to enable result replication.
△ Less
Submitted 21 September, 2023;
originally announced September 2023.
-
Thermodynamics-inspired Macroscopic States of Bounded Swarms
Authors:
Hossein Haeri,
Kshitij Jerath,
Jacob Leachman
Abstract:
The collective behavior of swarms is extremely difficult to estimate or predict, even when the local agent rules are known and simple. The presented work seeks to leverage the similarities between fluids and swarm systems to generate a thermodynamics-inspired characterization of the collective behavior of robotic swarms. While prior works have borrowed tools from fluid dynamics to design swarming…
▽ More
The collective behavior of swarms is extremely difficult to estimate or predict, even when the local agent rules are known and simple. The presented work seeks to leverage the similarities between fluids and swarm systems to generate a thermodynamics-inspired characterization of the collective behavior of robotic swarms. While prior works have borrowed tools from fluid dynamics to design swarming behaviors, they have usually avoided the task of generating a fluids-inspired macroscopic state (or macrostate) description of the swarm. This work will bridge the gap by seeking to answer the following question: is it possible to generate a small set of thermodynamics-inspired macroscopic properties that may later be used to quantify all possible collective behaviors of swarm systems? In this paper, we present three macroscopic properties analogous to pressure, temperature, and density of a gas, to describe the behavior of a swarm that is governed by only attractive and repulsive agent interactions. These properties are made to satisfy an equation similar to the ideal gas law, and also generalized to satisfy the virial equation of state for real gases. Finally, we investigate how swarm specifications such as density and average agent velocity affect the system macrostate.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
Unsupervised Learning of Distributional Properties can Supplement Human Labeling and Increase Active Learning Efficiency in Anomaly Detection
Authors:
Jaturong Kongmanee,
Mark Chignell,
Khilan Jerath,
Abhay Raman
Abstract:
Exfiltration of data via email is a serious cybersecurity threat for many organizations. Detecting data exfiltration (anomaly) patterns typically requires labeling, most often done by a human annotator, to reduce the high number of false alarms. Active Learning (AL) is a promising approach for labeling data efficiently, but it needs to choose an efficient order in which cases are to be labeled, an…
▽ More
Exfiltration of data via email is a serious cybersecurity threat for many organizations. Detecting data exfiltration (anomaly) patterns typically requires labeling, most often done by a human annotator, to reduce the high number of false alarms. Active Learning (AL) is a promising approach for labeling data efficiently, but it needs to choose an efficient order in which cases are to be labeled, and there are uncertainties as to what scoring procedure should be used to prioritize cases for labeling, especially when detecting rare cases of interest is crucial. We propose an adaptive AL sampling strategy that leverages the underlying prior data distribution, as well as model uncertainty, to produce batches of cases to be labeled that contain instances of rare anomalies. We show that (1) the classifier benefits from a batch of representative and informative instances of both normal and anomalous examples, (2) unsupervised anomaly detection plays a useful role in building the classifier in the early stages of training when relatively little labeling has been done thus far. Our approach to AL for anomaly detection outperformed existing AL approaches on three highly unbalanced UCI benchmarks and on one real-world redacted email data set.
△ Less
Submitted 13 July, 2023;
originally announced July 2023.
-
Implementing Active Learning in Cybersecurity: Detecting Anomalies in Redacted Emails
Authors:
Mu-Huan Chung,
Lu Wang,
Sharon Li,
Yuhong Yang,
Calvin Giang,
Khilan Jerath,
Abhay Raman,
David Lie,
Mark Chignell
Abstract:
Research on email anomaly detection has typically relied on specially prepared datasets that may not adequately reflect the type of data that occurs in industry settings. In our research, at a major financial services company, privacy concerns prevented inspection of the bodies of emails and attachment details (although subject headings and attachment filenames were available). This made labeling…
▽ More
Research on email anomaly detection has typically relied on specially prepared datasets that may not adequately reflect the type of data that occurs in industry settings. In our research, at a major financial services company, privacy concerns prevented inspection of the bodies of emails and attachment details (although subject headings and attachment filenames were available). This made labeling possible anomalies in the resulting redacted emails more difficult. Another source of difficulty is the high volume of emails combined with the scarcity of resources making machine learning (ML) a necessity, but also creating a need for more efficient human training of ML models. Active learning (AL) has been proposed as a way to make human training of ML models more efficient. However, the implementation of Active Learning methods is a human-centered AI challenge due to potential human analyst uncertainty, and the labeling task can be further complicated in domains such as the cybersecurity domain (or healthcare, aviation, etc.) where mistakes in labeling can have highly adverse consequences. In this paper we present research results concerning the application of Active Learning to anomaly detection in redacted emails, comparing the utility of different methods for implementing active learning in this context. We evaluate different AL strategies and their impact on resulting model performance. We also examine how ratings of confidence that experts have in their labels can inform AL. The results obtained are discussed in terms of their implications for AL methodology and for the role of experts in model-assisted email anomaly screening.
△ Less
Submitted 2 March, 2023; v1 submitted 1 March, 2023;
originally announced March 2023.
-
DECISIVE Benchmarking Data Report: sUAS Performance Results from Phase I
Authors:
Adam Norton,
Reza Ahmadzadeh,
Kshitij Jerath,
Paul Robinette,
Jay Weitzen,
Thanuka Wickramarathne,
Holly Yanco,
Minseop Choi,
Ryan Donald,
Brendan Donoghue,
Christian Dumas,
Peter Gavriel,
Alden Giedraitis,
Brendan Hertel,
Jack Houle,
Nathan Letteri,
Edwin Meriaux,
Zahra Rezaei Khavas,
Rakshith Singh,
Gregg Willcox,
Naye Yoni
Abstract:
This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) perfo…
▽ More
This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. Using those 20 test methods, over 230 tests were conducted across 8 sUAS platforms: Cleo Robotics Dronut X1P (P = prototype), FLIR Black Hornet PRS, Flyability Elios 2 GOV, Lumenier Nighthawk V3, Parrot ANAFI USA GOV, Skydio X2D, Teal Golden Eagle, and Vantage Robotics Vesper. Best in class criteria is specified for each applicable test method and the sUAS that match this criteria are named for each test method, including a high-level executive summary of their performance.
△ Less
Submitted 20 January, 2023; v1 submitted 18 January, 2023;
originally announced January 2023.
-
DECISIVE Test Methods Handbook: Test Methods for Evaluating sUAS in Subterranean and Constrained Indoor Environments, Version 1.1
Authors:
Adam Norton,
Reza Ahmadzadeh,
Kshitij Jerath,
Paul Robinette,
Jay Weitzen,
Thanuka Wickramarathne,
Holly Yanco,
Minseop Choi,
Ryan Donald,
Brendan Donoghue,
Christian Dumas,
Peter Gavriel,
Alden Giedraitis,
Brendan Hertel,
Jack Houle,
Nathan Letteri,
Edwin Meriaux,
Zahra Rezaei Khavas,
Rakshith Singh,
Gregg Willcox,
Naye Yoni
Abstract:
This handbook outlines all test methods developed under the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstac…
▽ More
This handbook outlines all test methods developed under the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. For sUAS deployment in subterranean and constrained indoor environments, this puts forth two assumptions about applicable sUAS to be evaluated using these test methods: (1) able to operate without access to GPS signal, and (2) width from prop top to prop tip does not exceed 91 cm (36 in) wide (i.e., can physically fit through a typical doorway, although successful navigation through is not guaranteed). All test methods are specified using a common format: Purpose, Summary of Test Method, Apparatus and Artifacts, Equipment, Metrics, Procedure, and Example Data. All test methods are designed to be run in real-world environments (e.g., MOUT sites) or using fabricated apparatuses (e.g., test bays built from wood, or contained inside of one or more shipping containers).
△ Less
Submitted 20 January, 2023; v1 submitted 1 November, 2022;
originally announced November 2022.
-
Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior
Authors:
Hossein Haeri,
Reza Ahmadzadeh,
Kshitij Jerath
Abstract:
In this work, we integrate `social' interactions into the MARL setup through a user-defined relational network and examine the effects of agent-agent relations on the rise of emergent behaviors. Leveraging insights from sociology and neuroscience, our proposed framework models agent relationships using the notion of Reward-Sharing Relational Networks (RSRN), where network edge weights act as a mea…
▽ More
In this work, we integrate `social' interactions into the MARL setup through a user-defined relational network and examine the effects of agent-agent relations on the rise of emergent behaviors. Leveraging insights from sociology and neuroscience, our proposed framework models agent relationships using the notion of Reward-Sharing Relational Networks (RSRN), where network edge weights act as a measure of how much one agent is invested in the success of (or `cares about') another. We construct relational rewards as a function of the RSRN interaction weights to collectively train the multi-agent system via a multi-agent reinforcement learning algorithm. The performance of the system is tested for a 3-agent scenario with different relational network structures (e.g., self-interested, communitarian, and authoritarian networks). Our results indicate that reward-sharing relational networks can significantly influence learned behaviors. We posit that RSRN can act as a framework where different relational networks produce distinct emergent behaviors, often analogous to the intuited sociological understanding of such networks.
△ Less
Submitted 13 July, 2022; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Inefficiencies in Digital Advertising Markets
Authors:
Brett R Gordon,
Kinshuk Jerath,
Zsolt Katona,
Sridhar Narayanan,
Jiwoong Shin,
Kenneth C Wilbur
Abstract:
Digital advertising markets are growing and attracting increased scrutiny. This paper explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking and ad fraud. These topics are not unique to digital advertising, but each manifests in new ways in markets for digital ads. We identify relevant findings…
▽ More
Digital advertising markets are growing and attracting increased scrutiny. This paper explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking and ad fraud. These topics are not unique to digital advertising, but each manifests in new ways in markets for digital ads. We identify relevant findings in the academic literature, recent developments in practice, and promising topics for future research.
△ Less
Submitted 22 February, 2020; v1 submitted 18 December, 2019;
originally announced December 2019.
