The ability to create effective multi-agent organizations is key to the development of larger, mo... more The ability to create effective multi-agent organizations is key to the development of larger, more diverse multi-agent systems. Organizational control provides longterm organizational goals, roles, and responsibilities as guidelines for each agent. Organizational design and instantiation is the process that accepts a set of organizational goals, performance requirements, agents, and resources and assigns responsibilities and roles to each agent. We present a prescriptive organizational design and instantiation process for ...
In our work on real-time problem solving we have found that the interface between the decision-ma... more In our work on real-time problem solving we have found that the interface between the decision-maker and the real-time scheduler needs to be complex and bidirectional. We argue that this interface can usefully be modeled as a negotiation process. In this paper we present the details of the interface, as well as our scheduler that is capable of scheduling real-time tasks and providing the information required by the interface.
In a multi-agent system, an environment imposes tasks on a group of agents. Upon successful compl... more In a multi-agent system, an environment imposes tasks on a group of agents. Upon successful completion, the agents get rewarded. Any interesting environment requires the agents to decide, which tasks they want to work on. By modeling a multi-agent system statistically and exploring the model, we can gain deep insights into the way such decisions should be made.
When collaborative agents cannot observe the global state of an un- certain environment, they mus... more When collaborative agents cannot observe the global state of an un- certain environment, they must communicate in order to achieve their common goals, which is evaluated through a global utility function . We develop a multi- agent extension to Markov decision processes in which communication is an ex- plicit action that incurs a cost. Thus, agents have to decide not
In this paper, we formulate agent's decision process under the framework of Markov de- cision... more In this paper, we formulate agent's decision process under the framework of Markov de- cision processes, and in particular, the multi-agent extension to Markov decision process that includes agent communication decisions. We model communication as the way for each agent to obtain local state information in other agents, by paying a certain communication cost. Thus, agents have to decide not
Autonomous Agents & Multiagent Systems/Agent Theories, Architectures, and Languages, 1999
Abstract. Commitments,play a central role in multi-agent coordination. How- ever, they are inhere... more Abstract. Commitments,play a central role in multi-agent coordination. How- ever, they are inherently uncertain and it is important to ta ke these uncertainties into account during planning and scheduling. This paper addresses the problem of handling the uncertainty in commitments. We propose a new model of commit- ment that incorporates the uncertainty, the use of continge ncy analysis to reduce
In this paper, we formulate agent's decision process under the framework of Markov decision proce... more In this paper, we formulate agent's decision process under the framework of Markov decision processes, and in particular, the multi-agent extension to Markov decision process that includes agent communication decisions. We model communication as the way for each agent to obtain local state information in other agents, by paying a certain communication cost. Thus, agents have to decide not only which local action to perform, but also whether it is worthwhile to perform a communication action before deciding the local action. We believe that this would provide a foundation for formal study of coordination activities and may lead to some insights to the design of agent coordination policies, and heuristic approaches in particular. An example problem is studied under this framework and its implications to coordination are discussed.
Abstract An algorithm is presented for fitting an expression composed of continuous and discontin... more Abstract An algorithm is presented for fitting an expression composed of continuous and discontinuous primitive functions to real-valued data points. The data modeling problem comes from the need to infer task structure for making coordination decisions for multi-agent systems. The presence of discontinuous primitive functions requires a novel approach.
We propose an algorithm that given a problem formulated as a Distributed Bayesian Network, finds ... more We propose an algorithm that given a problem formulated as a Distributed Bayesian Network, finds a coordination strategy which minimizes the communication costs while achieving the desired confidence level of the global solution. We developed a system based on this algorithm which models the communication decision process for any given problem structure as a Markov Decision Process and use dynamic programming to produce the optimal communication strategy. To reduce the computational cost of the MDP approach, we further propose an algorithm based on the concept of Mutual Information to approximate the optimal solution. Experimental results for both systems are given to illustrate the effectiveness of the algorithms.
This report outlines the IPUS paradigm, named for Integrated Processing and Understanding of Sign... more This report outlines the IPUS paradigm, named for Integrated Processing and Understanding of Signals, which permits sophisticated interaction between theory-based problem solving in signal processing and heuristic problem-solving in signal interpretation. The need for such a paradigm arises in signal understanding domains that require the processing of complicated interacting signals under variable signal-to-noise ratios. One such application is sound understanding, in the context of which we report on a testbed experiment illustrating the functionality of key IPUS architecture components.
The functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm is one approac... more The functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm is one approach for organizing distributed problem solving among homogeneous, cooperating agents. The idea behind the FA/C model is that agents should produce tentative, partial results based on only local information and then exchange these results, exploiting the constraints that exist among their local subproblems to resolve the uncertainties and global inconsistencies that result from the use of incomplete information. While several FA/C systems have been implemented, there has been little formal analysis of the quality of the solutions that can be produced using the approach or of the conditions that are necessary for the approach to be e ective. This paper reports on work we have done to formally analyze the FA/C model in the context of distributed sensor interpretation (SI). Several results are presented that compare the quality of solutions produced by a distributed FA/C system to those produced by an equivalent centralized system, based on particular agent problem-solving and coordination strategies. We rst establish that while it is possible for an FA/C system to produce the same solution as a centralized system, this requires the use of interpretation and coordination strategies that are impractical for most SI applications. Because of this we then consider the e ect of \approximate" interpretation and coordination strategies, given some assumptions about the characteristics of the domain.
Sensor interpretation involves the determination of high-level explanations of sensor data. The i... more Sensor interpretation involves the determination of high-level explanations of sensor data. The interpretation process is based on the use of abduction. Interpretation systems incrementally construct hypotheses using abductive inferences to identify possible explanations for the data and, conversely, possible support for the hypotheses. We have developed and implemented a new blackboard-based interpretation framework called RESUN. One of the key features of RESUN is that it uses a model of the sources of uncertainty in abductive interpretation inferences to create explicit, symbolic representations (called SOUs) of the reasons why hypotheses are uncertain. The symbolic SOUs make it possible for the system to understand the reasons why its hypotheses are uncertain so that it can dynamically select the most appropriate methods for resolving uncertainty. Our model of uncertainty de nes a set of classes of SOUs that are applicable to interpretation problems which can be posed as abduction problems. Each interpretation application may require slightly different instances of each of the classes of SOUs to best represent uncertainty. We have implemented the RESUN framework using a simulated aircraft monitoring system and have run experiments that demonstrate how the SOUs enable the use of more e ective interpretation strategies. To verify the generality of the approach, we are also using RESUN to implement a sound understanding testbed.
This paper reports on extensions that have been made to the DRESUN testbed for research on distri... more This paper reports on extensions that have been made to the DRESUN testbed for research on distributed situation assessment (DSA). These extensions involve issues that have arisen in modeling the beliefs of other agents when dealing with inter-agent communication of incomplete and con icting evidence, and evidence at multiple levels of abstraction. The extensions support highly directed exchanges of evidence among agents because they better represent the uncertainties that occur when DRESUN agents exchange incomplete and con icting information. This is important in FA/C systems because agents must share results in order to satisfy their local goals as well as the overall system goals. Thus, sharing must be done e ciently for an FA/C approach to be e ective. These issues will arise in any distributed problem solving application involving interacting subproblems, when agents must function without complete and up-to-date information.
In the functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm, agents pro... more In the functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm, agents produce tentative, partial results based on local information only, and then exploit the constraints among these local results to resolve uncertainties and global inconsistencies. However, there has never been any formal analysis of the quality of the solutions that are produced by the approach or of the conditions that are necessary for the approach to be successful. This paper represents a rst step in formally analyzing the quality of solutions that can be produced by FA/C systems, within the context of distributed interpretation. Two theorems that compare the quality of solutions produced by a distributed system to those produced by an equivalent centralized system are presented. The theorems relate solution quality to agent problem-solving and coordination strategies. The analysis is based on an abstract model of the DRESUN system for distributed sensor interpretation. While the paper concentrates on sensor interpretation, we expect to extend the work to apply to FA/C systems in general.
The dominant existing routing strategies employed in peerto-peer(P2P) based information retrieval... more The dominant existing routing strategies employed in peerto-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in that the neighboring agents may not be connected to more relevant agents. In this paper, an online reinforcement-learning based approach is developed to take advantage of the dynamic run-time characteristics of P2P IR systems as represented by information about past search sessions. Specifically, agents maintain estimates on the downstream agents' abilities to provide relevant documents for incoming queries. These estimates are updated gradually by learning from the feedback information returned from previous search sessions. Based on this information, the agents derive corresponding routing policies. Thereafter, these agents route the queries based on the learned policies and update the estimates based on the new routing policies. Experimental results demonstrate that the learning algorithm improves considerably the routing performance on two test collection sets that have been used in a variety of distributed IR studies.
The ability to create effective multi-agent organizations is key to the development of larger, mo... more The ability to create effective multi-agent organizations is key to the development of larger, more diverse multi-agent systems. Organizational control provides longterm organizational goals, roles, and responsibilities as guidelines for each agent. Organizational design and instantiation is the process that accepts a set of organizational goals, performance requirements, agents, and resources and assigns responsibilities and roles to each agent. We present a prescriptive organizational design and instantiation process for ...
In our work on real-time problem solving we have found that the interface between the decision-ma... more In our work on real-time problem solving we have found that the interface between the decision-maker and the real-time scheduler needs to be complex and bidirectional. We argue that this interface can usefully be modeled as a negotiation process. In this paper we present the details of the interface, as well as our scheduler that is capable of scheduling real-time tasks and providing the information required by the interface.
In a multi-agent system, an environment imposes tasks on a group of agents. Upon successful compl... more In a multi-agent system, an environment imposes tasks on a group of agents. Upon successful completion, the agents get rewarded. Any interesting environment requires the agents to decide, which tasks they want to work on. By modeling a multi-agent system statistically and exploring the model, we can gain deep insights into the way such decisions should be made.
When collaborative agents cannot observe the global state of an un- certain environment, they mus... more When collaborative agents cannot observe the global state of an un- certain environment, they must communicate in order to achieve their common goals, which is evaluated through a global utility function . We develop a multi- agent extension to Markov decision processes in which communication is an ex- plicit action that incurs a cost. Thus, agents have to decide not
In this paper, we formulate agent's decision process under the framework of Markov de- cision... more In this paper, we formulate agent's decision process under the framework of Markov de- cision processes, and in particular, the multi-agent extension to Markov decision process that includes agent communication decisions. We model communication as the way for each agent to obtain local state information in other agents, by paying a certain communication cost. Thus, agents have to decide not
Autonomous Agents & Multiagent Systems/Agent Theories, Architectures, and Languages, 1999
Abstract. Commitments,play a central role in multi-agent coordination. How- ever, they are inhere... more Abstract. Commitments,play a central role in multi-agent coordination. How- ever, they are inherently uncertain and it is important to ta ke these uncertainties into account during planning and scheduling. This paper addresses the problem of handling the uncertainty in commitments. We propose a new model of commit- ment that incorporates the uncertainty, the use of continge ncy analysis to reduce
In this paper, we formulate agent's decision process under the framework of Markov decision proce... more In this paper, we formulate agent's decision process under the framework of Markov decision processes, and in particular, the multi-agent extension to Markov decision process that includes agent communication decisions. We model communication as the way for each agent to obtain local state information in other agents, by paying a certain communication cost. Thus, agents have to decide not only which local action to perform, but also whether it is worthwhile to perform a communication action before deciding the local action. We believe that this would provide a foundation for formal study of coordination activities and may lead to some insights to the design of agent coordination policies, and heuristic approaches in particular. An example problem is studied under this framework and its implications to coordination are discussed.
Abstract An algorithm is presented for fitting an expression composed of continuous and discontin... more Abstract An algorithm is presented for fitting an expression composed of continuous and discontinuous primitive functions to real-valued data points. The data modeling problem comes from the need to infer task structure for making coordination decisions for multi-agent systems. The presence of discontinuous primitive functions requires a novel approach.
We propose an algorithm that given a problem formulated as a Distributed Bayesian Network, finds ... more We propose an algorithm that given a problem formulated as a Distributed Bayesian Network, finds a coordination strategy which minimizes the communication costs while achieving the desired confidence level of the global solution. We developed a system based on this algorithm which models the communication decision process for any given problem structure as a Markov Decision Process and use dynamic programming to produce the optimal communication strategy. To reduce the computational cost of the MDP approach, we further propose an algorithm based on the concept of Mutual Information to approximate the optimal solution. Experimental results for both systems are given to illustrate the effectiveness of the algorithms.
This report outlines the IPUS paradigm, named for Integrated Processing and Understanding of Sign... more This report outlines the IPUS paradigm, named for Integrated Processing and Understanding of Signals, which permits sophisticated interaction between theory-based problem solving in signal processing and heuristic problem-solving in signal interpretation. The need for such a paradigm arises in signal understanding domains that require the processing of complicated interacting signals under variable signal-to-noise ratios. One such application is sound understanding, in the context of which we report on a testbed experiment illustrating the functionality of key IPUS architecture components.
The functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm is one approac... more The functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm is one approach for organizing distributed problem solving among homogeneous, cooperating agents. The idea behind the FA/C model is that agents should produce tentative, partial results based on only local information and then exchange these results, exploiting the constraints that exist among their local subproblems to resolve the uncertainties and global inconsistencies that result from the use of incomplete information. While several FA/C systems have been implemented, there has been little formal analysis of the quality of the solutions that can be produced using the approach or of the conditions that are necessary for the approach to be e ective. This paper reports on work we have done to formally analyze the FA/C model in the context of distributed sensor interpretation (SI). Several results are presented that compare the quality of solutions produced by a distributed FA/C system to those produced by an equivalent centralized system, based on particular agent problem-solving and coordination strategies. We rst establish that while it is possible for an FA/C system to produce the same solution as a centralized system, this requires the use of interpretation and coordination strategies that are impractical for most SI applications. Because of this we then consider the e ect of \approximate" interpretation and coordination strategies, given some assumptions about the characteristics of the domain.
Sensor interpretation involves the determination of high-level explanations of sensor data. The i... more Sensor interpretation involves the determination of high-level explanations of sensor data. The interpretation process is based on the use of abduction. Interpretation systems incrementally construct hypotheses using abductive inferences to identify possible explanations for the data and, conversely, possible support for the hypotheses. We have developed and implemented a new blackboard-based interpretation framework called RESUN. One of the key features of RESUN is that it uses a model of the sources of uncertainty in abductive interpretation inferences to create explicit, symbolic representations (called SOUs) of the reasons why hypotheses are uncertain. The symbolic SOUs make it possible for the system to understand the reasons why its hypotheses are uncertain so that it can dynamically select the most appropriate methods for resolving uncertainty. Our model of uncertainty de nes a set of classes of SOUs that are applicable to interpretation problems which can be posed as abduction problems. Each interpretation application may require slightly different instances of each of the classes of SOUs to best represent uncertainty. We have implemented the RESUN framework using a simulated aircraft monitoring system and have run experiments that demonstrate how the SOUs enable the use of more e ective interpretation strategies. To verify the generality of the approach, we are also using RESUN to implement a sound understanding testbed.
This paper reports on extensions that have been made to the DRESUN testbed for research on distri... more This paper reports on extensions that have been made to the DRESUN testbed for research on distributed situation assessment (DSA). These extensions involve issues that have arisen in modeling the beliefs of other agents when dealing with inter-agent communication of incomplete and con icting evidence, and evidence at multiple levels of abstraction. The extensions support highly directed exchanges of evidence among agents because they better represent the uncertainties that occur when DRESUN agents exchange incomplete and con icting information. This is important in FA/C systems because agents must share results in order to satisfy their local goals as well as the overall system goals. Thus, sharing must be done e ciently for an FA/C approach to be e ective. These issues will arise in any distributed problem solving application involving interacting subproblems, when agents must function without complete and up-to-date information.
In the functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm, agents pro... more In the functionally-accurate, cooperative (FA/C) distributed problem-solving paradigm, agents produce tentative, partial results based on local information only, and then exploit the constraints among these local results to resolve uncertainties and global inconsistencies. However, there has never been any formal analysis of the quality of the solutions that are produced by the approach or of the conditions that are necessary for the approach to be successful. This paper represents a rst step in formally analyzing the quality of solutions that can be produced by FA/C systems, within the context of distributed interpretation. Two theorems that compare the quality of solutions produced by a distributed system to those produced by an equivalent centralized system are presented. The theorems relate solution quality to agent problem-solving and coordination strategies. The analysis is based on an abstract model of the DRESUN system for distributed sensor interpretation. While the paper concentrates on sensor interpretation, we expect to extend the work to apply to FA/C systems in general.
The dominant existing routing strategies employed in peerto-peer(P2P) based information retrieval... more The dominant existing routing strategies employed in peerto-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in that the neighboring agents may not be connected to more relevant agents. In this paper, an online reinforcement-learning based approach is developed to take advantage of the dynamic run-time characteristics of P2P IR systems as represented by information about past search sessions. Specifically, agents maintain estimates on the downstream agents' abilities to provide relevant documents for incoming queries. These estimates are updated gradually by learning from the feedback information returned from previous search sessions. Based on this information, the agents derive corresponding routing policies. Thereafter, these agents route the queries based on the learned policies and update the estimates based on the new routing policies. Experimental results demonstrate that the learning algorithm improves considerably the routing performance on two test collection sets that have been used in a variety of distributed IR studies.
Uploads
Papers by Victor Lesser