Article

Automated resource-driven mission phasing techniques for constrained agents

Authors:

Edmund H. DurfeeAuthors Info & Claims

AAMAS '05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems

July 2005

Pages 331 - 338

https://doi.org/10.1145/1082473.1082524

Published: 25 July 2005 Publication History

Abstract

A constrained agent is limited in the actions that it can take at any given time, and a challenging problem is to design policies for such agents to do the best they can despite their limitations. One way of improving agent performance is to break larger tasks into phases, where the constrained agent is better able to handle each phase and can reconfigure its limited capabilities differently for each phase. In this paper, we present algorithms for automating the process of finding and using mission phases for constrained agents. We analyze several variations of this problem that correspond to different classes of important constrained-agent problems, and show through analysis and experiments that our techniques can increase an agent's rewards for varying levels of constraints on the agent and on the phases.

References

[1]

E. Altman. Constrained Markov decision processes with total cost criteria: Lagrange approach and dual LP. Methods and Models in Operations Research, 48:387--417, 1998.

[2]

E. Altman. Constrained Markov Decision Processes. Chapman and HALL/CRC, New York, 1999.

[3]

A. Bockmayr and Y. Dimopolous. Mixed integer programming models for planning problems. In CP'98 Workshop on Constraint Problem Reformulation, 1998.

[4]

W. Cook, W. Cunningham, W. Pulleyblank, and A. Schrijver. Combinatorial Optimization. John Wiley & Sons, New York, 1998.

Digital Library

[5]

D. A. Dolgov and E. H. Durfee. Constructing optimal policies for agents with constrained architectures. Technical Report CSE-TR-476-03, University of Michigan, 2003.

Digital Library

[6]

D. A. Dolgov and E. H. Durfee. Constructing optimal policies for agents with constrained architectures (abstract). In Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, pages 974--975, 2003.

Digital Library

[7]

E. Feinberg. Constrained discounted Markov decision processes and Hamiltonian cycles. Mathematics of Operations Research, 25:130--140, 2000.

Digital Library

[8]

M. Hauskrecht, N. Meuleau, L. P. Kaelbling, T. Dean, and C. Boutilier. Hierarchical solution of Markov decision processes using macro-actions. In Proceedings of Uncertainty in Artificial Intelligence, pages 220--229, 1998.

Digital Library

[9]

J. Hoffmann, J. Porteous, and L. Sebastia. Ordered landmarks in planning. Journal of Artificial Intelligence Research, 22:215--278, 2004.

Digital Library

[10]

L. Kallenberg. Linear programming and finite Markovian control problems. Mathematisch Centrum, Amsterdam, 1983.

[11]

H. Kautz and J. Walser. Integer optimization models of AI planning problems. Knowledge Engineering Review, 15(1):101--117, 2000.

Digital Library

[12]

D. J. Musliner, E. H. Durfee, and K. G. Shin. CIRCA: A cooperative intelligent real time control architecture. IEEE Transactions on Systems, Man, and Cybernetics, 23(6):1561--1574, 1993.

[13]

D. J. Musliner, E. H. Durfee, and K. G. Shin. World modeling for the dynamic construction of real-time control plans. Artificial Intelligence, 74(1):83--127, 1995.

Digital Library

[14]

R. Parr. Flexible decomposition algorithms for weakly coupled Markov decision problems. In Proceedings of Uncertainty in Artificial Intelligence, pages 422--430, 1998.

Digital Library

[15]

J. Porteous, L. Sebastia, and J. Hoffmann. On the extraction, ordering, and usage of landmarks in planning. In Proceedings of the 6th European Conference on Planning (ECP 01), pages 37--48, 2001.

[16]

M. L. Puterman. Markov Decision Processes. John Wiley & Sons, New York, 1994.

Digital Library

[17]

P. van Beek and X. Chen. CPlan: A constraint programming approach to planning. In Proceedings of National Conference on Artificial Intelligence, pages 585--590, 1999.

Digital Library

[18]

T. Vossen, M. Ball, A. Lotem, and D. Nau. On the use of integer programming models in AI planning. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, Stockholm, Sweden, pages 304--309, 1999.

Digital Library

[19]

L. A. Wolsey. Integer Programming. John Wiley & Sons, New York, 1998.

Cited By

Henrichs ELesch VStraesser MKounev SKrupitzer C(2022)A literature review on optimization techniques for adaptation planning in adaptive systemsInformation and Software Technology10.1016/j.infsof.2022.106940149:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.infsof.2022.106940
Wu JDurfee EDurfee EYokoo MHuhns MShehory O(2007)Sequential resource allocation in multiagent systems with uncertaintiesProceedings of the 6th international joint conference on Autonomous agents and multiagent systems10.1145/1329125.1329265(1-8)Online publication date: 14-May-2007
https://dl.acm.org/doi/10.1145/1329125.1329265
Dolgov DDurfee E(2006)Resource allocation among agents with MDP-induced preferencesJournal of Artificial Intelligence Research10.5555/1622572.162258727:1(505-549)Online publication date: 1-Dec-2006
https://dl.acm.org/doi/10.5555/1622572.1622587
Show More Cited By

Index Terms

Automated resource-driven mission phasing techniques for constrained agents
1. Computing methodologies
  1. Artificial intelligence
2. Theory of computation
  1. Design and analysis of algorithms
    1. Algorithm design techniques
      1. Dynamic programming

Recommendations

Resource-driven mission-phasing techniques for constrained agents in stochastic environments

Because an agent's resources dictate what actions it can possibly take, it should plan which resources it holds over time carefully, considering its inherent limitations (such as power or payload restrictions), the competing needs of other agents for ...
Read More
Mission-phasing techniques for constrained agents in stochastic environments
Read More
Sequential resource allocation in multiagent systems with uncertainties
AAMAS '07: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems

Exchanging scarce resources during execution among a group of agents is one way to improve the overall performance in multiagent systems with limited shared resources, but implementing optimal sequential resource allocation is often a nontrivial problem ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AAMAS '05: Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems

July 2005

1407 pages

ISBN:1595930930

DOI:10.1145/1082473

Program Chairs:
Michal Pechoucek
Czech Republic
,
Donald Steiner
USA
,
Simon Thompson
UK

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

AAMAS05

Sponsor:

SIGAI

AAMAS05: AAMAS '05 - Fourth International Joint Conference on Autonomous Agents and Multiagent Systems 2005

July 25 - 29, 2005

The Netherlands

Acceptance Rates

Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
150
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Henrichs ELesch VStraesser MKounev SKrupitzer C(2022)A literature review on optimization techniques for adaptation planning in adaptive systemsInformation and Software Technology10.1016/j.infsof.2022.106940149:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.infsof.2022.106940
Wu JDurfee EDurfee EYokoo MHuhns MShehory O(2007)Sequential resource allocation in multiagent systems with uncertaintiesProceedings of the 6th international joint conference on Autonomous agents and multiagent systems10.1145/1329125.1329265(1-8)Online publication date: 14-May-2007
https://dl.acm.org/doi/10.1145/1329125.1329265
Dolgov DDurfee E(2006)Resource allocation among agents with MDP-induced preferencesJournal of Artificial Intelligence Research10.5555/1622572.162258727:1(505-549)Online publication date: 1-Dec-2006
https://dl.acm.org/doi/10.5555/1622572.1622587
Wu JDurfee ENakashima HWellman MWeiss GStone P(2006)Mathematical programming for deliberation scheduling in time-limited domainsProceedings of the fifth international joint conference on Autonomous agents and multiagent systems10.1145/1160633.1160789(874-881)Online publication date: 8-May-2006
https://dl.acm.org/doi/10.1145/1160633.1160789
Plamondon PChaib-draa BBenaskeur A(2006)An efficient resource allocation approach in real-time stochastic environmentProceedings of the 19th international conference on Advances in Artificial Intelligence: Canadian Society for Computational Studies of Intelligence10.1007/11766247_5(49-60)Online publication date: 7-Jun-2006
https://dl.acm.org/doi/10.1007/11766247_5

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents