research-article

DUCT: An Upper Confidence Bound Approach to Distributed Constraint Optimization Problems

Authors:

Brammert Ottens,

Christos Dimitrakakis,

Boi FaltingsAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 8, Issue 5

Article No.: 69, Pages 1 - 27

https://doi.org/10.1145/3066156

Published: 12 July 2017 Publication History

Abstract

We propose a distributed upper confidence bound approach, DUCT, for solving distributed constraint optimization problems. We compare four variants of this approach with a baseline random sampling algorithm, as well as other complete and incomplete algorithms for DCOPs. Under general assumptions, we theoretically show that the solution found by DUCT after T steps is approximately T⁻¹-close to the optimal. Experimentally, we show that DUCT matches the optimal solution found by the well-known DPOP and O-DPOP algorithms on moderate-size problems, while always requiring less agent communication. For larger problems, where DPOP fails, we show that DUCT produces significantly better solutions than local, incomplete algorithms. Overall, we believe that DUCT is a practical, scalable algorithm for complex DCOPs.

References

[1]

Sayed Ali, Sven Koenig, and Milind Tambe. 2005. Preprocessing techniques for accelerating the DCOP algorithm ADOPT. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’05). 1041--1048.

Digital Library

[2]

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. 2002. Finite time analysis of the multiarmed bandit problem. Machine Learning 47, 2/3 (2002), 235--256.

Digital Library

[3]

Balabhaskar Balasundaram and Sergiy Butenko. 2006. Graph domination, coloring and cliques in telecommunications. In Handbook of Optimization in Telecommunications. 865--890.

[4]

Sébastien Bubeck, Rémi Munos, Gilles Stoltz, and Csaba Szepesvári. 2011. X-armed bandits. Journal of Machine Learning Research 12 (2011), 1655--1695.

Digital Library

[5]

Nicolò Cesa-Bianchi and Gábor Lugosi. 2006. Prediction, Learning and Games. Cambridge University Press, Cambridge, UK.

Digital Library

[6]

Pierre-Arnaud Coquelin and Rémi Munos. 2007. Bandit algorithms for tree search. In Proceedings of the 23rd Conference Conference on Uncertainty in Artificial Intelligence, Ron Par and Linda van der Gaag (Eds.). Vancouver, Canada.

Digital Library

[7]

Rina Dechter. 2006. Tractable Structures for Constraint Satisfaction Problems. Elsevier Science, New York, NY, Chapter 7, 209--244.

[8]

Rina Dechter and Robert Mateescu. 2004. Mixtures of deterministic-probabilistic networks and their AND/OR search space. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI'04). 120--129.

Digital Library

[9]

Stephen Fitzpatrick and Lambert Meertens. 2003. Distributed coordination through anarchic optimization. In Distributed Sensor Networks. Springer, 257--295.

[10]

Sylvain Gelly and David Silver. 2005. Achieving master level play in 9 x 9 computer go. In Proceedings of the 23th National Conference on Artificial Intelligence (AAAI’08). 1537--1540.

Digital Library

[11]

Amir Gershman, Amnon Meisels, and Roie Zivan. 2006. Asynchronous forward-bounding for distributed constraints optimization. In Proceedings of the 17th Eureopean Conference on Artificial Intelligence (ECAI’06). 103--107.

Digital Library

[12]

Vibhav Gogate and Rina Dechter. 2003. A new algorithm for sampling CSP solutions uniformly at random. In Proceedings of the 12th International Conference on Principles and Practice of Constraint Programming (CP’06). 711--715.

Digital Library

[13]

Youssef Hamadi, Christian Bessiere, and Joel Quinqueton. 1998. Backtracking in distributed constraint networks. In Proceedings of the 13th European Conference on Artificial Intelligence (ECAI'98). 219--223.

[14]

Katsutoshi Hirayama and Makoto Yokoo. 2003. Distributed partial constraint satisfaction problem. In Proceedings of the 9th International Conference on Principles and Practice of Constraint Programming (CP’03). 222--236.

Digital Library

[15]

Robert Kleinberg. 2005. Nearly tight bounds for the continuum-armed bandit problem. In Advances in Neural Information Processing Systems 17. MIT Press, 697--704.

Digital Library

[16]

Levente Kocsis and Csaba Szepesvári. 2006. Bandit based Monte-Carlo planning. In Proceedings of the 17th European Conference on Machine Learning (ECML’06).

Digital Library

[17]

Thomas Léauté and Boi Faltings. 2011a. Coordinating logistics operations with privacy guarantees. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI'11). 2482--2487.

Digital Library

[18]

Thomas Léauté and Boi Faltings. 2011b. Distributed constraint optimization under stochastic uncertainty. In Proceedings of the 25th National Conference on Artificial Intelligence (AAAI’11). 68--73.

Digital Library

[19]

Thomas Léauté, Brammert Ottens, and Boi Faltings. 2010. Ensuring privacy through distributed computation in multiple-depot vehicle routing problems. In Proceedings of the ECAI’10 Workshop on Artificial Intelligence and Logistics (AILog’10).

[20]

Thomas Léauté, Brammert Ottens, and Radoslaw Szymanek. 2009. FRODO 2.0: An open-source framework for distributed constraint optimization. In Proceedings of the IJCAI’09 Distributed Constraint Reasoning Workshop (DCR’09). 160--164.

[21]

Rajiv T. Maheswaran, Jonathan P. Pearce, and Milind Tambe. 2004. Distributed algorithms for DCOP: A graphical-game-based approach. In Proceedings of ISCA PDCS’04. 432--439.

[22]

Rajiv T. Maheswaran, Milind Tambe, Emma Bowring, Jonathan P. Pearce, and Pradeep Varakantham. 2003. Taking DCOP to the real world: Efficient complete solutions for distributed multi-event scheduling. In Proceedings of the 2nd International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’03). 310--317.

Digital Library

[23]

Pragnesh J. Modi, Wei-Min Shen, Milind Tambe, and Makoto Yokoo. 2005. ADOPT: Asynchronous distributed constraint optimization with quality guarantees. Artificial Intelligence 161 (2005), 149--180.

Digital Library

[24]

Duc Thien Nguyen, William Yeoh, and Hoong Chuin Lau. 2013. Distributed Gibbs: A memory-bounded sampling-based DCOP algorithm. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 167--174.

Digital Library

[25]

Brammert Ottens, Christos Dimitrakakis, and Boi Faltings. 2012. DUCT: An upper confidence bound approach to distributed constraint optimization problems. In Proceedings of the 26th National Conference on Artificial Intelligence (AAAI’12).

Digital Library

[26]

Brammert Ottens and Boi Faltings. 2008a. Asynchronous Open DPOP. In Proceedings of the 10th International Workshop on Distributed Constraint Reasoning (AAMAS’08). http://liawww.epfl.ch /Publications/Archive/Ottens2008.pdf.

Digital Library

[27]

Brammert Ottens and Boi Faltings. 2008b. Coordinating agent plans through distributed constraint optimization. In Proceedings of the ICAPS’08 Multiagent Planning Workshop (MASPLAN’08).

[28]

Adrian Petcu. 2006. FRODO: A FRamework for Open/Distributed Constraint Optimization. Technical Report 2006/001. Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland. 10 pages.

[29]

Adrian Petcu and Boi Faltings. 2005. DPOP: A scalable method for multiagent constraint optimization. In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI’05). 266--271.

Digital Library

[30]

Adrian Petcu and Boi Faltings. 2006. O-DPOP: An algorithm for open/distributed constraint optimization. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI’06). 703--708.

Digital Library

[31]

Pierre Rust, Gauthier Picard, and Fano Ramparany. 2016. Using message-passing DCOP algorithms to solve energy-efficient smart environment configuration problems. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI'16).

Digital Library

[32]

Bab Satomi, Yongjoon Joe, Atsushi Iwasaki, and Makoto Yokoo. 2011. Real-time solving of quantified CSPs based on monte-carlo game tree search. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI'11). 655--661.

Digital Library

[33]

David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, and others. 2016. Mastering the game of go with deep neural networks and tree search. Nature 529, 7587 (2016), 484--489.

[34]

Ruben Stranders, Long Tran-Tranh, Francesco Fave, Alex Rogers, and Nicholas Jennings. 2012. DCOPs and bandits: Exploration and exploitation in decentralised coordination. In Proceedings of the 11th International Conference on Autonomous Agents and Multi Agent Systems (AAMAS'12).

Digital Library

[35]

Evan A. Sultanik, Robert N. Lass, and William C. Regli. 2007. DCOPolis: A framework for simulating and deploying distributed constraint optimization algorithms. In Proceedings of the 9th International Workshop on Distributed Constraint Reasoning (CP-DCR’07).

[36]

Toby Walsh (Ed.). 2011. Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI’11). AAAI Press.

Digital Library

[37]

William Yeoh, Ariel Felner, and Sven Koenig. 2010. BnB-ADOPT: An asynchronous branch-and-bound DCOP algorithm. Journal of Artificial Intelligence Research (JAIR) 38 (2010), 85--133.

[38]

Makoto Yokoo, Edmund H. Durfee, Toru Ishida, and Kazuhiro Kuwabara. 1998. The distributed constraint satisfaction problem: Formalization and algorithms. IEEE Transactions on Knowledge and Data Engineering 10, 5 (Sep/Oct 1998), 673--685.

Digital Library

[39]

Weixiong Zhang, Guandong Wang, Zhao Xing, and Lars Wittenburg. 2003. A comparative study of distributed constraint algorithms. In Distributed Sensor Networks. Springer, 319--338.

Cited By

Zhou XLi GSun ZLiu ZChen WWu JLiu JFeng RZeng G(2024)D-Bot: Database Diagnosis System using Large Language ModelsProceedings of the VLDB Endowment10.14778/3675034.367504317:10(2514-2527)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.14778/3675034.3675043
Gao JChen ZChen DZhang WLi Q(2024)Toward fast belief propagation for distributed constraint optimization problems via heuristic searchAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09643-y38:1Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10458-024-09643-y
Damle STriastcyn AFaltings BGujar S(2024)Differentially private multi-agent constraint optimizationAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09636-x38:1Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10458-024-09636-x
Show More Cited By

Index Terms

DUCT: An Upper Confidence Bound Approach to Distributed Constraint Optimization Problems
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
      1. Cooperation and coordination
      2. Multi-agent systems
    2. Search methodologies
      1. Discrete space search

Recommendations

An Efficient Algorithm for Solving Dynamic Complex DCOP Problems
WI-IAT '09: Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 02

Multi Agent Systems and the Distributed Constraint Optimization Problem (DCOP) formalism offer several asynchronous and optimal algorithms for solving naturally distributed optimization problems efficiently. There has been good application of this ...
Speeding up distributed pseudo-tree optimization procedures with cross edge consistency to solve DCOPs
Abstract
The Distributed Pseudo-tree Optimization Procedure (DPOP) is a well-known message passing algorithm that provides optimal solutions to Distributed Constraint Optimization Problems (DCOPs) in cooperative multi-agent systems. However, the ...
An improved DPOP algorithm based on breadth first search pseudo-tree for distributed constraint optimization

Depth First Search (DFS) pseudo-tree is popularly used as the communication structure in complete algorithms for solving Distributed Constraint Optimization Problems (DCOPs) from multiagent systems. The advantage of a DFS pseudo-tree lies in its ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 8, Issue 5

September 2017

261 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3120923

Editor:
Yu Zheng
Microsoft Research, China

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2017

Accepted: 01 March 2017

Revised: 01 February 2017

Received: 01 November 2016

Published in TIST Volume 8, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Swiss National Science Foundation
Marie Curie Intra-European Fellowship

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
221
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou XLi GSun ZLiu ZChen WWu JLiu JFeng RZeng G(2024)D-Bot: Database Diagnosis System using Large Language ModelsProceedings of the VLDB Endowment10.14778/3675034.367504317:10(2514-2527)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.14778/3675034.3675043
Gao JChen ZChen DZhang WLi Q(2024)Toward fast belief propagation for distributed constraint optimization problems via heuristic searchAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09643-y38:1Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10458-024-09643-y
Damle STriastcyn AFaltings BGujar S(2024)Differentially private multi-agent constraint optimizationAutonomous Agents and Multi-Agent Systems10.1007/s10458-024-09636-x38:1Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10458-024-09636-x
Shi MXiao SFeng X(2023)An Adaptive Ant Colony Algorithm Based on Local Information Entropy to Solve Distributed Constraint Optimization ProblemsInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142359013937:09Online publication date: 21-Jul-2023
https://doi.org/10.1142/S0218001423590139
Gao JChen ZZhang C(2023)Pretrained Parameter Configurator for Large Neighborhood Search to Solve Weighted Constraint Satisfaction Problems2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191732(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191732
Zhang JMao XLi LWu H(2023)TPMARL-MSS Thought-Perception Multi-Agent Reinforcement Learning Meeting Scheduling System2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys)10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00080(542-549)Online publication date: 17-Dec-2023
https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00080
Gao JChen ZChen DZhang WPelachaud CTaylor MFaliszewski PMascardi V(2022)Beyond Uninformed Search: Improving Branch-and-bound Based Acceleration Algorithms for Belief Propagation via Heuristic StrategiesProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3536045(1592-1594)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.5555/3535850.3536045
Chen CNing LZhou RZhang YZhou CFeng S(2022)A GNN-Based Variable Partition Framework for DCOPsWireless Communications & Mobile Computing10.1155/2022/80038872022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/8003887
Li FHe JZhou MFang B(2022)VLSs: A Local Search Algorithm for Distributed Constraint Optimization ProblemsInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142259006636:03Online publication date: 12-Jan-2022
https://doi.org/10.1142/S0218001422590066
Dai ZLiu CYe YHan RYuan YWang GTang J(2022)AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement LearningIEEE INFOCOM 2022 - IEEE Conference on Computer Communications10.1109/INFOCOM48880.2022.9796732(1029-1038)Online publication date: 2-May-2022
https://doi.org/10.1109/INFOCOM48880.2022.9796732
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents