research-article

MapReduce-based optimization of overlay networks using particle swarm optimization

Author:

Simone A. LudwigAuthors Info & Claims

GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

Pages 1031 - 1038

https://doi.org/10.1145/2576768.2598269

Published: 12 July 2014 Publication History

Abstract

An overlay network is a virtual network that is built on top of the real network such as the Internet. Cloud computing, peer-to-peer networks, and client-server applications are examples of overlay networks since their nodes run on top of the Internet. The major needs of overlay networks are content distribution and caching, file sharing, improved routing, multicast and streaming, ordered message delivery, and enhanced security and privacy. The focus of this paper is the optimization of overlay networks using a Particle Swarm Optimization (PSO) approach. However, since the ever growing need for more infrastructure causes the number of network nodes to grow significantly, the parallelization of the PSO approach becomes a necessity. In this paper, the MapReduce concept, proposed by Google, is adopted for the PSO approach in order to be able to optimize large-scale networks. MapReduce is easy to implement since it is based on the divide and conquer method, and implementation frameworks such has Hadoop allow for scalability and fault tolerance. Experiments of the MapReduce based PSO algorithm are performed to investigate the solution quality and scalability of the approach.

References

[1]

S. Tarkoma, "Overlay Networks: Toward Information Networking", CRC Press, Auerbach Publications, ISBN: 978--1--4398--1371--3, 2010.

Digital Library

[2]

N. Lam, Z. Dziong, L. G. Mason, "Service Overlay Network Design with Reliability Constraints", Proceedings of IEEE 7th International Workshop on the Design of Reliable Communication Networks, Washington, D.C., USA, 2009.

[3]

T. Baeck, D. Fogel, and Z. Michalewicz, "Handbook of Evolutionary Computation", IOP Publ. Ltd., Bristol, UK, 1997.

Digital Library

[4]

G. D. Caro, M. Dorigo, "AntNet: distributed stigmergetic control for communications networks", J. Artif. Int. Res. 9, 1 (December 1998), 317--365, 1998.

Digital Library

[5]

J. Montoya, Y. Donoso, E. Montoya, D. Echeverri, "Multiobjective model for multicast overlay networks over IP/MPLS using MOEA", Proceedings of International Conference on Optical Network Design and Modeling, 1--6, 2008.

[6]

A. Abraham, H. Liu, Y. Badr, C. Grosan, "A multi-swarm approach for neighbor selection in peer-to-peer networks", Proceedings of the 5th international conference on Soft computing as transdisciplinary science and technology (CSTST '08), ACM, New York, NY, USA, 178--184, 2008.

Digital Library

[7]

D. Montana, T. Hussain, T. Saxen, "Adaptive reconfiguration of data networks using genetic algorithms", Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1141--1149, San Francisco, CA, USA, 2002.

Digital Library

[8]

A. J. Ramirez, D. B. Knoester, B. H. C. Cheng, P. K. McKinley, "Plato: A Genetic Algorithm Approach to Run-Time Reconfiguration in Autonomic Computing Systems", Journal of Cluster Computing, 2010.

Digital Library

[9]

S. A. Ludwig, "Nature-Inspired Reconfiguration of Overlay Networks", Proceedings of Third World Congress on Nature and Biologically Inspired Computing (NaBIC), Salamanca, Spain, 2011.

[10]

S. A. Ludwig, "Scalability Analysis: Reconfiguration of Overlay Networks Using Nature-Inspired Algorithms", in Advances in Intelligent Modelling and Simulation: Artificial Intelligence-based Models and Techniques in Scalable Computing, Studies in Computational Intelligence series, vol. 422, pp. 137--154, Springer, 2012.

[11]

MPI (Message Passing Interface), 2014, http://www-unix.mcs.anl.gov/mpi/.

[12]

PVM (Parallel Virtual Machine), 2014, http://www.csm.ornl.gov/pvm/.

[13]

C. Zhou, "Fast parallelization of differential evolution algorithm using MapReduce", Proceedings of the 12th annual conference on Genetic and evolutionary computation, 2010.

Digital Library

[14]

C. T. Chu, S. K. Kim, Y. A. Lin, et al., "Map-Reduce for Machine Learning on Multicore", In NIPS (2006), pp. 281--288 by edited by Bernhard Schoelkopf, John C. Platt, Thomas Hoffman.

[15]

K. Kambatla, N. Rapolu, S. Jagannathan, and A. Grama, "Asynchronous Algorithms in MapReduce", Proceedings of the 2010 IEEE International Conference on Cluster Computing (CLUSTER '10), Washington, DC, USA, 245--254, 2010.

Digital Library

[16]

A. Verma, X. Llora, D. E. Goldberg, and R. H. Campbell, "Scaling Genetic Algorithms Using MapReduce", Proceedings of the 2009 Ninth International Conference on Intelligent Systems Design and Applications (ISDA '09), Washington, DC, USA, 2009.

Digital Library

[17]

J. Schaffer and L. Shelman, "On Crossover as an Evolutionary Viable Strategy", In R. Belw and L. Booker, editors, Proceedisn of the 4th International Conference on Genetic Algorithms, Morgan Kaufmann, 1991.

[18]

C. Jin, C. Vecchiola, R. Buyya, "MRPGA: An Extension of MapReduce for Parallelizing Genetic Algorithms", Proceedings of IEEE Fourth International Conference on eScience, 2008.

Digital Library

[19]

A. W. McNabb, C. K. Monson, K. D. Seppi, "Parallel PSO using MapReduce", Proceedings of Congress of Evolutionary Computation (CEC), 2007.

[20]

I. Aljarah and S. A. Ludwig, "MapReduce Intrusion Detection System based on a Particle Swarm Optimization Clustering Algorithm", Proceedings of the 2013 IEEE Congress on Evolutionary Computation, Cancun, Mexico, June 2013.

[21]

K. Keeton, C. Santos, D. Beyer, J. Chase, J. Wilkes, "Designing for disasters", Proceedings of the 3rd USENIX Conference on File and Storage Technologies, pp. 59--62. Berkeley, CA, USA, 2004.

Digital Library

[22]

J. Kennedy, R. Eberhart, "Particle swarm optimization", Proceedings of IEEE International Conference on Neural Networks, Perth, Western Australia, 1995.

[23]

A. Engelbrecht, "Computational Intelligence - An Introduction", 2nd Edition, Wiley, 2007.

Digital Library

[24]

M. Clerc, "Discrete particle swarm optimization - illustrated by the traveling salesman problem". New Optimization Techniques in Engineering, In Studies in Fuzziness and Soft Computing, Springer, 2004.

[25]

Apache Software Foundation, Hadoop MapReduce, 2011. http://hadoop.apache.org/mapreduce.

Cited By

Li LLi YBose SShen G(2023)A generic parallel optimization framework for solving hard problems in optical networksComputer Communications10.1016/j.comcom.2022.12.023199(177-185)Online publication date: Feb-2023
https://doi.org/10.1016/j.comcom.2022.12.023
Ma KLiu SLin YYu ZJi K(2017)Parallel Grouping Particle Swarm Optimization with Stream Processing Paradigm2017 IEEE 19th International Conference on High Performance Computing and Communications Workshops (HPCCWS)10.1109/HPCCWS.2017.00010(22-26)Online publication date: Dec-2017
https://doi.org/10.1109/HPCCWS.2017.00010
Agarwal SRanjan P(2017)MR-TP-QFPSO: map reduce two phases quantum fuzzy PSO for feature selectionInternational Journal of System Assurance Engineering and Management10.1007/s13198-017-0682-99:4(888-900)Online publication date: 2-Nov-2017
https://doi.org/10.1007/s13198-017-0682-9
Show More Cited By

Index Terms

MapReduce-based optimization of overlay networks using particle swarm optimization
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction

Recommendations

Center particle swarm optimization

Center particle swarm optimization algorithm (CenterPSO) is proposed where a center particle is incorporated into linearly decreasing weight particle swarm optimization (LDWPSO). Unlike other ordinary particles in LDWPSO, the center particle has no ...
Euclidean Particle Swarm Optimization
ICINIS '09: Proceedings of the 2009 Second International Conference on Intelligent Networks and Intelligent Systems

Particle swarm optimization (PSO) is a swarm intelligence algorithm, has been successfully applied to many engineering optimization problems and shown its high search speed in these applications. However, as the dimension and the number of local optima ...
Using quantum-behaved particle swarm optimization algorithm to solve non-linear programming problems
Distributed Algorithms in Science and Engineering

In this paper, we focus on solving non-linear programming (NLP) problems using quantum-behaved particle swarm optimization (QPSO). After a brief introduction to the original particle swarm optimization (PSO), we describe the origin and development of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

July 2014

1478 pages

ISBN:9781450326629

DOI:10.1145/2576768

Editor-in-chief:
Christian Igel
Ruhr University of Bochum, University of Copenhagen
,
General Chair:
Dirk V. Arnold
Dalhousie University

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Office of Experimental Program to Stimulate Competitive Research

Conference

GECCO '14

Sponsor:

SIGEVO

GECCO '14: Genetic and Evolutionary Computation Conference

July 12 - 16, 2014

BC, Vancouver, Canada

Acceptance Rates

GECCO '14 Paper Acceptance Rate 180 of 544 submissions, 33%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
172
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li LLi YBose SShen G(2023)A generic parallel optimization framework for solving hard problems in optical networksComputer Communications10.1016/j.comcom.2022.12.023199(177-185)Online publication date: Feb-2023
https://doi.org/10.1016/j.comcom.2022.12.023
Ma KLiu SLin YYu ZJi K(2017)Parallel Grouping Particle Swarm Optimization with Stream Processing Paradigm2017 IEEE 19th International Conference on High Performance Computing and Communications Workshops (HPCCWS)10.1109/HPCCWS.2017.00010(22-26)Online publication date: Dec-2017
https://doi.org/10.1109/HPCCWS.2017.00010
Agarwal SRanjan P(2017)MR-TP-QFPSO: map reduce two phases quantum fuzzy PSO for feature selectionInternational Journal of System Assurance Engineering and Management10.1007/s13198-017-0682-99:4(888-900)Online publication date: 2-Nov-2017
https://doi.org/10.1007/s13198-017-0682-9
Anitta J Jeseela Judith J(2016)Efficient Overlay-Based Parallel Data Mining architecture2016 International Conference on Energy Efficient Technologies for Sustainability (ICEETS)10.1109/ICEETS.2016.7583812(538-543)Online publication date: Apr-2016
https://doi.org/10.1109/ICEETS.2016.7583812
Ludwig S(2016)Running krill herd algorithm on Hadoop: A performance study2016 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC.2016.7744100(2504-2510)Online publication date: Jul-2016
https://doi.org/10.1109/CEC.2016.7744100
Ludwig SDawar D(2015)Parallelization of Enhanced Firework Algorithm using MapReduceInternational Journal of Swarm Intelligence Research10.4018/IJSIR.20150401026:2(32-51)Online publication date: 1-Apr-2015
https://dl.acm.org/doi/10.4018/IJSIR.2015040102

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents