research-article

Experiences with open overlays: a middleware approach to network heterogeneity

Authors:

Gordon S. Blair,

Francois TaianiAuthors Info & Claims

ACM SIGOPS Operating Systems Review, Volume 42, Issue 4

Pages 123 - 136

https://doi.org/10.1145/1357010.1352606

Published: 01 April 2008 Publication History

Abstract

In order to provide an increasing number of functionalities and benefit from sophisticated and application-tailored services from the network, distributed applications are led to integrate an ever-widening range of networking technologies. As these applications become more complex, this requirement for 'network heterogeneity' is becoming a crucial issue in their development. Although progress has been made in the networking community in addressing such needs through the development of network overlays, we claim in this paper that the middleware community has been slow to integrate these advances into middleware architectures, and, hence, to provide the foundational bedrock for heterogeneous distributed applications. In response, we propose our 'open overlays' framework. This framework, which is part of a wider middleware architecture, accommodates 'overlay plug-ins', allows physical nodes to support multiple overlays, supports the stacking of overlays to create composite protocols, and adopts a declarative approach to configurable deployment and dynamic reconfigurability. The framework has been in development for a number of years and supports an extensive range of overlay plug-ins including popular protocols such as Chord and Pastry. We report on our experiences with the open overlays framework, evaluate it in detail, and illustrate its application in a detailed case study of network heterogeneity.

References

[1]

A. Andersen, G. S. Blair, V. Goebel, R. Karlsen, T. Stabell-Kul and W. Yu. Arctic Beans: Configurable and Reconfigurable Enterprise Component Architectures. IEEE Distributed Systems Online, 2 (7), November 2001.

[2]

S. Behnel and A. Buchmann. Overlay Networks - Implementation by Specification. In Proceedings of the ACM/IFIP/Usenix International Middleware Conference, pages 401--410, Grenoble, France, November 2005.

Digital Library

[3]

K. Beven, R. Romanowicz, F. Pappenberger, P. Young and M. Werner. The Uncertainty Cascade in Flood Forecasting. In Proceedings of the ACTIF meeting on Flood Risk, Tromsø Norway, October 2005.

[4]

G. Blair, G. Coulson, A. Andersen, L. Blair, M. Clarke, F. Costa, H. Duran-Limon, T. Fitzpatrick, L. Johnston, R. Moreira, N. Parlavantzas and K. Saikoski. The Design and Implementation of Open ORB V2. IEEE Distributed Systems Online, 2(6), September 2001.

Digital Library

[5]

D. Bromberg and V. Issarny. INDISS: Interoperable Discovery System for Networked Services. In Proceedings of the ACM/IFIP/Usenix International Middleware Conference, pages 164--183, Greoble, France November 2005.

Digital Library

[6]

L. Capra, W. Emmerich and C. Mascolo. CARISMA: Context-Aware Reflective Middleware System for Mobile Applications. IEEE Transactions on Software Engineering, 29(10):929--945, October 2003.

Digital Library

[7]

M. Castro, P. Druschel, A. Kermarrec, and A. Rowstron. SCRIBE: A Large-scale and Decentralized Application-level Multicast Infrastructure. IEEE Journal on Selected Areas in Communications (JSAC), 20(8):1489--1499, October 2002.

Digital Library

[8]

B. Cooper. Trading off Resources Between Overlapping Overlays. In Proceedings of the ACM/IFIP/USENIX 7th International Middleware Conference, pages 101--120 Melbourne, Australia, December 2006.

Digital Library

[9]

C. Cortes, G. Blair and P. Grace. A Multi-protocol Framework for Ad-hoc Service Discovery. In Proceedings of the 4th International Workshop on Middleware for Pervasive and Ad-Hoc Computing (MPAC '06), Melbourne, Australia, November 2006.

Digital Library

[10]

P. Costa, G. Coulson, R. Gold, M. Lad, C. Mascolo, L. Mottola, G. Picco, T. Sivaharan, N. Weerasinghe and S. Zachariadis. The RUNES Middleware for Networked Embedded Systems and its Application in a Disaster Management Scenario. In Proceedings of the. 5th IEEE International Conference on Pervasive Computing and Communications (PERCOM'07), pages 69--78, White Plains, New York, March 2007.

Digital Library

[11]

G. Coulson, G. Blair, D. Hutchison, A. Joolia, K. Lee, J. Ueyama, A. Gomes and Y. Ye. NETKIT: A Software Component-Based Approach to Programmable Networking. ACM SIGCOMM CCR, 33(5): 55--66, October 2003.

Digital Library

[12]

G. Coulson, G. Blair, P. Grace, A. Joolia, K. Lee, K., and J. Ueyama, J. A Component Model for Building Systems Software. In Proceedings of Software Engineering and Applications (SEA'04), Cambridge, MA, USA, ACTA Press, ISBN 0-88986-425-X, November 2004.

[13]

G. Coulson, D. Kuo and J.Brooke. Sensor Networks + Grid Computing = A New Challenge for the Grid? IEEE Distributed Systems Online, 7(12), http://dsonline.computer.org/portal/pages/dsonline/2006/12/o12002.html, December 2006.

Digital Library

[14]

G. Cugola, A. Murphy and G. Picco. Content-Based Publish-Subscribe in a Mobile Environment. Handbook of Mobile Middleware, Corradi, A., and Bellavista, P. eds., pages 257--285, Auerbach Publications, 2006.

[15]

F. Dabek, B. Zhao, P. Druschel, J. Kubiatowicz and I. Stoica. Towards a Common API for Structured P2P Overlays. In Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS), pages 33--44, Berkeley, CA, USA, February 2003.

[16]

N. Davies, A. Friday, S. Wade and G. Blair. L²imbo: A Distributed Systems Platform for Mobile Computing. Mobile Networking Applications, 3(2):143--156, August 1998.

Digital Library

[17]

V. Eide, F. Eliassen and O. Lysne. Supporting Distributed Processing of Time-based Media Streams. In Proceedings of Distributed Objects and Applications (DOA'01), pages 281--288, IEEE, Rome, Italy, September 2001.

Digital Library

[18]

M. Fleury and F. Reverbel. The JBoss Extensible Server. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference, pages 344--373, Rio, Brazil, June 2003.

Digital Library

[19]

I. Foster. Globus Toolkit Version 4: Software for Service-Oriented Systems. In Proceedings of the IFIP International Conference on Network and Parallel Computing, Springer-Verlag LNCS 3779, pages 2--13, 2006.

Digital Library

[20]

N. Furmento, A. Mayer, S. McGough, S. Newhouse, T. Field and J. Darlington. ICENI: Optimisation of Component Applications within a Grid Environment. Parallel Computing, 28(12):1753--1772, December 2002.

Digital Library

[21]

A. Ganesh, A. Kermarrec and L. Massoulie. SCAMP: Peer-to-peer lightweight membership service for large-scale group communication, In Proceedings of the 3rd Int. Workshop on Networked Group Communication, London, UK, 2001.

Digital Library

[22]

P. Grace, G. Blair and S. Samuel. ReMMoC: A Reflective Middleware to Support Mobile Client Interoperability. In Proceedings of the International Symposium on Distributed Objects and Applications (DOA), pages 1170--1187, Catania, Sicily, Italy, November 2003.

[23]

P. Grace, G. Coulson, G. Blair, L. Mathy, W. Yeung, W. Cai, D. Duce, and C. Cooper. GridKit: Pluggable Overlay Networks for Grid Computing. In Proceedings of the International Symposium on Distributed Objects and Applications, pages 1463--1481, Cyprus, October 2004.

[24]

P. Grace, G. Coulson, G. Blair and B. Porter. Deep Middleware for the Divergent Grid. In Proceedings of the 6th IFIP/ACM/USENIX International Middleware Conference, pages 334--353, Grenoble, France, November 2005.

Digital Library

[25]

P. Grace, G. Coulson, G. Blair and B. Porter. A Distributed Architecture Meta Model for Self-Managed Middleware. In Proceedings of the 5th International Workshop on Adaptive and Reflective Middleware (ARM'06), co-located with Middleware 2006, Melbourne, Australia, November 2006.

Digital Library

[26]

K. Henricksen and R. Robinson. A Survey of Middleware for Sensor Networks: State-of-the-art and Future Directions. In Proceedings of the International Workshop on Middleware For Sensor Networks, Melbourne, Australia, November 06, MidSens'06, Vol. 218, ACM Press, 2006.

Digital Library

[27]

V. Hingne, A. Joshi, T. Finin, H. Kargupta and E. Houstis. Towards a Pervasive Grid. In Proceedings of the 17th International Symposium on Parallel and Distributed Processing, IPDPS, Nice, France, April 2003.

Digital Library

[28]

D. Hughes, P. Greenwood, G. Coulson, G. Blair, F. Pappenberger, P. Smith and K. Beven. An Intelligent and Adaptable Flood Monitoring and Warning System. In Proceedings of the 5th UK E-Science All Hands Meeting (AHM'06), Nottingham, UK, September 2006 http://www.allhands.org.uk/2006/proceedings/proceedings/.

[29]

A. Joseph, A. de Lespinasse, J. Tauber, D. Gifford and M. Kaashoek. Rover: a Toolkit for Mobile Information Access. In Proceedings of the 15th ACM Symposium on Operating Systems Principles, Copper Mountain, Colorado, United States, Jones, ed., SOSP'95, pages 156--171, ACM Press, December 1995.

Digital Library

[30]

F. Kon, M. Román, P. Liu, J. Mao, T. Yamane, L. Magalhães and R. Campbell. Monitoring, Security, and Dynamic Configuration with the dynamicTAO Reflective ORB. In Proceedings of the IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing (Middleware'2000), pages 121--143, New York, NY, USA, April 2000.

Digital Library

[31]

B. Lagaisse and W. Joosen. True and Transparent Distributed Composition of Aspect-Components. In Proceedings of the International ACM/IFIP/Usenix Middleware Conference, LNCS 4290, pages 42--61, Melbourne, December 2006.

Digital Library

[32]

B. Li, J. Guo and M. Wan. iOverlay: A Lightweight Middleware Infrastructure for Overlay Application Implementations. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference (Middleware 2004), pages 135--154, Toronto, Canada, October 2004.

Digital Library

[33]

B. Loo, T. Condie, J. Hellerstein, P. Maniatis, T. Roscoe and I. Stoica. Implementing Declarative Overlays. SIGOPS OSR, 39(5):75--90, 2005.

Digital Library

[34]

B. Maniymaran, M Bertier, and A. Kermarrec. Build One, Get One Free: Leveraging the Coexistence of Multiple P2P Overlay Networks. In Proceedings of ICDCS 2007, Toronto, Canada, June 2007.

Digital Library

[35]

L. Mathy, R. Canonico and D. Hutchinson. An Overlay Tree Building Control Protocol. In Proceedings of the 3rd International COST264 Workshop on Networked Group Communication, pages 76--87, London, UK, November 2001.

Digital Library

[36]

B. Porter, F. Taiani and G. Coulson. Generalised Repair for Overlay Networks. In Proceedings of International Symposium on Reliable Distributed Systems (SRDS 2006), pages 132--142, Leeds, UK, October 2006.

Digital Library

[37]

O. Riva. Contory: A Middleware for the Provisioning of Context Information on Smart Phones. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference, pages 219--239, Melbourne, Australia, December 2006.

Digital Library

[38]

A. Rodriguez, C. Killian, S. Bhat, D. Kostic and A. Vahdat. MACEDON: Methodology for automatically creating, evaluating, and designing overlay networks. In Proceedings of the USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI2004), pages 267--280, San Francisco, CA, USA, March 2004.

Digital Library

[39]

M. Roman, D. Mickunas, F. Kon and R. Campbell. LegORB and Ubiquitous CORBA. In Proceedings of the Workshop on Reflective Middleware, IFIP/ACM Middleware'2000, IBM Palisades Executive Conference Center, NY, April 2000.

[40]

M. Roman and N. Islam. Dynamically Programmable and Reconfigurable Middleware Services. In Proceedings of the 5th ACM/IFIP/USENIX International Conference on Middleware, pages 372--396, Toronto, Canada, November 2004.

Digital Library

[41]

A. Rowstron and P. Druschel. Pastry: Scalable, Distributed Object Location and Routing for Large-scale Peer-to-Peer Systems. In Proceedings of the ACM/IFIP/USENIX International Conference on Middleware, pages 329--350, Heidelberg, Germany November, 2001.

Digital Library

[42]

S. Sadjadi, P. McKinley and E. Kasten. Architecture and Operation of an Adaptable Communication Substrate. In Proceedings of the 9th IEEE International Workshop on Future Trends of Distributed Computing Systems (FTDCS'03), pages 46--55, San Juan, Puerto Rico, May 2003.

Digital Library

[43]

A. Sage and C. Cuppan. On the Systems Engineering and Management of Systems of Systems and Federations of Systems. Information, Knowledge, Systems Management, 2(4): 325--345, 2001.

Digital Library

[44]

I. Stoica, R. Morris, R. Karger, M. Kaashoek and H. Balakarishnan. Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications, In Proceedings of ACM SIG-COMM, pages 149--160 San Diego, August 2001.

Digital Library

[45]

R. van Renesse, Y. Minsky and M. Hayden. A Gossip-Based Failure Detection Service. In Proceedings of the 1st IFIP International Conference on Middleware, pages 55--70, Lake District, UK, September 1998.

Digital Library

[46]

R. van Renesse, K. Birman, M. Hayden, A. Vaysburd, and D. Karr. Adaptive Systems Using Ensemble. Software Practice and Experience, 28(9): 963--979, August 1998.

Digital Library

[47]

G. Xiaohui and K. Nahrstedt. An Event-Driven, User-Centric, QoS-aware Middleware Framework for Ubiquitous Multimedia Applications. In Proceedings of 9th ACM Multimedia (Multimedia Middleware Workshop), Ottawa, Canada, October 2001.

Digital Library

[48]

C. Zhang and H. Jacobsen. Refactoring Middleware with Aspects. IEEE Transactions on Parallel and Distributed Systems, 14(11):1058--1073, November 2003.

Digital Library

Cited By

Bouget SBromberg YTaïani FVentresque A(2017)Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFNDistributed Applications and Interoperable Systems10.1007/978-3-319-59665-5_7(101-114)Online publication date: 24-May-2017
https://doi.org/10.1007/978-3-319-59665-5_7
Sobhy DBahsoon RMinku LKazman R(2016)Diversifying Software Architecture for Sustainability: A Value-Based PerspectiveSoftware Architecture10.1007/978-3-319-48992-6_4(55-63)Online publication date: 3-Nov-2016
https://doi.org/10.1007/978-3-319-48992-6_4
Daniels WProença JClarke DJoosen WHughes DKruchten PBecker SSchneider J(2015)RefractionProceedings of the 18th International ACM SIGSOFT Symposium on Component-Based Software Engineering10.1145/2737166.2737168(27-36)Online publication date: 4-May-2015
https://dl.acm.org/doi/10.1145/2737166.2737168
Show More Cited By

Index Terms

Experiences with open overlays: a middleware approach to network heterogeneity
1. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Organizing principles for web applications

Recommendations

Experiences with open overlays: a middleware approach to network heterogeneity
Eurosys '08: Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008

In order to provide an increasing number of functionalities and benefit from sophisticated and application-tailored services from the network, distributed applications are led to integrate an ever-widening range of networking technologies. As these ...
Virtual overlays: an approach to the management of competing or collaborating overlay structures
DAIS'08: Proceedings of the 8th IFIP WG 6.1 international conference on Distributed applications and interoperable systems

Overlay networks are a technique whereby application developers create virtual customized networks on top of physical networks. Recent implementations of peer-to-peer applications such as file sharing and VoIP have increasingly meant that overlay ...
A formal framework for middleware behavioural specification

The number of open specifications of middleware systems and middleware services is increasing. Despite their complexity, they are traditionally described through APIs (the operation signatures) and informal prose (the behaviour). This fact often leads ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review

ACM SIGOPS Operating Systems Review Volume 42, Issue 4

EuroSys '08

May 2008

321 pages

ISSN:0163-5980

DOI:10.1145/1357010

Issue’s Table of Contents

Eurosys '08: Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008
April 2008
346 pages
ISBN:9781605580135
DOI:10.1145/1352592
General Chair:
Joe Sventek
University of Glasgow, UK
,
Program Chair:
Steven Hand
University of Cambridge, UK

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2008

Published in SIGOPS Volume 42, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
680
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bouget SBromberg YTaïani FVentresque A(2017)Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFNDistributed Applications and Interoperable Systems10.1007/978-3-319-59665-5_7(101-114)Online publication date: 24-May-2017
https://doi.org/10.1007/978-3-319-59665-5_7
Sobhy DBahsoon RMinku LKazman R(2016)Diversifying Software Architecture for Sustainability: A Value-Based PerspectiveSoftware Architecture10.1007/978-3-319-48992-6_4(55-63)Online publication date: 3-Nov-2016
https://doi.org/10.1007/978-3-319-48992-6_4
Daniels WProença JClarke DJoosen WHughes DKruchten PBecker SSchneider J(2015)RefractionProceedings of the 18th International ACM SIGSOFT Symposium on Component-Based Software Engineering10.1145/2737166.2737168(27-36)Online publication date: 4-May-2015
https://dl.acm.org/doi/10.1145/2737166.2737168
Malatras A(2015)State-of-the-art survey on P2P overlay networks in pervasive computing environmentsJournal of Network and Computer Applications10.1016/j.jnca.2015.04.01455(1-23)Online publication date: Sep-2015
https://doi.org/10.1016/j.jnca.2015.04.014
Hakiri ABerthou PGokhale ASchmidt DThierry G(2014)Supporting SIP-based end-to-end Data Distribution Service QoS in WANsJournal of Systems and Software10.1016/j.jss.2014.03.07895(100-121)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1016/j.jss.2014.03.078
Hughes DBencomo NMorin BHuygens CShen ZMan K(2013)S-Theory: A Unified Theory of Multi-paradigm Software DevelopmentGrid and Pervasive Computing10.1007/978-3-642-38027-3_78(715-722)Online publication date: 2013
https://doi.org/10.1007/978-3-642-38027-3_78
Urzaiz GVilla DVillanueva FLopez J(2012)Process-in-Network: A Comprehensive Network Processing ApproachSensors10.3390/s12060811212:6(8112-8134)Online publication date: 12-Jun-2012
https://doi.org/10.3390/s120608112
Thoelen KHughes DMatthys NFang LDobson SQiang YBai WMan KGuan SPreuveneers DMichiels SHuygens CJoosen W(2012)A reconfigurable component model with semantic type system for dynamic WSN applicationsJournal of Internet Services and Applications10.1007/s13174-012-0064-03:3(277-290)Online publication date: 23-Aug-2012
https://doi.org/10.1007/s13174-012-0064-0
Bouget SBromberg YMercier HRiviere ETaiani F(2018)Mind the Gap: Autonomous Detection of Partitioned MANET Systems using Opportunistic Aggregation2018 IEEE 37th Symposium on Reliable Distributed Systems (SRDS)10.1109/SRDS.2018.00025(143-152)Online publication date: Oct-2018
https://doi.org/10.1109/SRDS.2018.00025
Filho RPorter B(2017)Defining Emergent Software Using Continuous Self-Assembly, Perception, and LearningACM Transactions on Autonomous and Adaptive Systems10.1145/309269112:3(1-25)Online publication date: 20-Sep-2017
https://dl.acm.org/doi/10.1145/3092691
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents