Article

A Peer-to-Peer Replica Location Service Based on a Distributed Hash Table

Authors:

Martin FrankAuthors Info & Claims

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

Page 56

https://doi.org/10.1109/SC.2004.7

Published: 06 November 2004 Publication History

Abstract

A Replica Location Service (RLS) allows registration and discovery of data replicas. In earlier work, we proposed an RLS framework and described the performance and scalability of an RLS implementation in Globus Toolkit Version 3.0. In this paper, we present a Peer-to-Peer Replica Location Service (P-RLS) with properties of self-organization, fault-tolerance and improved scalability. P-RLS uses the Chord algorithm to self-organize PRLS servers and exploits the Chord overlay network to replicate P-RLS mappings adaptively. Our performance measurements demonstrate that update and query latencies increase at a logarithmic rate with the size of the P-RLS network, while the overhead of maintaining the P-RLS network is reasonable. Our simulation results for adaptive replication demonstrate that as the number of replicas per mapping increases, the mappings are more evenly distributed among P-RLS nodes. We introduce a predecessor replication scheme and show it reduces query hotspots of popular mappings by distributing queries among nodes.

References

[1]

{1} A. Chervenak, E. Deelman, I. Foster, L. Guy, W. Hoschek, A. Iamnitchi, C. Kesselman, P. Kunst, M. Ripeanu, B, Schwartzkopf, H, Stockinger, K. Stockinger, B. Tierney, "Giggle: A Framework for Constructing Sclable Replica Location Services," presented at SC2002 Conference, Baltimore, MD, November 2002.

Digital Library

[2]

{2} A. L. Chervenak, Naveen Palavalli, Shishir Bharathi, Carl Kesselman, Robert Schwartzkopf, "Performance and Scalability of a Replica Location Service," presented at High Performance Distributed Computing Conference (HPDC-13), Honolulu, HI, June 2004.

Digital Library

[3]

{3} "The Earth Systems Grid." http://www.earthsystemsgrid.org.

[4]

{4} "LIGO - Laser Interferometer Gravitational Wave Observatory." http://www.ligo.caltech.edu/.

[5]

{5} I. Stoica, Robert Morris, David Karger, M. Frans Kaashoek, and Hari Balakrishnan, "Chord: A scalable peer-to-peer lookup service for internet applications," presented at ACM SIGCOMM, 2001.

Digital Library

[6]

{6} B. Bloom, "Space/Time Trade-offs in Hash Coding with Allowable Errors," Communications of ACM, vol. 13, pp. 422-426.

Digital Library

[7]

{7} P. Avery and I. Foster, "The GriPhyN Project: Towards Petascale Virtual Data Grids," 2001. www.griphyn.org.

[8]

{8} E. Deelman, J. Blythe, Y. Gil, and C. Kesselman, "Pegasus: Planning for Execution in Grids," GriPhyN Project Technical Report 2002-20.

[9]

{9} E. Deelman, et. al, "Mapping Abstract Complex Workflows onto Grid Environments," Journal of Grid Computing, vol. 1, pp. 25-39.

[10]

{10} "Gnutella." http://gnutella.wego.com.

[11]

{11} M. Ripeanu, I. Foster, and A. Iamnitchi., "Mapping the Gnutella network: properties of large-scale peer-to-peer systems and implications for system design," IEEE Internet Computing Journal.

Digital Library

[12]

{12} S. Saroiu, P. K. Gummadi, and S. D. Gribble, "A Measurement Study of Peer-to-Peer File Sharing Systems," presented at Multimedia Computing and Networking, 2002.

[13]

{13} S. Sen, Jia Wong, "Analyzing peer-to-peer traffic across large networks," presented at Proceedings of the Second ACM SIGCOMM Workshop on Internet Measurment, November 2002.

Digital Library

[14]

{14} S. Ratnasamy, S. Shenker, and I. Stoica, "Routing algorithms for DHTs: Some open questions," presented at IPTPS02, Cambridge, USA, March 2002.

Digital Library

[15]

{15} B. Y. Zhao, J. D. Kubiatowicz, and A. D. Joseph, "Tapestry: An infrastructure for fault-resilient wide-area location and routing," U.C. Berkeley, Berkeley Technical Report UCB-CSD-01-1141, April 2001.

Digital Library

[16]

{16} A. Rowstron, P. Druschel., "Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems," presented at International Conference on Distributed Systems Platforms (Middleware), November 2001.

Digital Library

[17]

{17} S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, "A Scalable Content-Addressable Network," presented at ACM SIGCOMM, August 2001.

Digital Library

[18]

{18} F. Kaashoek, David R. Karger, "Koorde: A Simple Degree-optimal Hash Table," presented at 2nd International Workshop on Peer-to-Peer Systems (IPTPS '03), February, 2003.

[19]

{19} C. Baru, R. Moore, et al., "The SDSC Storage Resource Broker," presented at CASCON'98 Conference.

Digital Library

[20]

{20} O. Tatebe, et al., "Worldwide Fast File Replication on Grid Datafarm," presented at 2003 Computing in High Energy and Nuclear Physics (CHEP03), March 2003.

[21]

{21} L. Guy, P. Kunszt, E. Laure, H. Stockinger, K. Stockinger, "Replica Management in Data Grids," presented at Global Grid Forum 5.

[22]

{22} M. Ripeanu, Ian Foster, "A Decentralized, Adaptive, Replica Location Mechanism," presented at 11th IEEE International Symposium on High Performance Distributed Computing (HPDC-11), Edinburgh, Scotland, June, 2002.

Digital Library

[23]

{23} Y. Breitbart, H. Korth, "Replication and Consistency: Being Lazy Helps Sometimes," presented at 16th ACM SIGACT/SIGMOD Symposium on the Principles of Database Systems, Tucson, AZ, 1997.

Digital Library

[24]

{24} J. Gray, P. Helland, P. O'Neil, D. Shasha, "The Dangers of Replication and a Solution," presented at ACM SIGMOD Conference, 1996.

Digital Library

[25]

{25} K. Petersen, et al., "Flexible Update Propagation for Weakly Consistent Replication," presented at 16th ACM Symposium on Operating Systems Principles (SOSP-16), Saint Malo, France, 1997.

Digital Library

[26]

{26} D. B. Terry, K. Petersen, M. J. Spreitzer, and M. M. Theimer, "The Case for Non-transparent Replication: Examples from Bayou," presented at 14th International Conference on Data Engineering, 1998.

[27]

{27} Wiesmann, M., F. Pedone, A. Schiper, B. Kemme, G. Alonso, "Database Replication Techniques: A Three Paramater Classification," presented at 19th IEEE Symposium on Reliable Distributed Systems, Nuernberg, Germany, 2000.

Digital Library

[28]

{28} J. Sidell, P.M. Aoki, A. Sah, C. Staelin, M. Stonebraker, A. Yu, "Data Replication in Mariposa, "Data Replication in Mariposa," presented at 12th International Conference on Data Engineering, New Orleans, LA, 1996.

Digital Library

[29]

{29} J. T. W. Page, R. G. Guy, G. J. Popek, J. S. Heidemann, W. Mak, and D. Rothmeier, "Management of Replicated Volume Location Data in the Ficus Replicated File System," presented at USENIX Conference, 1991.

[30]

{30} R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, B. Lyon, "Design and implementation of the Sun Network File System," presented at USENIX Conference, June 1985.

[31]

{31} G. Popek, The Locus Distributed System Architecture: The MIT Press, 1986.

Digital Library

[32]

{32} M. Satyanarayanan, J. J. Kistler, P. Kumar, M. E. Okasaki, E. H. Siegel, D. C. Steere, "Coda: A Highly Available System for a Distributed Workstation Environment," IEEE Transactions on Computers, vol. 39(4), pp. 447-459, April 1990.

Digital Library

[33]

{33} E. R. Zayas, C. F. Everhart, "Design and Specification of the Cellular Andrew Environment," Carnegie-Mellon University Technical Report CMU-ITC-070, August 1988.

[34]

{34} J. Aspnes, Gauri Shah, "Skip Graphs," presented at Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, January 2003.

Digital Library

[35]

{35} N. Harvey, M. Jones, S. Saroiu, M. Theimer, and A. Wolman, "SkipNet: A Scalable Overlay Network with Practical Locality Properties," presented at Fourth USENIX Symposium on Internet Technologies and Systems (USITS '03), Seattle, WA, March 2003.

Digital Library

[36]

{36} C. Plaxton, R. Rajaraman, A. Richa, "Accessing nearby copies of replicated objects in a distributed environment," presented at ACM SPAA, Newport, Rhode Island, June 1997.

Digital Library

[37]

{37} M. Castro, P. Druschel, Y. C. Hu, A. Rowstron, "Topology-aware routing in structured peer-to-peer overlay networks," presented at Intl. Workshop on Future Directions in Distributed Computing, June 2002.

[38]

{38} W. Pugh, "Skip Lists: A Probabilistic Alternative to BalancedTrees," presented at Workshop on Algorithms and Data Structures, 1989.

Digital Library

[39]

{39} H. Zhang, A. Goel, R. Govindan, "Incremental Optimization In Distributed Hash Table Systems," presented at ACM SIGMETRICS, 2003.

Digital Library

[40]

{40} E. Sit, R. Morris, "Security considerations for peer-to-peer distributed hash tables," presented at 1st International Workshop on Peer-to-Peer Systems (IPTPS '02), Cambridge, MA, March 2002.

Digital Library

[41]

{41} M. Castro, P. Drushel, A. Ganesh, A. Rowstron, and D. Wallach, "Secure routing for structured peer-to-peer overlay networks," presented at 5th Usenix Symposium on Operating Systems Design and Implementation (OSDI), Boston, MA, 2002.

Digital Library

Cited By

Vorhemus CSchikuta E(2017)A data-oriented architecture for loosely coupled real-time information systemsProceedings of the 19th International Conference on Information Integration and Web-based Applications & Services10.1145/3151759.3151770(472-481)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.1145/3151759.3151770
Tian YRhodes P(2016)A location service for partial spatial replicas implementing an R-tree in a relational databaseJournal of Parallel and Distributed Computing10.1016/j.jpdc.2016.01.00390:C(9-21)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.jpdc.2016.01.003
Su YAgrawal GWoodring JMyers KWendelberger JAhrens JParashar MWeissman JEpema DFigueiredo R(2013)Taming massive distributed datasetsProceedings of the 22nd international symposium on High-performance parallel and distributed computing10.1145/2493123.2462906(13-24)Online publication date: 17-Jun-2013
https://dl.acm.org/doi/10.1145/2493123.2462906
Show More Cited By

Recommendations

Reducing data replication overhead in DHT based peer-to-peer system
HPCC'06: Proceedings of the Second international conference on High Performance Computing and Communications

DHT based p2p systems are appeared to provide scalable storage services with idle resource from many unreliable clients. If a DHT is used in storage intensive applications where data loss must be minimized, quick replication is especially important to ...
Search and replication in unstructured peer-to-peer networks
ACM International Conference on Supercomputing 25th Anniversary Volume

Decentralized and unstructured peer-to-peer networks such as Gnutella are attractive for certain applications because they require no centralized directories and no precise control over network topology or data placement. However, the flooding-based ...
Analysis of search and replication in unstructured peer-to-peer networks
SIGMETRICS '05: Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems

This paper investigates the effect of the number of file replicas on search performance in unstructured peer-to-peer networks. We observe that for a search network with a random graph topology where file replicas are uniformly distributed, the hop ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '04: Proceedings of the 2004 ACM/IEEE conference on Supercomputing

November 2004

724 pages

ISBN:0769521533

General Chair:
Jeff Huskamp

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

IEEE Computer Society

United States

Publication History

Published: 06 November 2004

Check for updates

Author Tags

Qualifiers

Article

Conference

SC '04

Sponsor:

SIGARCH
IEEE-CS

SC '04: International Conference for High Performance Computing, Networking, Storage and Analysis

November 6 - 12, 2004

Acceptance Rates

SC '04 Paper Acceptance Rate 60 of 200 submissions, 30%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
26
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vorhemus CSchikuta E(2017)A data-oriented architecture for loosely coupled real-time information systemsProceedings of the 19th International Conference on Information Integration and Web-based Applications & Services10.1145/3151759.3151770(472-481)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.1145/3151759.3151770
Tian YRhodes P(2016)A location service for partial spatial replicas implementing an R-tree in a relational databaseJournal of Parallel and Distributed Computing10.1016/j.jpdc.2016.01.00390:C(9-21)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1016/j.jpdc.2016.01.003
Su YAgrawal GWoodring JMyers KWendelberger JAhrens JParashar MWeissman JEpema DFigueiredo R(2013)Taming massive distributed datasetsProceedings of the 22nd international symposium on High-performance parallel and distributed computing10.1145/2493123.2462906(13-24)Online publication date: 17-Jun-2013
https://dl.acm.org/doi/10.1145/2493123.2462906
Su YAgrawal GWoodring JMyers KWendelberger JAhrens JParashar MWeissman JEpema DFigueiredo R(2013)Taming massive distributed datasetsProceedings of the 22nd international symposium on High-performance parallel and distributed computing10.1145/2462902.2462906(13-24)Online publication date: 17-Jun-2013
https://dl.acm.org/doi/10.1145/2462902.2462906
Ma JLiu WGlatard T(2013)A classification of file placement and replication methods on gridsFuture Generation Computer Systems10.1016/j.future.2013.02.00629:6(1395-1406)Online publication date: 1-Aug-2013
https://dl.acm.org/doi/10.1016/j.future.2013.02.006
Brown PZhu MWu QLu XBalman MByna S(2011)Network-aware data movement advisorProceedings of the first international workshop on Network-aware data management10.1145/2110217.2110222(31-40)Online publication date: 14-Nov-2011
https://dl.acm.org/doi/10.1145/2110217.2110222
Urdaneta GPierre GSteen M(2011)A survey of DHT security techniquesACM Computing Surveys10.1145/1883612.188361543:2(1-49)Online publication date: 4-Feb-2011
https://dl.acm.org/doi/10.1145/1883612.1883615
Tian YRhodes P(2011)A Fast Location Service for Partial Spatial ReplicasProceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing10.1109/Grid.2011.32(190-197)Online publication date: 21-Sep-2011
https://dl.acm.org/doi/10.1109/Grid.2011.32
Pérez JGarcía-Carballeira FCarretero JCalderón AFernández J(2010)Branch replication schemeFuture Generation Computer Systems10.1016/j.future.2009.05.01526:1(12-20)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1016/j.future.2009.05.015
Gu YGrossman R(2010)SectorFuture Generation Computer Systems10.1016/j.future.2009.05.00926:5(720-728)Online publication date: 1-May-2010
https://dl.acm.org/doi/10.1016/j.future.2009.05.009
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten