research-article

Decentralized resource discovery mechanisms for distributed computing in peer-to-peer environments

Authors:

Joan Manuel Marques,

Xavier VilajosanaAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 45, Issue 4

Article No.: 54, Pages 1 - 40

https://doi.org/10.1145/2501654.2501668

Published: 30 August 2013 Publication History

Abstract

Resource discovery is an important part of distributed computing and resource sharing systems, like grids and utility computing. Because of the increasing importance of decentralized and peer-to-peer environments, characterized by high dynamism and churn, a number of resource discovery mechanisms, mainly based on peer-to-peer techniques, have been presented recently. We present and classify them according to criteria like their topology and the degree of achievement of various common requirements of great importance for the targeted environments, as well as compare their reported performance. These classifications intend to provide an intuitive vision of the strengths and weaknesses of each system.

References

[1]

Abdullah, T. and Bertels, K. 2011. Nature-inspired self-organization in p2p ad hoc grids. In Proceedings of the 7^th International Conference on Autonomic and Autonomous Systems (ICAS'11). 134--139.

[2]

Abdullah, T., Mhamdi, L., Pourebrahimi, B., and Bertels, K. 2009a. Resource discovery with dynamic matchmakers in ad hoc grid. In Proceedings of the 4^th International Conference on Systems (ICONS'09). IEEE Computer Society, Los Alamitos, CA, 138--144.

Digital Library

[3]

Abdullah, T., Alima, L. O., Sokolov, V., Calomme, D., and Bertels, K. 2009b. Hybrid resource discovery mechanism in ad hoc grid using structured overlay. In Proceedings of the 22^nd International Conference on Architecture of Computing Systems. M. Berekovic, C. Mller-Schloer, C. Hochberger, and S. Wong, Eds., Lecture Notes In Computer Science, vol. 5455, Springer, 108--119.

Digital Library

[4]

Aspnes, J. and Shah, G. 2003. Skip graphs. In Proceedings of the 14^th Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, Philadelphia, PA, 384--393.

Digital Library

[5]

Basu, S., Banerjee, S., Sharma, P., and Lee, S. 2005. NodeWiz: Peer-to-peer resource discovery for grids. In Proceedings of the 5^th IEEE International Symposium on Cluster Computing and the Grid (CCGRID'05). Vol. 1, IEEE Computer Society, Los Alamitos, CA, 213--220.

Digital Library

[6]

Beckmann, N., Kriegel, H., Schneider, R., and Seeger, B. 1990. The r&ast;-tree: An efficient and robust access method for points and rectangles. SIGMOD Rec. 19, 2, 322--331.

Digital Library

[7]

Bhagwan, R., Mahadevan, P., Varghese, G., Voelker, G. M. 2004. Cone: A distributed heap-based approach to resource selection. Tech. rep. CS2004-0784, University of California, San Diego.

[8]

Bharambe, A. R., Agrawal, M., and Seshan, S. 2004. Mercury: Supporting scalable multi-attribute range queries. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM'04). ACM Press, New York, NY, 353--366.

Digital Library

[9]

Brunner, R., Caminero, A., Rana, O., Freitag, F., and Navarro, L. 2012. Network-aware summarisation for resource discovery in p2p-content networks. Future Gener. Comput. Syst. 28, 3, 563--572.

Digital Library

[10]

Cai, M., Frank, M., Chen, J., and Szekely, P. 2003. MAAN: A multi-attribute addressable network for grid information services. In Proceedings of the 4^th International Workshop on Grid Computing. IEEE Computer Society, Los Alamitos, CA, 184.

Digital Library

[11]

Chang-Yen, I., Smith, D., and Tzeng, N. 2008. Structured peer-to-peer resource discovery for computational grids. In Proceedings of the 15^th ACM Mardi Gras Conference: From Lightweight MashUps To Lambda Grids: Understanding the Spectrum of Distributed Computing Requirements, Applications, Tools, Infrastructures, Interoperability, and the Incremental Adoption of Key Capabilities (MG'08). Vol. 320, ACM Press, New York, NY, 1--8.

Digital Library

[12]

Cheema, A. S., Muhammad, M., and Gupta, I. 2005. Peer-to-peer discovery of computational resources for grid applications. In Proceedings of the 6^th IEEE/ACM International Workshop on Grid Computing. IEEE Computer Society, Los Alamitos, CA, 179--185.

Digital Library

[13]

Clarke, I., Sandberg, O., Wiley, B., and Hong, T. W. 1999. Freenet: A distributed anonymous information storage and retrieval system. Freenet white paper. http://freenetproject.org/freenet.pdf.

[14]

Costa, P., Napper, J., Pierre, G., and Steen, M. V. 2009. Autonomous resource selection for decentralized utility computing. In Proceedings of the 29^th IEEE International Conference on Distributed Computing Systems (ICDCS'09). IEEE Computer Society, Los Alamitos, CA, 561--570.

Digital Library

[15]

Cotop. 2013. home page. http://codeen.cs.princeton.edu/cotop/.

[16]

Crainiceanu, A., Linga, P., Gehrke, J., and Shanmugasundaram, J. 2004. Querying peer-to-peer networks using p-trees. In Proceedings of the 7^th International Workshop on the Web and Databases (WebDB'04), collocated with ACM SIGMOD/PODS. Vol. 67, ACM Press, New York, NY, 25--30.

Digital Library

[17]

Delmastro, F., Conti, M., and Gregori, E. 2006. P2p common api for structured overlay networks: A cross-layer extension. In Proceedings of the International Symposium on World of Wireless, Mobile and Multimedia Networks. IEEE Computer Society, Los Alamitos, CA, 593--597.

Digital Library

[18]

Lua, E. K., Crowcroft, J., Pias, M., Sharma, R., and Lim, S. 2005. A survey and comparison of peer-to-peer overlay network schemes. IEEE Comm. Surv. Tutorials 7, 2, 72--93.

Digital Library

[19]

Fidler, E., Jacobsen, H., Li, G., and Mankovski, S. 2005. The padres distributed publish/subscribe system. In Feature Interactions in Telecommunications and Software Systems VIII, 12--30.

[20]

Fisher, D. 1987. Knowledge acquisition via incremental conceptual clustering. Mach. Learn. 2, 2, 139--172.

Digital Library

[21]

Forestiero, A., Leonardi, E., Mastroianni, C., and Meo, M. 2010. Self-chord: A bio-inspired p2p framework for self-organizing distributed systems. IEEE/ACM Trans. Netw. 18, 5, 1651--1664.

Digital Library

[22]

Ganesan, P., Yang, B., and Garcia-Molina, H. 2004. One torus to rule them all: Multi-dimensional queries in p2p systems. In Proceedings of the 7^th International Workshop on the Web and Databases (WebDB'04) colocated with ACM SIGMOD/PODS. Vol. 67, ACM Press, New York, 19--24.

Digital Library

[23]

Gao, J. and Steenkiste, P. 2004. An adaptive protocol for efficient support of range queries in dht based systems. In Proceedings of the 12^th IEEE International Conference on Network Protocols (ICNP'04). IEEE Computer Society, Los Alamitos, CA, 239--250.

Digital Library

[24]

Gnutella Protocol Specification Wiki. 2013. http://wiki.limewire.org/index.php&quest;title=GDF.

[25]

Iamnitchi, A. and Foster, I. T. 2001. On fully decentralized resource discovery in grid environments. In Proceedings of the 2^nd International Workshop on Grid Computing. C. A. Lee, Ed., Lecture Notes In Computer Science, vol. 2242, Springer, 51--62.

Digital Library

[26]

Iamnitchi, A., Foster, I., and Nurmi, D. C. 2002. A Peer-to-peer approach to resource location in grid environments. In Proceedings of the 11^th IEEE International Symposium on High Performance Distributed Computing. IEEE Computer Society, Los Alamitos, CA, 419.

Digital Library

[27]

Kim, J.-S., Nam, B., Marsh, M., Keleher, P., Bhattacharjee, B., Richardson, D., Wellnitz, D., and Sussman, A. 2007. Creating a robust desktop grid using peer-to-peer services. In Proceedings of the National Science Foundation Next Generation Software Workshop (NSFNGS'07). IEEE Computer Society, Los Alamitos, CA, 1--7.

[28]

Kim, J.-S., Nam, B., Marsh, M., Keleher, P., Bhattacharjee, B., and Sussman, A. 2008. Integrating categorical resource types into a p2p desktop grid system. In Proceedings of the 9^th IEEE/ACM International Conference on Grid Computing. IEEE Computer Society, Los Alamitos, CA, 284--291.

Digital Library

[29]

Kazaa Media Desktop. 2013. http://www.kazaa.com/.

[30]

Liu, B., Lee, W., and Lee, D. L. 2005. Supporting complex multi-dimensional queries in p2p systems. In Proceedings of the 25^th IEEE international Conference on Distributed Computing Systems (ICDCS'05). IEEE Computer Society, Los Alamitos, CA, 155--164.

Digital Library

[31]

Lo, V., Zhou, D., Zappala, D., Liu, Y., and Zhao, S. 2004. Cluster computing on the fly: P2p scheduling of idle cycles in the internet. In Proceedings of the 3^rd International Conference on Peer-to-Peer Systems (IPTPS'04). Lecture Notes in Computer Science, vol. 3279, Springer, 227--236.

Digital Library

[32]

March, V., Teo, Y. M., and Wang, X. 2007. DGRID: A dht-based resource indexing and discovery scheme for computational grids. In Proceedings of the 5^th Australasian Symposium on Grid Computing and e-Research. Vol. 68, 41--48.

Digital Library

[33]

Mastroianni, C., Talia, D., and Verta, O. 2005. A super-peer model for resource discovery services in large-scale grids. Future Gener. Comput. Syst. 21, 8, 1235--1248.

Digital Library

[34]

Moreno-Vozmediano, R. 2006. Resource discovery in ad-hoc grids. In Proceedings of the International Workshop on Grid Computing Security and Resource Management (ICCS'06). Lecture Notes in Computer Science, vol. 3994, Springer, 1031--1038.

Digital Library

[35]

Oppenheimer, D., Albrecht, J., Patterson, D., and Vahdat, A. 2005. Design and implementation tradeoffs for wide-area resource discovery. In Proceedings of the 14^th IEEE International Symposium on High Performance Distributed Computing (HPDC'05). IEEE Computer Society, Los Alamitos, CA, 113--124.

Digital Library

[36]

Padmanabhan, A., Ghosh, S., Wang, S. 2010. A self-organized grouping (sog) framework for efficient grid resource discovery. J. Grid Comput. 8, 3, 365--389.

[37]

Peterson, L., Bavier, A., Fiuczynski, M. E., and Muir, S. 2006. Experiences building planet lab. In Proceedings of the 7^th Symposium on Operating Systems Design and Implementation. USENIX Association, Berkeley, CA, 351--366.

Digital Library

[38]

Planetlab. 2013. home page. http://www.planet-lab.org/.

[39]

Ramabhadran, S., Ratnasamy, S., Hellerstein, J., Shenker, S. 2004. Prefix hash tree an indexing data structure over distributed hash tables. IRB Tech. rep. http://berkeley.intel-research.net/sylvia/pht.pdf.

[40]

Ranjan, R., Chan, L., Harwood, A., Karunasekera, S., and Buyya, R. 2007. Decentralised resource discovery service for large scale federated grids. In Proceedings of the 3^rd IEEE International Conference on E-Science and Grid Computing (E-SCIENCE'07). IEEE Computer Society, Los Alamitos, CA, 379--387.

Digital Library

[41]

Ranjan, R., Harwood, A., Buyya, R. 2008. Peer-to-peer-based resource discovery in global grids: A tutorial. IEEE Comm. Surv. Tutorials 10, 2, 6--33.

Digital Library

[42]

Ratnasamy, S., Francis, P., Handley, M., Karp, R., and Schenker, S. 2001. A scalable content-addressable network. In Proceedings of the Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM'01). ACM Press, New York, 161--172.

Digital Library

[43]

Rowstron, A. I. and Druschel, P. 2001. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms. R. Guerraoui, Ed., Lecture Notes In Computer Science, vol. 2218, Springer, 329--350.

Digital Library

[44]

Schmidt, C. and Parashar, M. 2004. Enabling flexible queries with guarantees in p2p Systems. IEEE Internet Comput. 8, 3, 19--26.

Digital Library

[45]

Schmidt, C. and Parashar, M. 2008. Squid: Enabling search in dht-based systems. J. Parallel Distrib. Comput. 68, 7, 962--975.

Digital Library

[46]

Sha-1 Standard. 2013. http://csrc.nist.gov/publications/fips/fips180-3/fips180-3 final.pdf.

[47]

Smith, D., Tzeng, N., and Ghantous, M. M. 2008. FaSReD: Fast and scalable resource discovery in support of multiple resource range requirements for computational grids. In Proceedings of the 7^th IEEE International Symposium on Network Computing and Applications (NCA'08). IEEE Computer Society, Los Alamitos, CA, 45--51.

Digital Library

[48]

Spence, D. and Harris, T. 2003. XenoSearch: Distributed resource discovery in the xenoserver open platform. In Proceedings of the 12^th IEEE International Symposium on High Performance Distributed Computing. IEEE Computer Society, Los Alamitos, CA, 216--225.

Digital Library

[49]

Stoica, I., Morris, R., Liben-Nowell, D., Karger, D. R., Kaashoek, M. F., Dabek, F., and Balakrishnan, H. 2003. Chord: A scalable peer-to-peer lookup protocol for internet applications. IEEE/ACM Trans. Netw. 11, 1, 17--32.

Digital Library

[50]

Suryanarayana, G. and Taylor, R. N. 2004. A survey of trust management and resource discovery technologies in peer-to-peer applications. Tech. rep. UCI-ISR-04-6, UCI Institute for Software Research.

[51]

Sword. 2013. home page. http://sword.cs.williams.edu/.

[52]

Tanin, E., Harwood, A., and Samet, H. 2007. Using a distributed quadtree index in peer-to-peer networks. The VLDB J. 16, 2, 165--178.

Digital Library

[53]

Thain, D., Tannenbaum, T., and Livny, M. 2005 Distributed computing in practice: The Condor experience. Concurr. Pract. Exper. 17, 2--4, 323--356.

Digital Library

[54]

The Globus Alliance: Information Services In The Globus Toolkit 2 Release. 2013. http://www.globus.org/mds/mdstechnologybrief draft4.pdf.

[55]

Van Renesse, R. and Bozdog, A. 2004. Willow: DHT, aggregation, and publish/subscribe in one protocol. In Proceedings of the 3^rd International Conference on Peer-to-Peer Systems (IPTPS'04). Lecture Notes in Computer Science, vol. 3279, Springer, 173--183.

Digital Library

[56]

Yan, W., Hu, S., Muthusamy, V., Jacobsen, H., and Zha, L. 2009. Efficient event-based resource discovery. In Proceedings of the 3^rd ACM International Conference on Distributed Event-Based Systems (DEBS'09). ACM Press, New York.

Digital Library

[57]

Zhou, D. and Lo, V. 2004. Cluster computing on the fly: Resource discovery in a cycle sharing peer-to-peer system. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGRID'04). IEEE Computer Society, Los Alamitos, CA, 66--73.

Digital Library

Cited By

Khatibi ESharifi MMirtaheri S(2019)DPAS: A dynamic popularity-aware search mechanism for unstructured P2P systemsPeer-to-Peer Networking and Applications10.1007/s12083-019-00831-2Online publication date: 12-Nov-2019
https://doi.org/10.1007/s12083-019-00831-2
Lucchese F(2018)From P2P to NoSQL: A continuous metric for classifying large-scale storage systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2017.11.017113(227-249)Online publication date: Mar-2018
https://doi.org/10.1016/j.jpdc.2017.11.017
Zarrin JAguiar RBarraca J(2018)Resource discovery for distributed computing systems: A comprehensive surveyJournal of Parallel and Distributed Computing10.1016/j.jpdc.2017.11.010113(127-166)Online publication date: Mar-2018
https://doi.org/10.1016/j.jpdc.2017.11.010
Show More Cited By

Index Terms

Decentralized resource discovery mechanisms for distributed computing in peer-to-peer environments

Recommendations

Autonomous topology optimisation for unstructured Peer-To-Peer networks

The topology of a Peer-To-Peer (P2P) overlay network is usually constructed randomly without considering the characteristics of underlying physical links. Thus, searching and routing between peer nodes is often inefficient. This paper proposes a ...
Peer Interest-based Discovery for Decentralized Peer-to-Peer Systems
3PGCIC '10: Proceedings of the 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing

The success of content distribution oriented peer-to-peer systems heavily depends on the resource discovery mechanism. In case of large-scale distributed systems, this mechanism must be scalable and robust. The paper proposes a structured solution for ...
A survey on resource discovery mechanisms, peer-to-peer and service discovery frameworks

Service and resource discovery has become an integral part of modern networked systems. In this survey we give an overview of the existing solutions for service and resource discovery for a wide variety of network types. We cover techniques used in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 45, Issue 4

August 2013

490 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/2501654

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 August 2013

Accepted: 01 September 2012

Revised: 01 April 2012

Received: 01 January 2010

Published in CSUR Volume 45, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
753
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Khatibi ESharifi MMirtaheri S(2019)DPAS: A dynamic popularity-aware search mechanism for unstructured P2P systemsPeer-to-Peer Networking and Applications10.1007/s12083-019-00831-2Online publication date: 12-Nov-2019
https://doi.org/10.1007/s12083-019-00831-2
Lucchese F(2018)From P2P to NoSQL: A continuous metric for classifying large-scale storage systemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2017.11.017113(227-249)Online publication date: Mar-2018
https://doi.org/10.1016/j.jpdc.2017.11.017
Zarrin JAguiar RBarraca J(2018)Resource discovery for distributed computing systems: A comprehensive surveyJournal of Parallel and Distributed Computing10.1016/j.jpdc.2017.11.010113(127-166)Online publication date: Mar-2018
https://doi.org/10.1016/j.jpdc.2017.11.010
Bandara HJayasumana A(2015)P2P-Based, Multi-Attribute Resource Discovery under Real-World Resources and QueriesACM Transactions on Internet Technology10.1145/272913915:1(1-35)Online publication date: 12-Mar-2015
https://dl.acm.org/doi/10.1145/2729139
Chmaj GSelvaraj H(2015)UAV Cooperative Data Processing Using Distributed Computing PlatformProgress in Systems Engineering10.1007/978-3-319-08422-0_67(455-461)Online publication date: 2015
https://doi.org/10.1007/978-3-319-08422-0_67

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents