Adaptive Workload-Based Partitioning and Replication for RDF Graphs
Pages 250 - 258
Abstract
Distributed processing of RDF data requires partitioning of big and complex data sets. The partitioning affects the performance of the distributed query processing and the amount of data transfer between the network-connected nodes. Static graph partitioning aims to generate partitions with lowest number of edges in between but suffers high communication cost when a query trespasses a partition’s border, because then it requires moving partial results across the network. Workload-aware partitioning is an alternative but faces complex decisions regarding the storage space and the workload orientation. In this paper, we present an adaptive partitioning and replication strategy on three levels. We initialize our system with static partitioning where it collects and analyzes the received workload; then we let it adapt itself with two levels of dynamic replications, besides applying a weighting system to its initial static partitioning to decrease the ratio of border nodes.
References
[1]
Al-Ghezi, A., Wiese, L.: Space-adaptive and workload-aware replication and partitioning for distributed RDF triple stores. In: 29th International Workshop on Database and Expert Systems Applications (DEXA). Springer, Cham (2018)
[2]
Galárraga, L., Hose, K., Schenkel, R.: Partout: a distributed engine for efficient RDF processing. In: Proceedings of the 23rd International Conference on World Wide Web, pp. 267–268. ACM (2014)
[3]
Gurajada, S., Seufert, S., Miliaraki, I., Theobald, M.: TriAD: a distributed shared-nothing RDF engine based on asynchronous message passing. In: Proceedings of the ACM International Conference on Management of Data, pp. 289–300. ACM, New York (2014)
[4]
Hose, K., Schenkel, R.: WARP: workload-aware replication and partitioning for RDF. In: IEEE 29th International Conference on Data Engineering Workshops (ICDEW), pp. 1–6 (2013)
[5]
Huang J, Abadi DJ, and Ren K Scalable SPARQL querying of large RDF graphs Proc. VLDB Endow. 2011 4 11 1123-1134
[6]
Karypis G Padua D METIS and parMETIS Encyclopedia of Parallel Computing 2011 Boston Springer 1117-1124
[7]
Margo D and Seltzer M A scalable distributed graph partitioner Proc. VLDB Endow. 2015 8 12 1478-1489
[8]
Padiya, T., Kanwar, J.J., Bhise, M.: Workload aware hybrid partitioning. In: Proceedings of the 9th Annual ACM India Conference, pp. 51–58. ACM (2016)
[9]
Peng, P., Chen, L., Zou, L., Zhao, D.: Query workload-based RDF graph fragmentation and allocation. In: EDBT, pp. 377–388 (2016)
[10]
Shang, Z., Yu, J.X.: Catch the wind: graph workload balancing on cloud. In: Proceedings of the IEEE International Conference on Data Engineering, pp. 553–564. IEEE Computer Society, Washington, DC (2013)
[11]
Wu, B., Zhou, Y., Yuan, P., Liu, L., Jin, H.: Scalable SPARQL querying using path partitioning. In: 2015 IEEE 31st International Conference on Data Engineering (ICDE), pp. 795–806. IEEE (2015)
[12]
Xu Q, Wang X, Wang J, Yang Y, and Feng Z Chen L, Jensen CS, Shahabi C, Yang X, and Lian X Semantic-aware partitioning on RDF graphs Web and Big Data 2017 Cham Springer 149-157
[13]
Zhang, X., Chen, L., Tong, Y., Wang, M.: EAGRE: Towards scalable i/o efficient SPARQL query evaluation on the cloud. In: Jensen, C.S., Jermaine, C.M., Zhou, X. (eds.) ICDE, pp. 565–576. IEEE Computer Society (2013)
Recommendations
GraBi: Communication-Efficient and Workload-Balanced Partitioning for Bipartite Graphs
ICPP '20: Proceedings of the 49th International Conference on Parallel ProcessingMachine Learning and Data Mining (MLDM) applications, such as recommendation and topic modeling, generally represent their input data in bipartite graphs with two disjoint vertex-subsets connected only by edges between them. Despite the prevalence of ...
OntoSP: Ontology-Based Semantic-Aware Partitioning on RDF Graphs
Web Information Systems Engineering – WISE 2021AbstractWith the development of the Semantic Web, more and more fields construct and publish their domain knowledge in RDF format. Facing the challenge of managing massive and ever-growing RDF graph data in a distributed environment, it is necessary to ...
WawPart: Workload-Aware Partitioning of Knowledge Graphs
Advances and Trends in Artificial Intelligence. Artificial Intelligence PracticesAbstractLarge-scale datasets in the form of knowledge graphs are often used in numerous domains, today. A knowledge graph’s size often exceeds the capacity of a single computer system, especially if the graph must be stored in main memory. To overcome ...
Comments
Information & Contributors
Information
Published In
Sep 2018
520 pages
ISBN:978-3-319-98811-5
DOI:10.1007/978-3-319-98812-2
© Springer Nature Switzerland AG 2018.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 03 September 2018
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Aug 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in