Abstract
While several approaches to query a federation of SPARQL endpoints have been proposed in the literature, very little is known about the effectiveness of these approaches and the behavior of the resulting query engines for cases in which the number of federation members increases. The existing benchmarks that are typically used to evaluate SPARQL federation engines do not consider such a form of scalability. In this paper, we set out to close this knowledge gap by investigating the behavior of 4 state-of-the-art SPARQL federation engines using a novel benchmark designed for scalability experiments. Based on the benchmark, we show that scalability is a challenge for each of these engines, especially with respect to the effectiveness of their source selection & query decomposition approaches. FedShop is freely available online at: https://github.com/GDD-Nantes/FedShop.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
The Virtuoso database used at generation time is reused for execution.
- 5.
- 6.
References
Abdelaziz, I., Mansour, E., Ouzzani, M., Aboulnaga, A., Kalnis, P.: Lusail: a system for querying linked data at scale. Proc. VLDB Endow. 11(4), 485–498 (2017)
Acosta, M., Hartig, O., Sequeda, J.: Federated RDF query processing. In: Sakr, S., Zomaya, A.Y. (eds.) Encyclopedia of Big Data Technologies, pp. 754–761. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-77525-8_228
Acosta, M., Vidal, M.-E., Lampo, T., Castillo, J., Ruckhaus, E.: ANAPSID: an adaptive query processing engine for SPARQL endpoints. In: Aroyo, L., et al. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 18–34. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25073-6_2
Aluç, G., Hartig, O., Özsu, M.T., Daudjee, K.: Diversified stress testing of RDF data management systems. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 197–212. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11964-9_13
Bagan, G., Bonifati, A., Ciucanu, R., Fletcher, G.H.L., Lemay, A., Advokaat, N.: gmark: schema-driven generation of graphs and queries. In: 33rd IEEE International Conference on Data Engineering, ICDE 2017, San Diego, 19–22 April 2017, pp. 63–64. IEEE Computer Society (2017)
Bizer, C., Schultz, A.: The berlin SPARQL benchmark. Int. J. Semantic Web Inf. Syst. 5(2), 1–24 (2009)
Cheng, S., Hartig, O.: Fedqpl: a language for logical query plans over heterogeneous federations of RDF data sources. In: Indrawan-Santiago, M., Pardede, E., Salvadori, I.L., Steinbauer, M., Khalil, I., Kotsis, G. (eds.) Proceedings of the 22nd International Conference on Information Integration and Web-Based Applications & Services, Virtual Event (iiWAS 2.20)/Chiang Mai, 30 November–2 December 2020, pp. 436–445. ACM (2020)
Duan, S., Kementsietsidis, A., Srinivas, K., Udrea, O.: Apples and oranges: a comparison of RDF benchmarks and real RDF datasets. In: Sellis, T.K., Miller, R.J., Kementsietsidis, A., Velegrakis, Y. (eds.) Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD 2011), Athens, Greece, 12–16 June 2011, pp. 145–156. ACM (2011)
Görlitz, O., Staab, S.: SPLENDID: SPARQL endpoint federation exploiting VOID descriptions. In: Hartig, O., Harth, A., Sequeda, J.F. (eds.) Proceedings of the Second International Workshop on Consuming Linked Data (COLD2011), Bonn, 23 October 2011, vol. 782 of CEUR Workshop Proceedings. CEUR-WS.org (2011)
Guo, Y., Pan, Z., Heflin, J.: LUBM: a benchmark for OWL knowledge base systems. J. Web Semant. 3(2–3), 158–182 (2005)
Hasnain, A., Saleem, M., Ngonga Ngomo, A.-C., Rebholz-Schuhmann, D.: Extending largerdfbench for multi-source data at scale for SPARQL endpoint federation. In: Demidova, E., Zaveri, A., Simperl, E. (eds.) Emerging Topics in Semantic Technologies - ISWC 2018 Satellite Events [best papers from 13 of the workshops co-located with the ISWC 2018 conference], vol. 36 of Studies on the Semantic Web, pp. 203–218. IOS Press (2018)
Kostopoulos, C., Mouchakis, G., Troumpoukis, A., Prokopaki-Kostopoulou, N., Charalambidis, A., Konstantopoulos, S.: KOBE: Cloud-native open benchmarking engine for federated query processors. In: Verborgh, R., et al. (eds.) ESWC 2021. LNCS, vol. 12731, pp. 664–679. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77385-4_40
Montoya, G., Skaf-Molli, H., Molli, P., Vidal, M.-E.: Decomposing federated queries in presence of replicated fragments. J. Web Semant. 42, 1–18 (2017)
Montoya, G., Vidal, M.-E., Corcho, O., Ruckhaus, E., Buil-Aranda, C.: Benchmarking federated SPARQL query engines: are existing testbeds enough? In: Cudré-Mauroux, P., et al. (eds.) ISWC 2012. LNCS, vol. 7650, pp. 313–324. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35173-0_21
Oguz, D., Ergenc, B., Yin, S., Dikenelli, O., Hameurlain, A.: Federated query processing on linked data: a qualitative survey and open challenges. Knowl. Eng. Rev. 30(5), 545–563 (2015)
Quilitz, B., Leser, U.: Querying distributed RDF data sources with SPARQL. In: Bechhofer, S., Hauswirth, M., Hoffmann, J., Koubarakis, M. (eds.) ESWC 2008. LNCS, vol. 5021, pp. 524–538. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68234-9_39
Rakhmawati, N.A., Saleem, M., Lalithsena, S., Decker, S.: Qfed: query set for federated SPARQL query benchmark. In: Indrawan-Santiago, M., Steinbauer, M., Nguyen, H.-Q., Min Tjoa, A., Khalil, I., Anderst-Kotsis, G. (eds.) Proceedings of the 16th International Conference on Information Integration and Web-based Applications & Services, Hanoi, 4–6 December 2014, pp. 207–211. ACM (2014)
Saleem, M., Hasnain, A., Ngonga Ngomo, A.-C.: Largerdfbench: a billion triples benchmark for SPARQL endpoint federation. J. Web Semant. 48, 85–125 (2018)
Saleem, M., Khan, Y., Hasnain, A., Ermilov, I., Ngonga Ngomo, A.-C.: A fine-grained evaluation of SPARQL endpoint federation systems. Semant. Web 7(5), 493–518 (2016)
Saleem, M., Potocki, A., Soru, T., Hartig, O., Ngonga Ngomo, A.-C.: Costfed: cost-based query optimization for sparql endpoint federation. In: 14th International Conference on Semantic Systems (SEMANTICS), pp. 163–174. Elsevier (2018)
Schmidt, M., Görlitz, O., Haase, P., Ladwig, G., Schwarte, A., Tran, T.: FedBench: a benchmark suite for federated semantic data query processing. In: Aroyo, L., et al. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 585–600. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25073-6_37
Schmidt, M., Hornung, T., Meier, M., Pinkel, C., Lausen, G.: SP2Bench: a SPARQL performance benchmark. In: de Virgilio, R., Giunchiglia, F., Tanca, L. (eds.) Semantic Web Information Management: A Model-Based Perspective, pp. 371–393. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-04329-1_16
Schwarte, A., Haase, P., Hose, K., Schenkel, R., Schmidt, M.: FedX: a federation layer for distributed query processing on linked open data. In: Antoniou, G., et al. (eds.) The Semanic Web: Research and Applications, pp. 481–486. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21064-8_39
Wang, X., Tiropanis, T., Davis, H.C.: LHD: optimising linked data query processing using parallelisation. In: Bizer, C., Heath, T., Berners-Lee, T., Hausenblas, M., Auer, S. (eds.) Proceedings of the WWW2013 Workshop on Linked Data on the Web, Rio de Janeiro, 14 May, 2013, vol. 996 of CEUR Workshop Proceedings. CEUR-WS.org (2013)
Acknowledgments
This work is supported by the French ANR project DeKaloG (Decentralized Knowledge Graphs), ANR-19-CE23-0014, CE23 - Intelligence artificielle, the French CominLabs project MikroLog (The Microdata Knowledge Graph), and by Vetenskapsrådet (the Swedish Research Council, project reg. no. 2019-05655)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dang, MH., Aimonier-Davat, J., Molli, P., Hartig, O., Skaf-Molli, H., Le Crom, Y. (2023). FedShop: A Benchmark for Testing the Scalability of SPARQL Federation Engines. In: Payne, T.R., et al. The Semantic Web – ISWC 2023. ISWC 2023. Lecture Notes in Computer Science, vol 14266. Springer, Cham. https://doi.org/10.1007/978-3-031-47243-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-47243-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47242-8
Online ISBN: 978-3-031-47243-5
eBook Packages: Computer ScienceComputer Science (R0)
