Abstract
As the volume of semantic data increases rapidly, semantic reasoning becomes a very challenging task. Existing scalable reasoners focus on fragments of OWL 2 RL (eg. RDFS, OWL Horst), and cannot support Semantic Web Rules Language (SWRL) rules, which are widely used in real-world knowledge-based applications. As reasoning of OWL 2 RL ontology extended with SWRL rules can be implemented by materialization of Datalog programs, we propose an approach on parallel materialization of Datalog programs with Spark for scalable reasoning. Since existing scalable reasoners aimed for deterministic rule sets, they used rule-specific strategies for translation of rule execution to parallel jobs and performance optimization techniques. Thus, they cannot be easily extended to support application-specific semantics. In this paper, we propose a rule-independent automatic translation strategy, and several optimization techniques including a dynamic data partition model, a duplication removing strategy and a dependency-aware rule scheduling strategy. These techniques can generalize to vast application-specific semantic rules. Finally, we evaluate our approach with both synthetic and real knowledge bases. The experimental results show our implementation is scalable and the reasoning speed is comparable with that of CiChild, the state-of-the-art scalable reasoner for RDFS/OWL Horst semantics using rule-specific optimizations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lehmann, J., Isele, R., Jakob, M., et al.: DBpedia - a large-scale, multilingual knowledge base extracted from Wikipedia. J. Semant. Web 6(2), 167–195 (2015)
ter Horst, H.J., et al.: Completeness, decidability and complexity of entailment for RDF Schema and a semantic extension involving the OWL vocabulary. Web Semant. J. 3, 79–115 (2005)
Suchanek, F., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge. In: Proceedings of the 16th International Conference on World Wide Web (WWW) (2007)
Gu, R., Wang, S., Wang, F., et al.: Cichlid: efficient large scale RDFS/OWL reasoning with Spark. In: IPDPS, pp. 700–709 (2015)
Urbani, J., Kotoulas, S., Maassen, J., et al.: WebPIE: a web-scale parallel inference engine using MapReduce. J. Web Semant. 17(44), 59–75 (2012)
Dean, J., Ghemawat, S.: MapReduce: simplied data processing on large clusters. In: OSDI, pp. 137–147 (2004)
Peters, M., Sachweh, S., Zündorf, A.: Large scale rule-based reasoning using a laptop. In: Gandon, F., Sabou, M., Sack, H., d’Amato, C., Cudré-Mauroux, P., Zimmermann, A. (eds.) ESWC 2015. LNCS, vol. 9088, pp. 104–118. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18818-8_7
Xu, J., Zhang, W., Zhang, Z., et al.: Clustering-based acceleration for virtual machine image deduplication in the cloud environment. J. Syst. Softw. 121, 144–156 (2016)
Motik, B., Nenov, Y., Piro, R.E.F., et al.: Incremental update of Datalog materialisation: the backward/forward algorithm. In: AAAI, pp. 1560–1568 (2015)
Liu, C., Qi, G., Wang, H., Yu, Y.: Large scale fuzzy pD * reasoning using MapReduce. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 405–420. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25073-6_26
Carroll, J., Dickinson, I., et al.: Jena: implementing the semantic web recommendations. In: Proceedings of the 13th International Conference on World Wide Web, pp. 74–83 (2004)
Rajaraman, A., Ullman, J.D.: Mining of Massive Datasets. Cambridge University Press, New York (2011)
Urbani, J., et al.: DynamiTE: parallel materialization of dynamic RDF data. In: Alani, H., et al. (eds.) ISWC 2013. LNCS, vol. 8218, pp. 657–672. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41335-3_41
Guo, Y., Pan, Z.X., Heflin, J.: LUBM: a benchmark for OWL knowledge base systems. J. Web Semant. 3(2–3), 158–182 (2005)
Motik, B., Nenov, Y., Piro, R., et al.: Parallel materialisation of Datalog programs in centralised, main-memory RDF systems. In: Proceedings of the Twenty-Eighth Conference on Artificial Intelligence (AAAI), pp. 129–137 (2014)
Abiteboul, S., Hull, R., Vianu, V.: Foundations of Databases. Addison Wesley, Boston (1995)
Gao, J., Zhou, J.S., Zhou, C., et al.: GLog: a high level graph analysis system using MapReduce. In: ICDE, pp. 544–555 (2014)
Ullman, J.D.: Principles of Database and Knowledge-base Systems, vol. I. Computer Science Press, New York (1988)
Afrati, F.N., Ullman, J.D.: Transitive closure and recursive Datalog implemented on clusters. In: ICDT, pp. 132–143 (2012)
Subercaze, J., et al.: Inferray: fast in-memory RDF inference. VLDB 9, 468–479 (2016)
Acknowledgement
This work was partially supported by National Key research and Development Plan (2016YFB1000103), Chinese Academy of Sciences STS Project (KFJ-SW-STS-155)and National Science Technology Support Plan (2015BAF23B03).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Wu, H., Liu, J., Wang, T., Ye, D., Wei, J., Zhong, H. (2016). Parallel Materialization of Datalog Programs with Spark for Scalable Reasoning. In: Cellary, W., Mokbel, M., Wang, J., Wang, H., Zhou, R., Zhang, Y. (eds) Web Information Systems Engineering – WISE 2016. WISE 2016. Lecture Notes in Computer Science(), vol 10041. Springer, Cham. https://doi.org/10.1007/978-3-319-48740-3_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-48740-3_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48739-7
Online ISBN: 978-3-319-48740-3
eBook Packages: Computer ScienceComputer Science (R0)