research-article

OpenRuleBench: an analysis of the performance of rule engines

Authors:

Michael KiferAuthors Info & Claims

WWW '09: Proceedings of the 18th international conference on World wide web

Pages 601 - 610

https://doi.org/10.1145/1526709.1526790

Published: 20 April 2009 Publication History

Abstract

The Semantic Web initiative has led to an upsurge of the interest in rules as a general and powerful way of processing, combining, and analyzing semantic information. Since several of the technologies underlying rule-based systems are already quite mature, it is important to understand how such systems might perform on the Web scale. OpenRuleBench is a suite of benchmarks for analyzing the performance and scalability of different rule engines. Currently the study spans five different technologies and eleven systems, but OpenRuleBench is an open community resource, and contributions from the community are welcome. In this paper, we describe the tested systems and technologies, the methodology used in testing, and analyze the results.

References

[1]

H. Aıt-Kaci. Warren's abstract machine: a tutorial reconstruction. MIT Press, Cambridge, MA, 1991. http://www.freetechbooks.com/warren-s-abstract-machine-a-tutorial-reconstruction-t397.html.

Digital Library

[2]

K.R. Apt, H. Blair, and A. Walker. Towards a theory of declarative knowledge. In J. Minker, editor, Foundations of Deductive Databases and Logic Programming, pages 89--148. Morgan Kaufmann, Los Altos, CA, 1988.

Digital Library

[3]

I. Balbin, G. S. Port, K. Ramamohanarao, and K. Meenakshi. Efficient bottom-up computation of queries on stratified databases. J. Log. Program., 11(3--4):295--344, 1991.

Digital Library

[4]

C. Beeri and R. Ramakrishnan. On the power of magic. Journal of Logic Programming, 10:255--300, April 1991.

Digital Library

[5]

B. Bishop and F. Fischer. Iris -- integrated rule inference system. In International Workshop on Advancing Reasoning on the Web: Scalability and Commonsense (ARea 2008), June 2008.

[6]

F. Bry, N. Eisinger, T. Eiter, T. Furche, G. Gottlob, C. Ley, B. Linse, R. Pichler, and F. Wei. Foundations of Rule-Based Query Answering. In Proceedings of Summer School Reasoning Web 2007, Dresden, Germany (3rd--7th September 2007), volume 4634 of LNCS, pages 1--153. REWERSE, 2007.

Digital Library

[7]

F. Calimeri, S. Cozza, G. Ianni, and N. Leone. Computable functions in ASP: Theory and implementation. In Int'l Conference on Logic Programming, pages 407--424, December 2008.

Digital Library

[8]

The China benchmark suite, 2001. http://www.cs.unipr.it/China/Benchmarks/.

[9]

K.L. Clark. Negation as failure. In H. Gallaire and J. Minker, editors, Logic and Data Bases, pages 292--322. Plenum Press, 1978.

[10]

C.L. Forgy. Rete: a fast algorithm for the many pattern/many object pattern match problem. Atrificial Intelligence, 19(1):17--37, September 1982.

Digital Library

[11]

H. Garcia--Molina, J.D. Ullman, and J. Widom. Database Systems: The Complete Book. Prentice-Hall, 2008.

Digital Library

[12]

M. Gelfond and N. Leone. Logic programming and knowledge representation -- the A-Prolog perspective. Artificial Intelligence, 138(1--2):3--38, 2002.

Digital Library

[13]

M. Gelfond and V. Lifschitz. The stable model semantics for logic programming. In Logic Programming: Proceedings of the Fifth Conference and Symposium, pages 1070--1080, 1988.

[14]

Y. Guo, Z. Pan, and J. Heflin. LUBM: A benchmark for OWL knowledge base systems. Journal of Web Semantics, 2005.

Digital Library

[15]

M. Kifer, A. Bernstein, and P.M. Lewis. Database Systems: An Application Oriented Approach, Complete Version (2nd Edition). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2005.

Digital Library

[16]

M. Kifer, G. Lausen, and J. Wu. Logical foundations of object-oriented and frame-based languages. Journal of ACM, 42:741--843, July 1995.

Digital Library

[17]

M. Kifer and E.L. Lozinskii. Implementing logic programs as a database system. In IEEE 3-d Int'l Conference on Data Engineering, pages 375--385, February 1987.

Digital Library

[18]

O. Lassila and R.R. Swick (editors). Resource description framework (RDF) model and syntax specification. Technical report, W3C, February 1999. http://www.w3.org/TR/1999/REC-rdf-syntax-19990222/.

[19]

L. Ma, Y. Yang, Z. Qiu, G. Xie, Y. Pan, and S. Liu. Towards a complete OWL ontology benchmark. In European Semantic Web Conference, pages 125--139, 2006.

Digital Library

[20]

Openrulebench web site, 2008. http://rulebench.projects.semwebcentral.org.

[21]

Prolog benchmarking, 1985. http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/prolog/code/bench/0.html.

[22]

T.C. Przymusinski. On the declarative semantics of deductive databases and logic programs. In J. Minker, editor, Foundations of Deductive Databases and Logic Programming, pages 193--216. Morgan Kaufmann, Los Altos, CA, 1988.

Digital Library

[23]

Y Sure, S Staab, and J Angele. Ontoedit: Guiding ontology development by methodology and inferencing. In 1st International Conf. on Ontologies, Databases, and Applications of Semantics, 2002.

Digital Library

[24]

T. Swift and D.S. Warren. An abstract machine for SLG resolution: Definite programs. In Int'l Logic Programming Symposium, Cambridge, MA, November 1994. MIT Press.

Digital Library

[25]

H.J. ter Horst. Combining RDF and part of OWL with rules: Semantics, decidability, complexity. In International Semantic Web Conference (ISWC), pages 668--684, November 2005.

Digital Library

[26]

G. Terracina, N. Leone, V. Lio, and C. Panetta. Experimenting with recursive queries in database and logic programming systems. Journal of the Theory and Practice of Logic Programming, 8(2):129--165, 2008.

Digital Library

[27]

J.F. Ullman. Principles of Database and Knowledge-Base Systems, Volume 1. Computer Science Press, Rockville, MD, 1988.

Digital Library

[28]

A. Van Gelder. The alternating fixpoint of logic programs with negation. Journal of Computer and System Sciences, 47(1):185--221, 1993.

Digital Library

[29]

A. Van Gelder, K.A. Ross, and J.S. Schlipf. The well-founded semantics for general logic programs. Journal of ACM, 38(3):620--650, 1991.

Digital Library

[30]

D.S. Warren. Memoing for logic programming. Communications of ACM, 35(3):93--111, March 1992.

Digital Library

[31]

D.S. Warren. Programming in tabled prolog. Manuscript. http://www.cs.sunysb.edu/ warren/xsbbook/, 1999.

Cited By

Krygin AKarpenko PSychev O(2024)Performing First-Order-Logic Queries Over RDF Data: Interpreter Versus Compiler to Apache Jena RulesNovel and Intelligent Digital Systems: Proceedings of the 4th International Conference (NiDS 2024)10.1007/978-3-031-73344-4_46(537-548)Online publication date: 16-Oct-2024
https://doi.org/10.1007/978-3-031-73344-4_46
Ulliana FBisquert PCharoensit AColin RTornil FYeche Q(2024)Collaborative Benchmarking Rule-Reasoners with B-RunnerRules and Reasoning10.1007/978-3-031-72407-7_3(23-31)Online publication date: 11-Sep-2024
https://doi.org/10.1007/978-3-031-72407-7_3
Liu YStoller STong YTekle K(2023)Benchmarking for Integrating Logic Rules with Everything ElseElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.385.3385(12-26)Online publication date: 12-Sep-2023
https://doi.org/10.4204/EPTCS.385.3
Show More Cited By

Index Terms

OpenRuleBench: an analysis of the performance of rule engines
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Triggers and rules
  2. Information retrieval
    1. Evaluation of retrieval results

Recommendations

Backward Chaining Ontology Reasoning Systems with Custom Rules
WWW '16 Companion: Proceedings of the 25th International Conference Companion on World Wide Web

In the semantic web, content is tagged with "meaning" or "semantics" to facilitate machine processing and web searching. In general, question answering systems that are built on top of reasoning and inference face a number of difficult issues. In this ...
A performance benchmark over semantic rule checking approaches in construction industry

As more and more architectural design and construction data is represented using the Resource Description Framework (RDF) data model, it makes sense to take advantage of the logical basis of RDF and implement a semantic rule checking process as it is ...
A Testing Framework for OWL-DL Reasoning
SKG '08: Proceedings of the 2008 Fourth International Conference on Semantics, Knowledge and Grid

OWL and RDF/RDFS are ontological languages developed by the World Wide Web Consortium (W3C), which have become a de facto standard for the ontological descriptions in various domains. The evolution of these standards was influenced by the numerous ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '09: Proceedings of the 18th international conference on World wide web

April 2009

1280 pages

ISBN:9781605584874

DOI:10.1145/1526709

General Chairs:
Juan Quemada
DIT-UPM
,
Gonzalo León
DIT-UPM
,
Program Chairs:
Yoelle Maarek
Google Inc., Israel
,
Wolfgang Nejdl
L3S and Hannover University

Copyright © 2009 IW3C2 org.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WWW '09

Sponsor:

WWW '09: The 18th International World Wide Web Conference

April 20 - 24, 2009

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

78
Total Citations
View Citations
600
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Krygin AKarpenko PSychev O(2024)Performing First-Order-Logic Queries Over RDF Data: Interpreter Versus Compiler to Apache Jena RulesNovel and Intelligent Digital Systems: Proceedings of the 4th International Conference (NiDS 2024)10.1007/978-3-031-73344-4_46(537-548)Online publication date: 16-Oct-2024
https://doi.org/10.1007/978-3-031-73344-4_46
Ulliana FBisquert PCharoensit AColin RTornil FYeche Q(2024)Collaborative Benchmarking Rule-Reasoners with B-RunnerRules and Reasoning10.1007/978-3-031-72407-7_3(23-31)Online publication date: 11-Sep-2024
https://doi.org/10.1007/978-3-031-72407-7_3
Liu YStoller STong YTekle K(2023)Benchmarking for Integrating Logic Rules with Everything ElseElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.385.3385(12-26)Online publication date: 12-Sep-2023
https://doi.org/10.4204/EPTCS.385.3
Beden SLakshmanan KGiannetti CBeckmann A(2023)Steelmaking Predictive Analytics Based on Random Forest and Semantic ReasoningApplied Sciences10.3390/app13231277813:23(12778)Online publication date: 28-Nov-2023
https://doi.org/10.3390/app132312778
Angele KAngele JSimsek UFensel D(2023)SemReasoner - A High-Performance Knowledge Graph Store and Rule-Based ReasonerThe Semantic Web10.1007/978-3-031-33455-9_34(574-590)Online publication date: 28-May-2023
https://dl.acm.org/doi/10.1007/978-3-031-33455-9_34
Sychev OAnikin ADenisov M(2021)Inference Engines Performance in Reasoning Tasks for Intelligent Tutoring SystemsComputational Science and Its Applications – ICCSA 202110.1007/978-3-030-86960-1_33(471-482)Online publication date: 11-Sep-2021
https://doi.org/10.1007/978-3-030-86960-1_33
Zhao DSubotić PScholz B(2020)Debugging Large-scale DatalogACM Transactions on Programming Languages and Systems10.1145/337944642:2(1-35)Online publication date: 17-Apr-2020
https://dl.acm.org/doi/10.1145/3379446
HANNA BTRIEU LSON TDINH N(2020)An Application of ASP in Nuclear Engineering: Explaining the Three Mile Island Nuclear Accident ScenarioTheory and Practice of Logic Programming10.1017/S147106842000024120:6(926-941)Online publication date: 22-Sep-2020
https://doi.org/10.1017/S1471068420000241
Iglesias NJuarez JCampos M(2020)Comprehensive analysis of rule formalisms to represent clinical guidelinesArtificial Intelligence in Medicine10.1016/j.artmed.2019.101741103:COnline publication date: 1-Mar-2020
https://dl.acm.org/doi/10.1016/j.artmed.2019.101741
Qin YTao XHuang YLü J(2019)An index structure supporting rule activation in pervasive applicationsWorld Wide Web10.1007/s11280-017-0517-222:1(1-37)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.1007/s11280-017-0517-2
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents