OWL 2 DL is a complex logic with reasoning problems that have a high worst case complexity. Moder... more OWL 2 DL is a complex logic with reasoning problems that have a high worst case complexity. Modern reasoners perform mostly very well on naturally occurring ontologies of varying sizes and complexity. This performance is achieved through a suite of complex optimisations (with complex interactions) and elaborate engineering. While the formal basis of the core reasoner procedures are well understood, many optimisations are less so, and most of the engineering details (and their possible effect on reasoner correctness) are unreviewed by anyone but the reasoner developer. Thus, it is unclear how much confidence should be placed in the correctness of implemented reasoners. To date, there is no principled, correctness unit test-like suite for simple language features and, even if there were, it is unclear that passing such a suite would say much about correctness on naturally occurring ontologies. This problem is not merely theoretical: Divergence in behaviour (thus known bugginess of implementations) has been observed in the OWL Reasoner Evaluation (ORE) contests to the point where a simple, majority voting procedure has been put in place to resolve disagreements.
In this paper, we present a new technique for finding and resolving reasoner disagreement. We use justifications to cross check disagreements. Some cases are resolved automatically, others need to be manually verified. We evaluate the technique on a corpus of naturally occurring ontologies and a set of popular reasoners. We successfully identify several correctness bugs across different reasoners, identify causes for most of these, and generate appropriate bug reports and patches to ontologies to work around the bug.
DL reasoners are complex pieces of software that work on even more complex input which... more DL reasoners are complex pieces of software that work on even more complex input which makes manual verification difficult. A single ontology can have hundreds or thousands of classes and thus its classification involve an unsurveyable number of subsumption tests. We propose a new method for debugging classification across multiple reasoners which employs justifications generated from the set of entailments that reasoners disagree upon to determine the cause of the disagreement.
This report is an overview of the Banach-Tarski paradox, from the basic steps required to prove t... more This report is an overview of the Banach-Tarski paradox, from the basic steps required to prove the result up to the minimization of the number of pieces required. The paradox states that any pair of bounded non-empty sets in R3 are equidecomposable. A stress is placed on group and set theoretics with the results regarding paradoxical sets and groups proven generally before being applied to prove the paradox itself. I have drawn mainly upon Wagon's book, but have taken it upon myself to spell out or clarify areas in his proofs where I felt it necessary for understanding.
OWL 2 DL is a complex logic with reasoning problems that have a high worst case complexity. Moder... more OWL 2 DL is a complex logic with reasoning problems that have a high worst case complexity. Modern reasoners perform mostly very well on naturally occurring ontologies of varying sizes and complexity. This performance is achieved through a suite of complex optimisations (with complex interactions) and elaborate engineering. While the formal basis of the core reasoner procedures are well understood, many optimisations are less so, and most of the engineering details (and their possible effect on reasoner correctness) are unreviewed by anyone but the reasoner developer. Thus, it is unclear how much confidence should be placed in the correctness of implemented reasoners. To date, there is no principled, correctness unit test-like suite for simple language features and, even if there were, it is unclear that passing such a suite would say much about correctness on naturally occurring ontologies. This problem is not merely theoretical: Divergence in behaviour (thus known bugginess of implementations) has been observed in the OWL Reasoner Evaluation (ORE) contests to the point where a simple, majority voting procedure has been put in place to resolve disagreements.
In this paper, we present a new technique for finding and resolving reasoner disagreement. We use justifications to cross check disagreements. Some cases are resolved automatically, others need to be manually verified. We evaluate the technique on a corpus of naturally occurring ontologies and a set of popular reasoners. We successfully identify several correctness bugs across different reasoners, identify causes for most of these, and generate appropriate bug reports and patches to ontologies to work around the bug.
DL reasoners are complex pieces of software that work on even more complex input which... more DL reasoners are complex pieces of software that work on even more complex input which makes manual verification difficult. A single ontology can have hundreds or thousands of classes and thus its classification involve an unsurveyable number of subsumption tests. We propose a new method for debugging classification across multiple reasoners which employs justifications generated from the set of entailments that reasoners disagree upon to determine the cause of the disagreement.
This report is an overview of the Banach-Tarski paradox, from the basic steps required to prove t... more This report is an overview of the Banach-Tarski paradox, from the basic steps required to prove the result up to the minimization of the number of pieces required. The paradox states that any pair of bounded non-empty sets in R3 are equidecomposable. A stress is placed on group and set theoretics with the results regarding paradoxical sets and groups proven generally before being applied to prove the paradox itself. I have drawn mainly upon Wagon's book, but have taken it upon myself to spell out or clarify areas in his proofs where I felt it necessary for understanding.
Uploads
In this paper, we present a new technique for finding and resolving reasoner disagreement. We use justifications to cross check disagreements. Some cases are resolved automatically, others need to be manually verified. We evaluate the technique on a corpus of naturally occurring ontologies and a set of popular reasoners. We successfully identify several correctness bugs across different reasoners, identify causes for most of these, and generate appropriate bug reports and patches to ontologies to work around the bug.
In this paper, we present a new technique for finding and resolving reasoner disagreement. We use justifications to cross check disagreements. Some cases are resolved automatically, others need to be manually verified. We evaluate the technique on a corpus of naturally occurring ontologies and a set of popular reasoners. We successfully identify several correctness bugs across different reasoners, identify causes for most of these, and generate appropriate bug reports and patches to ontologies to work around the bug.