article

OOPS! (OntOlogy Pitfall Scanner!): An On-line Tool for Ontology Evaluation

Authors:

María Poveda-Villalón,

Asunción Gómez-Pérez,

Mari Carmen Suárez-FigueroaAuthors Info & Claims

International Journal on Semantic Web & Information Systems, Volume 10, Issue 2

Pages 7 - 34

https://doi.org/10.4018/ijswis.2014040102

Published: 01 April 2014 Publication History

Abstract

This paper presents two contributions to the field of Ontology Evaluation. First, a live catalogue of pitfalls that extends previous works on modeling errors with new pitfalls resulting from an empirical analysis of over 693 ontologies. Such a catalogue classifies pitfalls according to the Structural, Functional and Usability-Profiling dimensions. For each pitfall, we incorporate the value of its importance level (critical, important and minor) and the number of ontologies where each pitfall has been detected. Second, OOPS! (OntOlogy Pitfall Scanner!), a tool for detecting pitfalls in ontologies and targeted at newcomers and domain experts unfamiliar with description logics and ontology implementation languages. The tool operates independently of any ontology development platform and is available online. The evaluation of the system is provided both through a survey of users' satisfaction and worldwide usage statistics. In addition, the system is also compared with existing ontology evaluation tools in terms of coverage of pitfalls detected.

References

[1]

Aguado de Cea, G., Gómez-Pérez, A., Montiel-Ponsoda, E., & Suáárez-Figueroa, M. C. (2008). Natural language-based approach for helping in the reuse of ontology design patterns. In Knowledge engineering: Practice and patterns (pp. 32-47). Springer Berlin Heidelberg.

[2]

Allemang, D., & Hendler, J. (2011). Semantic web for the working ontologist: Effective modeling in RDFS and OWL. Elsevier.

[3]

Archer, P., Goedertier, S., & Loutas, N. (2012). D7. 1.3——Study on persistent URIs, with identification of best practices and recommendations on the topic for the MSs and the EC. PwC EU Services.

[4]

Barron, F. H., & Barrett, B. E. (1996). Decision quality using ranked attribute weights. Management Science, 42(11), 1515——1523.

[5]

Berners-Lee, T., Hendler, J., & Lassila, O. (2001). The semantic web. Scientific American, 284(5), 28——37. 11341160.

[6]

BlomqvistE.GangemiA.PresuttiV. (2009). Experiments on pattern-based ontology design. In Proceedings of the Fifth International Conference on Knowledge Capture (pp. 41-48). ACM. 10.1145/1597735.1597743

[7]

Burton-Jones, A., Storey, V. C., Sugumaran, V., & Ahluwalia, P. (2005). A semiotic metrics suite for assessing the quality of ontologies. Data & Knowledge Engineering, 55(1), 84——102.

Digital Library

[8]

Djedidi, R., & Aufaure, M. A. (2010). ONTO-EVOAL an ontology evolution approach guided by pattern modeling and quality evaluation. In Foundations of information and knowledge systems (pp. 286-305). Springer Berlin Heidelberg.

[9]

Duque-Ramos, A., Fernández-Breis, J. T., Stevens, R., & Aussenac-Gilles, N. (2011). OQuaRE: A SQuaRE-based approach for evaluating the quality of ontologies. Journal of Research and Practice in Information Technology, 43(2), 159.

[10]

Fellbaum, C. (1998). WordNet: An electronic lexical database. WordNet. Available at http://www. cogsci. princeton. edu/wn

[11]

Fernández-López, M., Gómez-Pérez, A., Sierra, J. P. & Sierra, A. P. (1999). Building a chemical ontology using methontology and the ontology design environment. Intelligent Systems and their Applications, IEEE, 14(1), 37-46.

[12]

Gangemi, A., Catenacci, C., Ciaramita, M., & Lehmann, J. (2006). Modelling ontology evaluation and validation (pp. 140——154). Springer Berlin Heidelberg.

[13]

Gómez-Pérez, A. (2004). Ontology evaluation. In Staab, S., & Studer, R. (Eds.), Handbook on ontologies (pp. 251——273). Springer.

[14]

Guarino, N., & Welty, C. A. (2009). An overview of OntoClean. In Handbook on ontologies (pp. 201——220). Springer Berlin Heidelberg.

[15]

Heath, T., & Bizer, C. (2011). Linked data: Evolving the web into a global data space. Synthesis Lectures on the Semantic Web: Theory and Technology, 1(1), 1-136.

[16]

Hogan, A., Harth, A., Passant, A., Decker, S., & Polleres, A. (2010). Weaving the pedantic web. In Linked Data on the Web Workshop.

[17]

Ji, Q., Haase, P., Qi, G., Hitzler, P., & Stadtmüüller, S. (2009). RaDON—repair and diagnosis in ontology networks. In The semantic web: Research and applications (pp. 863—867). Springer Berlin Heidelberg.

[18]

Keet, C. M., Suárez-Figueroa, M. C., & Poveda-Villalóón, M. (2013). The current landscape of pitfalls in ontologies. In Proceedings of the 5th International Conference on Knowledge Engineering and Ontology Development.

[19]

Miettinen, K. (1999). Non-linear multiobjective optimization. Kluwer Academic Publishers.

[20]

Noy, N. F., & McGuinness, D. L. (2001). Ontology development 101: A guide to creating your first ontology. Technical Report SMI-2001-0880, Standford Medical Informatics.

[21]

Pammer, V. (2010) PhD thesis: Automatic support for ontology evaluation review of entailed statements and assertional effects for OWL ontologies. Engineering Sciences. Graz University of Technology.

[22]

Pinto, H. S., Staab, S., & Tempich, C. (2004). DILIGENT: Towards a fine-grained methodology for distributed, loosely-controlled and evolvInG. In Proceedings of the 16th European Conference on Artificial Intelligence (Ecai 2004) (Vol. 110, p. 393). IOS Press.

[23]

Poveda-Villalón, M., Suárez-Figueroa, M. C., & Góómez-Pérez, A. (2010). A double classification of common pitfalls in ontologies. In Workshop on Ontology Quality at the 17th International Conference on Knowledge Engineering and Knowledge Management.

[24]

Poveda-Villalón, M., Suárez-Figueroa, M. C., & Góómez-Pérez, A. (2012). Validating ontologies with oops! In Knowledge Engineering and Knowledge Management (pp. 267—281). Springer Berlin Heidelberg.

[25]

Poveda-Villalón, M., Vatant, B., Suárez-Figueroa, M. C., & Góómez-Pérez, A. (2013). Detecting good practices and pitfalls when publishing vocabularies on the web. In Proceedings of the Workshop on Ontology Patterns at the 12th International Semantic Web Conference.

[26]

Presutti, V., Gangemi, A., David S., Aguado, G., Suárez-Figueroa, M.C., Montiel-Ponsoda, E. & Poveda, M. (2008) NeOn D2.5.1: A library of ontology design patterns: Reusable solutions for collaborative design of networked ontologies. NeOn project. (FP6-27595).

[27]

Rector, A., Drummond, N., Horridge, M., Rogers, J., Knublauch, H., & Stevens, R. etääal. (2004). OWL pizzas: Practical experience of teaching OWL-DL: Common errors & common patterns. In Engineering Knowledge in the Age of the Semantic Web (pp. 63——81). Springer Berlin Heidelberg.

[28]

Rodríguez, N. D., Cuéllar, M. P., Lilius, J., & Calvo-Flores, M. D. (2014). A survey on ontologies for human behavior recognition. ACM Computing Surveys, 46(4), 43.

Digital Library

[29]

Schober, D., Tudose, I., Svatek, V. & Boeker, M. (2012). OntoCheck: Verifying ontology naming conventions and metadata completeness in Protéégé 4. Journal of Biomedical Semantics, 3(Suppl 2), S4.

[30]

Staab, S., Studer, R., Schnurr, H. P., & Sure, Y. (2001). Knowledge processes and ontologies. IEEE Intelligent Systems, 16(1), 26——34.

Digital Library

[31]

Strasunskas, D., & Tomassen, S. L. (2008). The role of ontology in enhancing semantic searches: The EvOQS framework and its initial validation. International Journal of Knowledge and Learning, 4(4), 398-414.

[32]

Suárez-Figueroa, M. C. (2010) PhD Thesis: NeOn methodology for building ontology networks: Specification, scheduling and reuse. Universidad Politécnica de Madrid.

[33]

Suárez-Figueroa, M. C., Cea, G. A. D., & Góómez-Pérez, A. (2013). Lights and shadows in creating a glossary about ontology engineering. Terminology, 19(2), 202—236.

[34]

Suárez-Figueroa, M. C., Gómez-Pérez, A., Motta, E., & Gangemi, A. (Eds.). (2012). Ontology engineering in a networked world. Springer.

[35]

Suárez-Figueroa, M. C., Kamel, M., & Poveda-Villalóón, M. (2013). Benefits of natural language techniques in ontology evaluation: The OOPS! case. In Proceedings of the 10th International Conference on Terminology and Artificial Intelligence (TIA 2013) (pp. 107-110). ISBN: 978-2-9174-9025-9.

[36]

Winkler, R. L. & Hays, W. L. (1985). Statistics: Probability, inference, and decision.

Cited By

Walker JKoutsiana ENwachukwu MMeroño Peñuela ASimperl E(2024)The Promise and Challenge of Large Language Models for Knowledge Engineering: Insights from a HackathonExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650844(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650844
Spoladore DColombo VCampanella VLunetta CMondellini MMahroo ACerri FSacco M(2024)A Knowledge-based Decision Support System for recommending safe recipes to individuals with dysphagiaComputers in Biology and Medicine10.1016/j.compbiomed.2024.108193171:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108193
Sharma SJain S(2024)CovidO: an ontology for COVID-19 metadataThe Journal of Supercomputing10.1007/s11227-023-05509-480:1(1238-1267)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11227-023-05509-4
Show More Cited By

Recommendations

Validating ontologies with OOPS!
EKAW'12: Proceedings of the 18th international conference on Knowledge Engineering and Knowledge Management

Ontology quality can be affected by the difficulties involved in ontology modelling which may imply the appearance of anomalies in ontologies. This situation leads to the need of validating ontologies, that is, assessing their quality and correctness. ...
Ontology ABox Comparison
The Semantic Web: ESWC 2018 Satellite Events
Abstract
Correctness and completeness of facts are relevant criteria for the evaluation and selection of ontologies. They can be determined through the comparison with a gold standard. But a gold standard is usually not available. Hence, we propose to ...
IRI_Debug: An Ontology Evaluation Tool
Abstract
We use ontology to capture the knowledge about domains. There are considerable number of ontologies which have been developed. To develop a new ontology, we use the existing concepts of the ontologies according to the domain needs. The reusability ...

Comments

Information & Contributors

Information

Published In

cover image International Journal on Semantic Web & Information Systems

International Journal on Semantic Web & Information Systems Volume 10, Issue 2

April 2014

86 pages

ISSN:1552-6283

EISSN:1552-6291

Issue’s Table of Contents

Publisher

IGI Global

United States

Publication History

Published: 01 April 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Walker JKoutsiana ENwachukwu MMeroño Peñuela ASimperl E(2024)The Promise and Challenge of Large Language Models for Knowledge Engineering: Insights from a HackathonExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650844(1-9)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650844
Spoladore DColombo VCampanella VLunetta CMondellini MMahroo ACerri FSacco M(2024)A Knowledge-based Decision Support System for recommending safe recipes to individuals with dysphagiaComputers in Biology and Medicine10.1016/j.compbiomed.2024.108193171:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2024.108193
Sharma SJain S(2024)CovidO: an ontology for COVID-19 metadataThe Journal of Supercomputing10.1007/s11227-023-05509-480:1(1238-1267)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11227-023-05509-4
Van de Wynckel MValadez ISigner B(2024)FidMark: A Fiducial Marker Ontology for Semantically Describing Visual MarkersThe Semantic Web10.1007/978-3-031-60635-9_14(235-250)Online publication date: 26-May-2024
https://dl.acm.org/doi/10.1007/978-3-031-60635-9_14
Dasoulas IYang DDimou A(2024)MLSea: A Semantic Layer for Discoverable Machine LearningThe Semantic Web10.1007/978-3-031-60635-9_11(178-198)Online publication date: 26-May-2024
https://dl.acm.org/doi/10.1007/978-3-031-60635-9_11
Mäkelburg JJohn CAcosta M(2024)Automation of Electronic Invoice Validation Using Knowledge Graph TechnologiesThe Semantic Web10.1007/978-3-031-60626-7_14(253-269)Online publication date: 26-May-2024
https://dl.acm.org/doi/10.1007/978-3-031-60626-7_14
Tsaneva SSabou M(2023)Enhancing Human-in-the-Loop Ontology Curation Results through Task DesignJournal of Data and Information Quality10.1145/362696016:1(1-25)Online publication date: 6-Oct-2023
https://dl.acm.org/doi/10.1145/3626960
Wei TChen Y(2023)A Ding Ontology of Chinese BronzeJournal on Computing and Cultural Heritage 10.1145/360948416:4(1-12)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3609484
Hamed NGaglione AGluhak ARana OPerera C(2023)Query Interface for Smart City Internet of Things Data Marketplaces: A Case StudyACM Transactions on Internet of Things10.1145/36093364:3(1-39)Online publication date: 21-Sep-2023
https://dl.acm.org/doi/10.1145/3609336
Adamou APicca DHou YLoreto Granados-García P(2023)The Facets of Intangible Heritage in Southern Chinese Martial Arts: Applying a Knowledge-driven Cultural Contact Detection ApproachJournal on Computing and Cultural Heritage 10.1145/360670216:3(1-27)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.1145/3606702
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents