Automatic evaluation of complex alignments: : An instance-based approach
Authors: 
Elodie Thiéblin, Ollivier Haemmerlé, and Cássia TrojahnAuthors Info & Claims
Elodie Thiéblin, Ollivier Haemmerlé, and Cássia TrojahnAuthors Info & ClaimsAbstract
Ontology matching is the task of generating a set of correspondences (i.e., an alignment) between the entities of different ontologies. While most efforts on alignment evaluation have been dedicated to the evaluation of simple alignments (i.e., those linking one single entity of a source ontology to one single entity of a target ontology), the emergence of matchers providing complex alignments (i.e., those composed of correspondences involving logical constructors or transformation functions) requires new strategies for addressing the problem of automatically evaluating complex alignments. This paper proposes (i) a benchmark for complex alignment evaluation composed of an automatic evaluation system that relies on queries and instances, and (ii) a dataset about conference organisation. This dataset is composed of populated ontologies and a set of competency questions for alignment as SPARQL queries. State-of-the-art alignments are evaluated and a discussion on the difficulties of the evaluation task is provided.
References
[1]
A. Algergawy, M. Cheatham, D. Faria, A. Ferrara, I. Fundulaki, I. Harrow, S. Hertling, E. Jiménez-Ruiz, N. Karam, A. Khiat, P. Lambrix, H. Li, S. Montanelli, H. Paulheim, C. Pesquita, T. Saveta, D. Schmidt, P. Shvaiko, A. Splendiani, É. Thiéblin, C. Trojahn, J. Vatascinová, O. Zamazal and L. Zhou, in: Results of the Ontology Alignment Evaluation Initiative 2018, in: Proceedings of the 13th International Workshop on Ontology Matching co-located with the 17th International Semantic Web Conference, OM@ISWC 2018, Monterey, CA, USA, October 8, 2018, P. Shvaiko, J. Euzenat, E. Jiménez-Ruiz, M. Cheatham and O. Hassanzadeh, eds, CEUR Workshop Proceedings, Vol. 2288, CEUR-WS.org, 2018, pp. 76–116.
[2]
Y. An, X. Hu and I. Song, Learning to discover complex mappings from web forms to ontologies, in: 21st ACM International Conference on Information and Knowledge Management, CIKM’12, Maui, HI, USA, October 29–November 02, 2012, X. Chen, G. Lebanon, H. Wang and M.J. Zaki, eds, ACM, 2012, pp. 1253–1262.
[3]
J. David, J. Euzenat, P. Genevès and N. Layaïda, Evaluation of query transformations without data: Short paper, in: Companion of the the Web Conference 2018 on the Web Conference 2018, WWW 2018, Lyon, France, April 23–27, 2018, P. Champin, F. Gandon, M. Lalmas and P.G. Ipeirotis, eds, ACM, 2018, pp. 1599–1602.
[4]
J. David, J. Euzenat, F. Scharffe and C. Trojahn, in: The Alignment API 4.0, Semantic Web, Vol. 2, 2011, pp. 3–10.
[5]
R. Dhamankar, Y. Lee, A. Doan, A. Halevy and P. Domingos, IMAP: Discovering complex semantic matches between database schemas, in: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, SIGMOD ’04, Association for Computing Machinery, New York, NY, USA, 2004, pp. 383–394. ISBN 1581138598.
[6]
M. Ehrig and J. Euzenat, Relaxed precision and recall for ontology matching, in: Integrating Ontologies ’05, Proceedings of the K-CAP 2005 Workshop on Integrating Ontologies, Banff, Canada, October 2, 2005, B. Ashpole, M. Ehrig, J. Euzenat and H. Stuckenschmidt, eds, CEUR Workshop Proceedings, Vols 156, CEUR-WS.org, 2005.
[7]
J. Euzenat, A. Polleres and F. Scharffe, Processing Ontology Alignments with SPARQL, in: Second International Conference on Complex, Intelligent and Software Intensive Systems (CISIS-2008), Technical University of Catalonia, Barcelona, Spain, March 4–7, 2008, F. Xhafa and L. Barolli, eds, IEEE Computer Society, 2008, pp. 913–917.
[8]
J. Euzenat, M. Rosoiu and C. Trojahn, Ontology matching benchmarks: Generation, stability, and discriminability, Journal of Web Semantics 21 (2013), 30–48.
[9]
J. Euzenat and P. Shvaiko, Ontology Matching, 2nd edn, Springer, Berlin Heidelberg, 2013. ISBN 3642387209.
[10]
D. Faria, C. Pesquita, B.S. Balasubramani, T. Tervo, D. Carriço, R. Garrilha, F.M. Couto and I.F. Cruz, Results of AML participation in OAEI 2018, in: Proceedings of the 13th International Workshop on Ontology Matching co-located with the 17th International Semantic Web Conference, OM@ISWC 2018, Monterey, CA, USA, October 8, 2018, P. Shvaiko, J. Euzenat, E. Jiménez-Ruiz, M. Cheatham and O. Hassanzadeh, eds, CEUR Workshop Proceedings, Vol. 2288, CEUR-WS.org, 2018, pp. 125–131.
[11]
M. Grüninger and M.S. Fox, Methodology for the design and evaluation of ontologies, in: Workshop on Basic Ontological Issues in Knowledge Sharing, Vol. 15, 1995.
[12]
B. He, K.C. Chang and J. Han, Discovering complex matchings across web query interfaces: A correlation mining approach, in: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Washington, USA, August 22–25, 2004, W. Kim, R. Kohavi, J. Gehrke and W. DuMouchel, eds, ACM, 2004, pp. 148–157.
[13]
L. Hollink, M. van Assem, S. Wang, A. Isaac and G. Schreiber, Two variations on ontology alignment evaluation: Methodological issues, in: The Semantic Web: Research and Applications, 5th European Semantic Web Conference, ESWC 2008, Tenerife, Canary Islands, Proceedings, Spain, June 1–5, 2008, S. Bechhofer, M. Hauswirth, J. Hoffmann and M. Koubarakis, eds, Lecture Notes in Computer Science, Vol. 5021, Springer, 2008, pp. 388–401.
[14]
I. Horrocks, O. Kutz and U. Sattler, The even more irresistible SROIQ, in: Proceedings, Tenth International Conference on Principles of Knowledge Representation and Reasoning, Lake District of the United Kingdom, June 2–5, 2006, P. Doherty, J. Mylopoulos and C.A. Welty, eds, AAAI Press, 2006, pp. 57–67.
[15]
A. Isaac, D. Kramer, L. van der Meij, S. Wang, S. Schlobach and J. Stapel, Vocabulary matching for book indexing suggestion in linked libraries – a prototype implementation and evaluation, in: The Semantic Web – ISWC 2009, 8th International Semantic Web Conference, ISWC 2009, Proceedings, Chantilly, VA, USA, October 25–29, 2009, A. Bernstein, D.R. Karger, T. Heath, L. Feigenbaum, D. Maynard, E. Motta and K. Thirunarayan, eds, Lecture Notes in Computer Science, Vol. 5823, Springer, 2009, pp. 843–859.
[16]
A. Isaac, H. Matthezing, L. van der Meij, S. Schlobach, S. Wang and C. Zinn, Putting ontology alignment in context: Usage scenarios, deployment and evaluation in a library case, in: The Semantic Web: Research and Applications, 5th European Semantic Web Conference, ESWC 2008, Tenerife, Canary Islands, Proceedings, Spain, June 1–5, 2008, S. Bechhofer, M. Hauswirth, J. Hoffmann and M. Koubarakis, eds, Lecture Notes in Computer Science, Vol. 5021, Springer, 2008, pp. 402–417.
[17]
S. Jiang, D. Lowd, S. Kafle and D. Dou, Ontology matching with knowledge rules, in: Transactions on Large-Scale Data- and Knowledge-Centered Systems XXVIII: Special Issue on Database- and Expert-Systems Applications, A. Hameurlain, J. Küng, R. Wagner and Q. Chen, eds, Springer, Berlin, Heidelberg, 2016, pp. 75–95.
[18]
A. Maedche, B. Motik, N. Silva and R. Volz, MAFRA – a MApping FRAmework for distributed ontologies, in: Knowledge Engineering and Knowledge Management: Ontologies and the Semantic Web, A. Gómez-Pérez and V.R. Benjamins, eds, Springer, Berlin, Heidelberg, 2002, pp. 235–250. ISBN 978-3-540-45810-4.
[19]
K. Makris, N. Bikakis, N. Gioldasis and S. Christodoulakis, SPARQL-RW: Transparent query access over mapped RDF data sources, in: 15th International Conference on Extending Database Technology, EDBT ’12, Proceedings, Berlin, Germany, March 27–30, 2012, E.A. Rundensteiner, V. Markl, I. Manolescu, S. Amer-Yahia, F. Naumann and I. Ari, eds, ACM, 2012, pp. 610–613.
[20]
V. Maltese, F. Giunchiglia and A. Autayeu, Save up to 99% of your time in mapping validation, in: On the Move to Meaningful Internet Systems, OTM 2010 – Confederated International Conferences: CoopIS, IS, DOA and ODBASE, Hersonissos, Proceedings, Part II, Crete, Greece, October 25–29, 2010, R. Meersman, T.S. Dillon and P. Herrero, eds, Lecture Notes in Computer Science, Vol. 6427, Springer, 2010, pp. 1044–1060.
[21]
C. Meilicke and H. Stuckenschmidt, Incoherence as a basis for measuring the quality of ontology mappings, in: Proceedings of the 3rd International Workshop on Ontology Matching (OM-2008) Collocated with the 7th International Semantic Web Conference (ISWC-2008), Karlsruhe, Germany, October 26, 2008, P. Shvaiko, J. Euzenat, F. Giunchiglia and H. Stuckenschmidt, eds, CEUR Workshop Proceedings, Vol. 431, 2008, CEUR-WS.org.
[22]
B.P. Nunes, A.A.M. Caraballo, M.A. Casanova, K.K. Breitman and L.A.P.P. Leme, Complex matching of RDF datatype properties, in: Proceedings of the 6th International Workshop on Ontology Matching, Bonn, Germany, October 24, 2011, P. Shvaiko, J. Euzenat, T. Heath, C. Quix, M. Mao and I.F. Cruz, eds, CEUR Workshop Proceedings, Vol. 814, CEUR-WS.org, 2011.
[23]
A.G. Nuzzolese, A.L. Gentile, V. Presutti and A. Gangemi, Conference linked data: The ScholarlyData project, in: Proceedings, Part II, The Semantic Web – ISWC 2016 – 15th International Semantic Web Conference, Kobe, Japan, October 17–21, 2016, P. Groth, E. Simperl, A.J.G. Gray, M. Sabou, M. Krötzsch, F. Lécué, F. Flöck and Y. Gil, eds, Lecture Notes in Computer Science, Vol. 9982, 2016, pp. 150–158.
[24]
R. Parundekar, C.A. Knoblock and J.L. Ambite, Linking and building ontologies of linked data, in: The Semantic Web – ISWC 2010, P.F. Patel-Schneider, Y. Pan, P. Hitzler, P. Mika, L. Zhang, J.Z. Pan, I. Horrocks and B. Glimm, eds, Springer, Berlin, Heidelberg, 2010, pp. 598–614. ISBN 978-3-642-17746-0.
[25]
R. Parundekar, C.A. Knoblock and J.L. Ambite, Discovering concept coverings in ontologies of linked data sources, in: The Semantic Web – ISWC 2012 – 11th International Semantic Web Conference, Proceedings, Part I, Boston, MA, USA, November 11–15, 2012, P. Cudré-Mauroux, J. Heflin, E. Sirin, T. Tudorache, J. Euzenat, M. Hauswirth, J.X. Parreira, J. Hendler, G. Schreiber, A. Bernstein and E. Blomqvist, eds, Lecture Notes in Computer Science, Vol. 7649, Springer, 2012, pp. 427–443.
[26]
H. Qin, D. Dou and P. LePendu, Discovering executable semantic mappings between ontologies, in: On the Move to Meaningful Internet Systems 2007: CoopIS, DOA, ODBASE, GADA, and IS, OTM Confederated International Conferences CoopIS, DOA, ODBASE, GADA, and IS 2007, Proceedings, Part i, Vilamoura, Portugal, November 25–30, 2007, R. Meersman and Z. Tari, eds, Lecture Notes in Computer Science, Vol. 4803, Springer, 2007, pp. 832–849.
[27]
Y. Ren, A. Parvizi, C. Mellish, J.Z. Pan, K. van Deemter and R. Stevens, Towards competency question-driven ontology authoring, in: The Semantic Web: Trends and Challenges – 11th International Conference, ESWC 2014, Anissaras, Proceedings, Crete, Greece, May 25–29, 2014, V. Presutti, C. d’Amato, F. Gandon, M. d’Aquin, S. Staab and A. Tordai, eds, Lecture Notes in Computer Science, Vol. 8465, Springer, 2014, pp. 752–767.
[28]
D. Ritze, C. Meilicke, O. Sváb-Zamazal and H. Stuckenschmidt, A pattern-based ontology matching approach for detecting complex correspondences, in: Proceedings of the 4th International Workshop on Ontology Matching (OM-2009) Collocated with the 8th International Semantic Web Conference (ISWC-2009), Chantilly, USA, October 25, 2009, P. Shvaiko, J. Euzenat, F. Giunchiglia, H. Stuckenschmidt, N.F. Noy and A. Rosenthal, eds, CEUR Workshop Proceedings, Vols 551, CEUR-WS.org, 2009.
[29]
D. Ritze, J. Völker, C. Meilicke and O. Sváb-Zamazal, Linguistic analysis for complex ontology matching, in: Proceedings of the 5th International Workshop on Ontology Matching (OM-2010), Shanghai, China, November 7, 2010, P. Shvaiko, J. Euzenat, F. Giunchiglia, H. Stuckenschmidt, M. Mao and I.F. Cruz, eds, CEUR Workshop Proceedings, Vols 689, CEUR-WS.org, 2010.
[30]
R. Shearer, B. Motik and I. Horrocks, HermiT: A highly-efficient OWL reasoner, in: Proceedings of the Fifth OWLED Workshop on OWL: Experiences and Directions, Collocated with the 7th International Semantic Web Conference (ISWC-2008), Karlsruhe, Germany, October 26–27, 2008, C. Dolbear, A. Ruttenberg and U. Sattler, eds, CEUR Workshop Proceedings, Vols 432, CEUR-WS.org, 2008.
[31]
A. Solimando, E. Jiménez-Ruiz and G. Guerrini, Detecting and correcting conservativity principle violations in ontology-to-ontology mappings, in: Proceedings, Part II, The Semantic Web – ISWC 2014–13th International Semantic Web Conference, Riva del Garda, Italy, October 19–23, 2014, P. Mika, T. Tudorache, A. Bernstein, C. Welty, C.A. Knoblock, D. Vrandecic, P. Groth, N.F. Noy, K. Janowicz and C.A. Goble, eds, Lecture Notes in Computer Science, Vol. 8797, Springer, 2014, pp. 1–16.
[32]
A. Solimando, E. Jiménez-Ruiz and C. Pinkel, Evaluating ontology alignment systems in query answering tasks, in: Proceedings of the ISWC 2014 Posters & Demonstrations Track a Track Within the 13th International Semantic Web Conference, ISWC 2014, Riva del Garda, Italy, October 21, 2014, M. Horridge, M. Rospocher and J. van Ossenbruggen, eds, CEUR Workshop Proceedings, Vol. 1272, CEUR-WS.org, 2014, pp. 301–304.
[33]
G. Stapleton, J. Howse, A. Bonnington and J. Burton, A vision for diagrammatic ontology engineering, in: Proceedings of the International Workshop on Visualizations and User Interfaces for Knowledge Engineering and Linked Data Analytics Co-Located with 19th International Conference on Knowledge Engineering and Knowledge Management, VISUAL@EKAW 2014, Linköping, Sweden, November 24, 2014, V. Ivanova, T. Kauppinen, S. Lohmann, S. Mazumdar, C. Pesquita and K. Xu, eds, CEUR Workshop Proceedings, Vol. 1299, CEUR-WS.org, 2014, pp. 1–13.
[34]
W. Su, J. Wang and F.H. Lochovsky, Holistic schema matching for web query interfaces, in: Advances in Database Technology – EDBT 2006, 10th International Conference on Extending Database Technology, Proceedings, Munich, Germany, March 26–31, 2006, Y.E. Ioannidis, M.H. Scholl, J.W. Schmidt, F. Matthes, M. Hatzopoulos, K. Böhm, A. Kemper, T. Grust and C. Böhm, eds, Lecture Notes in Computer Science, Vol. 3896, Springer, 2006, pp. 77–94.
[35]
M.C. Suárez-Figueroa, A. Gómez-Pérez and M. Fernández-López, The NeOn methodology for ontology engineering, in: Ontology Engineering in a Networked World, M.C. Suárez-Figueroa, A. Gómez-Pérez, E. Motta and A. Gangemi, eds, Springer, 2012, pp. 9–34.
[36]
O. Šváb, V. Svátek, P. Berka, D. Rak and P. Tomášek, Ontofarm: Towards an experimental collection of parallel ontologies, in: Proceedings of the 4th International Semantic Web Conference (ISWC), Poster, 2005.
[37]
É. Thiéblin, Do competency questions for alignment help fostering complex correspondences? in: Proceedings of the EKAW Doctoral Consortium 2018 co-located with the 21st International Conference on Knowledge Engineering and Knowledge Management (EKAW 2018), Nancy, France, November 13, 2018, L. Hollink and F. Osborne, eds, CEUR Workshop Proceedings, Vol. 2306, CEUR-WS.org, 2018.
[38]
É. Thiéblin, F. Amarger, O. Haemmerlé, N. Hernandez and C.T. dos Santos, Rewriting SELECT SPARQL queries from 1: n complex correspondences, in: Proceedings of the 11th International Workshop on Ontology Matching co-located with the 15th International Semantic Web Conference (ISWC 2016), Kobe, Japan, October 18, 2016, P. Shvaiko, J. Euzenat, E. Jiménez-Ruiz, M. Cheatham, O. Hassanzadeh and R. Ichise, eds, CEUR Workshop Proceedings, Vol. 1766, CEUR-WS.org, 2016, pp. 49–60.
[39]
É. Thiéblin, F. Amarger, N. Hernandez, C. Roussey and C.T. dos Santos, Cross-querying LOD datasets using complex alignments: An application to agronomic taxa, in: Metadata and Semantic Research – 11th International Conference, MTSR 2017, Proceedings, Tallinn, Estonia, November 28–December 1, 2017, E. Garoufallou, S. Virkus, R. Siatri and D. Koutsomiha, eds, Communications in Computer and Information Science, Vol. 755, Springer, 2017, pp. 25–37.
[40]
É. Thiéblin, M. Cheatham, C.T. dos Santos, O. Zamazal and L. Zhou, The first version of the OAEI complex alignment benchmark, in: Proceedings of the ISWC 2018 Posters & Demonstrations, Industry and Blue Sky Ideas Tracks co-located with 17th International Semantic Web Conference (ISWC 2018), Monterey, USA, October 8–12, 2018, M. van Erp, M. Atre, V. López, K. Srinivas and C. Fortuna, eds, CEUR Workshop Proceedings, Vol. 2180, CEUR-WS.org, 2018.
[41]
É. Thiéblin, O. Haemmerlé, N. Hernandez and C. Trojahn, Task-oriented complex ontology alignment: Two alignment evaluation sets, in: The Semantic Web – 15th International Conference, ESWC 2018, Heraklion, Proceedings, Crete, Greece, June 3–7, 2018, A. Gangemi, R. Navigli, M. Vidal, P. Hitzler, R. Troncy, L. Hollink, A. Tordai and M. Alam, eds, Lecture Notes in Computer Science, Vol. 10843, Springer, 2018, pp. 655–670.
[42]
É. Thiéblin, O. Haemmerlé, N. Hernandez and C. Trojahn, Survey on complex ontology matching, Semantic Web 11(4) (2020), 689–727.
[43]
É. Thiéblin, O. Haemmerlé and C. Trojahn, Complex matching based on competency questions for alignment: A first sketch, in: Proceedings of the 13th International Workshop on Ontology Matching co-located with the 17th International Semantic Web Conference, OM@ISWC 2018, Monterey, CA, USA, October 8, 2018, P. Shvaiko, J. Euzenat, E. Jiménez-Ruiz, M. Cheatham and O. Hassanzadeh, eds, CEUR Workshop Proceedings, Vol. 2288, CEUR-WS.org, 2018, pp. 66–70.
[44]
É. Thiéblin, O. Haemmerlé and C. Trojahn, CANARD complex matching system: Results of the 2018 OAEI evaluation campaign, in: Proceedings of the 13th International Workshop on Ontology Matching co-located with the 17th International Semantic Web Conference, OM@ISWC 2018, Monterey, CA, USA, October 8, 2018, P. Shvaiko, J. Euzenat, E. Jiménez-Ruiz, M. Cheatham and O. Hassanzadeh, eds, CEUR Workshop Proceedings, Vol. 2288, CEUR-WS.org, 2018, pp. 138–143.
[45]
É. Thiéblin, O. Haemmerlé and C. Trojahn, Generating expressive correspondences: An approach based on user knowledge needs and A-box relation discovery, in: The Semantic Web – ISWC 2020 – 19th International Semantic Web Conference, 2020, Proceedings, Part I, Athens, Greece, November 2–6, J.Z. Pan, V.A.M. Tamma, C. d’Amato, K. Janowicz, B. Fu, A. Polleres, O. Seneviratne and L. Kagal, eds, Lecture Notes in Computer Science, Vol. 12506, Springer, 2020, pp. 565–583.
[46]
P.R.S. Visser, D.M. Jones, B.T.J.M. Capon and M.J.R. Shave, An analysis of ontological mismatches: Heterogeneity versus interoperability, in: AAAI 1997 Spring Symposium on Ontological Engineering, Stanford, USA, 1997, pp. 164–172.
[47]
B. Walshe, R. Brennan and D. O’Sullivan, Bayes-ReCCE: A Bayesian model for detecting restriction class correspondences in linked open data knowledge, Bases 12(2) (2016), 25–52.
[48]
O. Zamazal and V. Svátek, The ten-year OntoFarm and its fertilization within the onto-sphere, Journal of Web Semantics 43 (2017), 46–53.
[49]
L. Zhou, M. Cheatham, A. Krisnadhi and P. Hitzler, A complex alignment benchmark: GeoLink dataset, in: Proceedings, Part II, The Semantic Web – ISWC 2018–17th International Semantic Web Conference, Monterey, CA, USA, October 8–12, 2018, D. Vrandecic, K. Bontcheva, M.C. Suárez-Figueroa, V. Presutti, I. Celino, M. Sabou, L. Kaffee and E. Simperl, eds, Lecture Notes in Computer Science, Vol. 11137, Springer, 2018, pp. 273–288.
Index Terms
- Automatic evaluation of complex alignments: An instance-based approach
Index terms have been assigned to the content through auto-classification.
Recommendations
CAI: Complex Ontology Alignments Using Lexical Indexation
Mining Intelligence and Knowledge ExplorationAbstractOntology matching is the process of automatically determining the semantic equivalences between the concepts of two ontologies. However, simple ontology alignments are a domain largely studied. It link one entity of source ontology to one entity ...
Survey on complex ontology matching
The Sensors, Observations, Actuators and the Web of Linked Data and ThingsSimple ontology alignments, largely studied in the literature, link a single entity of a source ontology to a single entity of a target ontology. A limitation of these alignments is their lack of expressiveness which can be overcome by complex alignments. ...
A Fuzzy OWL Ontologies Embedding for Complex Ontology Alignments
Discovery ScienceAbstractThe semantic heterogeneity concern in the information integration can be handled by applying ontology alignment. The purpose of the ontology alignment procedure is to locate concepts that are semantically identical in two ontologies. But, one of ...
Comments
Information & Contributors
Information
Published In
© 2021 – IOS Press. All rights reserved.
Publisher
IOS Press
Netherlands
Publication History
Published: 01 January 2021
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in