short-paper

Challenges of Mathematical Information Retrievalin the NTCIR-11 Math Wikipedia Task

Authors:

Moritz Schubotz,

Abdou Youssef,

Volker Markl,

Howard S. CohlAuthors Info & Claims

SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 951 - 954

https://doi.org/10.1145/2766462.2767787

Published: 09 August 2015 Publication History

Get Access

Abstract

Mathematical Information Retrieval concerns retrieving information related to a particular mathematical concept. The NTCIR-11 Math Task develops an evaluation test collection for document sections retrieval of scientific articles based on human generated topics. Those topics involve a combination of formula patterns and keywords. In addition, the optional Wikipedia Task provides a test collection for retrieval of individual mathematical formula from Wikipedia based on search topics that contain exactly one formula pattern. We developed a framework for automatic query generation and immediate evaluation. This paper discusses our dataset preparation, topic generation and evaluation methods, and summarizes the results of the participants, with a special focus on the Wikipedia Task.

References

[1]

Formats for topics and submissions for the math2 task at ntcir-11. Technical report, NTCIR, 2014.

Google Scholar

[2]

Akiko Aizawa, Michael Kohlhase, and Iadh Ounis. NTCIR-10 Math Pilot Task Overview. In Proceedings of the 10th NTCIR Conference on Evaluation of Information Access Technologies, pages 654--661, Tokyo, Japan, 2013.

Google Scholar

[3]

Akiko Aizawa, Michael Kohlhase, Iadh Ounis, and Moritz Schubotz. NTCIR-11 Math-2 Task Overview. In Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, pages 88--98, 2014.

Google Scholar

[4]

Michael Kohlhase, Helena Mihaljevic-Brandt, Wolfram Sperber, and Olaf Teschke. Mathematical Formula Search. pages 56--57, September 2013.

Google Scholar

[5]

Michael Kohlhase, Corneliu Prodescu, and Christian Liguda. Xlsearch: A search engine for spreadsheets. In Simon Thorne et. al, editor, Proceedings of the EuSpRIG 2013 Conference "Spreadsheet Risk Management". July 4--5, London, United Kingdom, pages 47--58. Five Star Printing Ldt, Claydon, 2013.

Google Scholar

[6]

Matthias S. Reichenbach, Anurag Agarwal, and Richard Zanibbi. Rendering expressions to improve accuracy of relevance assessment for math search. Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval - SIGIR '14, pages 851--854, 2014.

Digital Library

Google Scholar

[7]

Moritz Schubotz and Gabriel Wicke. Mathoid: Robust, scalable, fast and accessible math rendering for wikipedia. In Stephen Watt et al., editor, Intelligent Computer Mathematics, volume 8543 of Lecture Notes in Computer Science, pages 224--235. Springer International Publishing, 2014.

Crossref

Google Scholar

[8]

Heinrich Stamerjohanns, Michael Kohlhase, Deyan Ginev, Catalin David, and Bruce Miller. Transforming large collections of scientific publications to xml. Mathematics in Computer Science, 3(3):299--307, 2010.

Crossref

Google Scholar

[9]

Ellen M. Voorhees. The TREC-8 Question Answering Track Report. TREC, 1999.

Google Scholar

[10]

Keita Del Valle Wangari, Richard Zanibbi, and Anurag Agarwal. Discovering real-world use cases for a multimodal math search interface. Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval - SIGIR '14, pages 947--950, 2014.

Digital Library

Google Scholar

Cited By

View all

Meadows JFreitas A(2023)Introduction to Mathematical Language Processing: Informal Proofs, Word Problems, and Supporting TasksTransactions of the Association for Computational Linguistics10.1162/tacl_a_0059411(1162-1184)Online publication date: 19-Sep-2023
https://doi.org/10.1162/tacl_a_00594
Dadure PPakray PBandyopadhyay S(2022)MathUSE: Mathematical information retrieval system using universal sentence encoder modelJournal of Information Science10.1177/0165551522107733550:1(66-84)Online publication date: 4-Mar-2022
https://doi.org/10.1177/01655515221077335
Dadure PPakray PBandyopadhyay S(2022)Embedding and generalization of formula with context in the retrieval of mathematical informationJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2021.05.01434:9(6624-6634)Online publication date: Oct-2022
https://doi.org/10.1016/j.jksuci.2021.05.014
Show More Cited By

Index Terms

Challenges of Mathematical Information Retrievalin the NTCIR-11 Math Wikipedia Task
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
    2. Metrics
2. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Semantification of Identifiers in Mathematics for Better Math Information Retrieval
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

Mathematical formulae are essential in science, but face challenges of ambiguity, due to the use of a small number of identifiers to represent an immense number of concepts. Corresponding to word sense disambiguation in Natural Language Processing, we ...
Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context
JCDL '18: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries

Mathematical formulae represent complex semantic information in a concise form. Especially in Science, Technology, Engineering, and Mathematics, mathematical formulae are crucial to communicate information, e.g., in scientific papers, and to perform ...
Diversified search evaluation: lessons from the NTCIR-9 INTENT task
Abstract
The evaluation of diversified web search results is a relatively new research topic and is not as well-understood as the time-honoured evaluation methodology of traditional IR based on precision and recall. In diversity evaluation, one topic may ...

Comments

Information & Contributors

Information

Published In

SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

August 2015

1198 pages

ISBN:9781450336215

DOI:10.1145/2766462

General Chair:
Ricardo Baeza-Yates
Yahoo Labs, USA
,
Program Chairs:
Mounia Lalmas
Yahoo Labs, UK
,
Alistair Moffat
University of Melbourne, Australia
,
Berthier Ribeiro-Neto
Google, Brazil, and UFMG, Brazil

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

German Science Foundation
German Ministry for Education and Research

Conference

SIGIR '15

Sponsor:

SIGIR

SIGIR '15: The 38th International ACM SIGIR conference on research and development in Information Retrieval

August 9 - 13, 2015

Santiago, Chile

Acceptance Rates

SIGIR '15 Paper Acceptance Rate 70 of 351 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
243
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Meadows JFreitas A(2023)Introduction to Mathematical Language Processing: Informal Proofs, Word Problems, and Supporting TasksTransactions of the Association for Computational Linguistics10.1162/tacl_a_0059411(1162-1184)Online publication date: 19-Sep-2023
https://doi.org/10.1162/tacl_a_00594
Dadure PPakray PBandyopadhyay S(2022)MathUSE: Mathematical information retrieval system using universal sentence encoder modelJournal of Information Science10.1177/0165551522107733550:1(66-84)Online publication date: 4-Mar-2022
https://doi.org/10.1177/01655515221077335
Dadure PPakray PBandyopadhyay S(2022)Embedding and generalization of formula with context in the retrieval of mathematical informationJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2021.05.01434:9(6624-6634)Online publication date: Oct-2022
https://doi.org/10.1016/j.jksuci.2021.05.014
Dadure PPakray PBandyopadhyay S(2022)A formula embedding approach for semantic similarity and relatedness between formulasConcurrency and Computation: Practice and Experience10.1002/cpe.714634:22Online publication date: 15-Jun-2022
https://doi.org/10.1002/cpe.7146
Dadure PPakray PBandyopadhyay S(2021)Mathematical Information Retrieval Trends and TechniquesDeep Natural Language Processing and AI Applications for Industry 5.010.4018/978-1-7998-7728-8.ch005(74-92)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-7728-8.ch005
Mansouri BZanibbi ROard DAgarwal A(2021)Overview of ARQMath-2 (2021): Second CLEF Lab on Answer Retrieval for Questions on MathExperimental IR Meets Multilinguality, Multimodality, and Interaction10.1007/978-3-030-85251-1_17(215-238)Online publication date: 14-Sep-2021
https://doi.org/10.1007/978-3-030-85251-1_17
Aizawa AKohlhase M(2020)Mathematical Information RetrievalEvaluating Information Retrieval and Access Tasks10.1007/978-981-15-5554-1_12(169-185)Online publication date: 2-Sep-2020
https://doi.org/10.1007/978-981-15-5554-1_12
Zanibbi ROard DAgarwal AMansouri B(2020)Overview of ARQMath 2020: CLEF Lab on Answer Retrieval for Questions on MathExperimental IR Meets Multilinguality, Multimodality, and Interaction10.1007/978-3-030-58219-7_15(169-193)Online publication date: 15-Sep-2020
https://doi.org/10.1007/978-3-030-58219-7_15
Fang WStones RMarbach TWang GLiu X(2019)Towards a Latin-Square Search Engine2019 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00110(727-735)Online publication date: Dec-2019
https://doi.org/10.1109/ISPA-BDCloud-SustainCom-SocialCom48970.2019.00110
Fraser DKane ATompa F(2018)Choosing Math Features for BM25 Ranking with Tangent-LProceedings of the ACM Symposium on Document Engineering 201810.1145/3209280.3209527(1-10)Online publication date: 28-Aug-2018
https://dl.acm.org/doi/10.1145/3209280.3209527
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Semantification of Identifiers in Mathematics for Better Math Information Retrieval

Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context

Diversified search evaluation: lessons from the NTCIR-9 INTENT task

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations