Can LLMs Master Math? Investigating Large Language Models on Math Stack Exchange

Abstract.

Large Language Models (LLMs) have demonstrated exceptional capabilities in various natural language tasks, often achieving performances that surpass those of humans. Despite these advancements, the domain of mathematics presents a distinctive challenge, primarily due to its specialized structure and the precision it demands. In this study, we adopted a two-step approach for investigating the proficiency of LLMs in answering mathematical questions. First, we employ the most effective LLMs, as identified by their performance on math question-answer benchmarks, to generate answers to 78 questions from the Math Stack Exchange (MSE). Second, a case analysis is conducted on the LLM that showed the highest performance, focusing on the quality and accuracy of its answers through manual evaluation. We found that GPT-4 performs best (nDCG of 0.48 and P@10 of 0.37) amongst existing LLMs fine-tuned for answering mathematics questions and outperforms the current best approach on ArqMATH3 Task1, considering P@10. Our Case analysis indicates that while the GPT-4 can generate relevant responses in certain instances, it does not consistently answer all questions accurately. This paper explores the current limitations of LLMs in navigating complex mathematical problem-solving. Through case analysis, we shed light on the gaps in LLM capabilities within mathematics, thereby setting the stage for future research and advancements in AI-driven mathematical reasoning. We make our code and findings publicly available for research: https://github.com/gipplab/LLM-Investig-MathStackExchange

1. Introduction

Large Language Models (LLMs) have been very popular because of their ability to solve natural language tasks, sometimes with human-like accuracy (Makridakis et al., 2023; Alaofi et al., 2023a, b). Pre-training and fine-tuning the models have led to a performance-winning approach in tasks such as translation, writing code, passing professional exams, etc (Min et al., 2023). The LLMs are used in academia and scientific research for knowledge extraction, ideas, processing, comparison, and across multiple disciplines (Meyer et al., 2023). LLms have also been successful in Question-Answering tasks (QA) in which they provide human-like answers to questions in natural language (Namazifar et al., 2021). Evaluation of LLMs on QA has been especially useful in detecting how accurate LLMs are in generating answers and finding cases in which it could halllucinate (Liu et al., 2024; Kamalloo et al., 2023).

Given that mathematical content is prevalent in science, technology, engineering, and mathematics (STEM), it is imperative to evaluate the capacity of LLMs to engage with mathematical language. The challenges of LLMs with mathematics are manifold; mathematics, with its rigorous logic and abstract concepts, is communicated through a specialized language that mixes symbols and syntax in complex arrangements (Satpute et al., 2024). Unlike natural language, mathematical expressions rely on unmentioned rules and assumptions, demanding explicit knowledge and a level of precision. Mathematical reasoning still poses a sizable challenge to even the most significant language models (Hong et al., 2024; Urrutia and Araya, 2024). There exists an evaluation of LLMs in answering math questions in which the expected answer is straightforward numerical value (Hendrycks et al., 2021), and an LLM-based solution (Gou et al., 2023) is the leading method on the performance scoreboard. In contrast, answers for proof or conceptual questions, which form a large part of the MSE, are not verifiable without human aid yet. However, the evaluation of models in generating answers for unorganized open questions has not been performed.

In this paper, we investigate using LLMs to answer open-ended questions (questions that cannot be answered with a yes or no response) from mathematics. We utilized questions from Math Stack Exchange (MSE), a platform dedicated to solving a wide array of mathematical questions posed by users, offering an ideal testing ground. We evaluate LLMs to find the most relevant answer. MSE encompasses questions that range from elementary level to advanced mathematics MSE questions require correctly applying mathematical principles and articulating complex reasoning clearly and understandably. By focusing on open-ended questions from MSE, we not only underscore the importance of these questions in fostering a deeper understanding of mathematical concepts but also provide a benchmark against which the progress of LLMs in mathematical reasoning can be measured. In this work, we set a base to explore the path forward for LLMs by evaluating them on generated answers and highlighting the challenges using a case study, thus bridging the gap between natural and mathematical language comprehension for LLMs. We make our code and findings publicly available for research.

2. Related work

Research on Large Language Models (LLMs) has extended into mathematical problem solving, albeit primarily focused on pre-university mathematics. Several datasets featuring high-school-level math questions and answers exist, such as those introduced by Mao et al. (Mao et al., 2024), Urrutia et al. (Urrutia and Araya, 2023), and Deb et al. (Deb et al., 2023). Hendrycks et al. (Hendrycks et al., 2021) introduced the MATH dataset with 12,500 challenging competition mathematics problems. Cobbe et al. (Cobbe et al., 2021) developed the GSM8K dataset containing a linguistically diverse collection of high school math problems. MathQA dataset by Amini et al. (Amini et al., 2019) contains 37k English multiple-choice math word problems covering multiple math domain categories by modeling operation programs corresponding to word problems in the AQuA dataset (Ling et al., 2017). The availability of datasets is hindered as primarily physical records are used for writing question answers instead of machine-processable and computer-aided resources (Beck et al., 2020; Satpute et al., 2023).

Specialized LLMs have been developed for particular mathematical fields. For instance, Trinh et al. (Trinh et al., 2024) released AlphaGeometry, a theorem prover for Euclidean plane geometry, which solved 25 out of 30 latest Olympiad-level problems within the time limit. To mitigate the issue of LLMs generating inaccurate information, Romera-Paredes(Romera-Paredes et al., 2024) developed FunSearch, which combines a pre-trained LLM with a systematic evaluator to outline verified problem-solving approaches. Moreover, LLMs equipped with integrated verification codes have demonstrated remarkable success on the MATH and GSM8K datasets, achieving accuracies near 100% in some cases (Zhou et al., 2023). The ToRA series (Gou et al., 2023), based on LLaMA-2 (Team, 2023) and CodeLLaMA (Rozière et al., 2024), represents the forefront of open-source models on the MATH dataset. ToRA models are based on LLaMA-2 (Team, 2023) and CodeLLaMA (Rozière et al., 2024) and were finetuned on a dataset of reasoning paths produced by GPT-4 (OpenAI, 2023) for the MATH and GSM-8 datasets. These models are fine-tuned on reasoning paths generated by GPT-4 for the MATH and GSM-8 datasets and self-verify their solutions by generating and executing code. However, leveraging executable code for Multiple-Selection Questions (MSE) potentially poses challenges due to the complexity of verifying reasoning steps and has not been evaluated so far. The MAmmoTH (Yue et al., 2023) family of models is trained on a dataset of chain-of-thought and program-of-thought rationales produced by GPT-4, but draws from a more diverse distribution of questions that were asked. The Llemma (Azerbayev et al., 2023) family of models have instead been finetuned on a general corpus of math-related documents and is intended to be a generalist model, leaving open the option of finetuning to specific tasks.

3. Dataset

Manual verification of answers to questions from the Mathematics Stack Exchange (MSE) is impractical due to the interdisciplinary nature of the questions and the expertise required to assess answer correctness. The ArqMATH competition dataset (Mansouri et al., 2022) offers a collection of MSE question-answer pairs. Task 1 from the competition’s third edition focused on retrieving relevant answers from MSE for 78 undergraduate-level mathematics questions. The relevancy of the top-ranked answers, as determined by competing systems, was assessed by student evaluators, averaging 450 evaluated answers per topic. While automated evaluation is feasible by excluding answers without evaluation scores, student assessments have shown some inaccuracies (Scharpf et al., 2020).

4. Methodology

For our evaluation, we select 6 LLMs. ToRA (Gou et al., 2023), LLeMa (Azerbayev et al., 2023), GPT-4 (OpenAI, 2023), and MAmmoTH (Yue et al., 2023) give their performance of existing MathQA datasets. We also consider MABOWDOR (Zhong et al., 2023), the best-performing approach for ArqMATH Task-1. Last, we consider Mistral 7B (Jiang et al., 2023), general purpose LLM that has shown sub-level performance with LLaMa, GPT-4 in prompt-based tasks.

We explore two scenarios for our experiments by employing LLMs to answer MSE questions. We perform a two-step procedure in the first scenario (Answer generation). First, we gave 78 questions and generated answers using selected LLMs except MABOWDOR. MABOWDOR uses BERT-based Desne Passage Retrieval, which can only generate embeddings. Second, we indexed the answer as embedding and all the other answers from ArqMATH and found the answer most similar to the generated answer from ArqMATH. In the second scenario (Question-Answer comparison), we use selected LLMs to generate embeddings of all potential answers from ArqMATH and generate embeddings of 78 questions as well. Ultimately, we find out which answer is most similar to the question.

5. Evaluation

This section presents the evaluation of two scenarios using metrics such as Mean Average Precision (mAP’), Precision@10 (P@10), normalized Discounted Cumulative Gain (nDCG’), and Binary Preference (BPref), with scores derived exclusively from evaluated ArqMATH dataset answers.

5.1. Answer generation

In this phase, answers were generated using the six selected models. These models’ outputs served as queries for retrieval within the ArqMATH answer pool. For retrieval, we used DPR vector embeddings (Zhong et al., 2023) and cosine similarity. Table 1 displays the comparative performance across all models. Additionally, we executed a DPR run for comparison, akin to the method utilized by MABOWDOR. The outcome reveals that models fine-tuned on mathematical tasks underperformed relative to the DPR benchmark. Among the three 7b models (Tora, LLeMa, and MAmmoTH), variations in their performance align with their results on the MATH dataset. Notably, increasing the model size of the top performer did not yield better results. The Mistral (Jiang et al., 2023) model, despite being the lowest scorer on the MATH dataset among the models considered, delivered performance comparable to that of Tora-7b. This suggests that models excelling on the MATH dataset might be overfitted to that particular task. Conversely, GPT-4 generated answers exhibited increased effectiveness over the DPR baseline, outperforming the current best approach on ArqMATH3 Task1, i.e., MABOWDOR (Zhong et al., 2023) considering P@10.

Model	nDCG’	mAP’	p@10	BPref
DPR (Cocomae)	0.464	0.191	0.324	0.192
Tora-7b-Code	0.400	0.159	0.279	0.170
Tora-13b-Code	0.355	0.140	0.266	0.154
LLema-7b	0.194	0.069	0.157	0.092
MAmmoTH-7b-Coder	0.369	0.145	0.253	0.166
Mistral-7b-Instruct	0.396	0.155	0.267	0.166
GPT-4* (right trunc.)	0.486	0.219	0.374	0.225
GPT-4§ (left trunc.)	0.473	0.210	0.367	0.215

Table 1. Performance of Large Language Models in generating accurate answers.(*: Answer truncated to initial 512 tokens, §:answer truncated to last 512 tokens)

5.2. Question-Answer comparison

This evaluation segment focused on matching questions with the most relevant answers using embeddings. Given the models’ original design for prompt-based answering, adjustments were necessary to facilitate embedding generation. To this end, we prefixed the prompt ”This passage text means in one word: ” and used the embedding of the last token’s last hidden state. To guide the LLMs, we introduced three math-related example answers: First, “This passage:’$E[X]=\int_{{-\infty}}^{{\infty}}xf(x)dx$’ means in one word:’Expectation’“. Second, “This passage:’$(x-a)^{2}+(y-b)^{2}=r^{2}$’ means in one word: ’Circle’“. Third, “This passage:’The distance between the center of an ellipse and either of its two foci. ’ means in one word:’Eccentricity’.”“ For reranking, we limited our focus to the top 10 results per query as determined by MABOWDOR (Zhong et al., 2023).

Following the discussions from Zhong et al. (Zhong, 2023), not all systems were assessed for answer reranking due to the underperformance of Tora-7b compared to the average ArqMATH approach. Given that LLeMa and MAmmoTH lagged behind Tora in the MATH and GSM benchmarks, their reranking efficacy was anticipated to be inferior. The analysis revealed Tora-7b’s Precision@10 to be inferior to all runs depicted in Table 1. This indicates that comparison question and answer embeddings might not solve the problem of retrieving relevant answers.

	nDCG’	mAP’	p@10	BPref
MABOWDOR	0.132	0.063	0.330	0.088
Tora-7b-Code	0.095	0.037	0.149	0.064
Ada-002	0.121	0.049	0.248	0.069
GPT-4	0.153	0.079	0.321	0.092

Table 2. Performance Answer retrieval given question as query embedding.

6. Case study

This case study examines the performance of answer generation by GPT-4 for a selected question, conducted by two annotators with expertise in Mathematics and Computer Science. Both annotators consulted at zbMATH Open, a comprehensive multilingual abstracting and reviewing service in pure and applied mathematics. The focus was on comparing the retrieval performance of GPT-4 and Dense Passage Retrieval (DPR), particularly on questions where GPT-4 enhanced retrieval and where DPR outperformed GPT-4. The analysis, depicted in Figure 1, indicates that GPT-4 improved precision in 38 of the 78 evaluated Mathematics Stack Exchange (MSE) questions, suggesting its efficacy in generating relevant answers for open-ended math questions on MSE compared to models fine-tuned on various MATHQA datasets. The study also explores the reasons behind ToRA’s underperformance on MSE questions despite its training on the MATH dataset.

Figure 1. Frequency of differences in P@10 values of DPR and GPT-4 (P@10^GPT-4 - P@10^DPR).

6.1. GPT-4

We delve into two specific cases: one where GPT-4’s answers improved retrieval performance in Precision@10 (P@10) and another where it had the opposite effect. These cases are outliers, with most topics showing P@10 variations between $-0.1$ and $0.2$. We do not mention answers due to space limitations, but all the recorded answers for GPT-4  and all the other LLMs are available in the repository.

6.1.1. A.384: Retrieval Boost

Figure 2. Question which is correctly answered by GPT-4.

The question shown in Figure 2, GPT-4’s answer strongly shows increases P@10 (from 0.0 of DPR to 0.6 of GPT-4). The reason is that the first retrieved results by the DPR do not feature a binomial coefficient at all, but the first results of the GPT-4 retrieved result include at least an expansion of $\binom{n}{2}$. We observe that, without context, DPR cannot infer any meaning from the formula. Hence, GPT-4 shows a good contextual understanding of the ground truth formula.

6.1.2. A.337: Retrival Worsened Figure 3

Figure 3. Question which is incorrectly answered by GPT-4. The reason for worsened retrieval shows generated answers irrelevant to the question.

P@10 decreases from 0.5 for the DPR run to 0.1 for GPT-4 for the following question (only the title is shown due to length): The answer given by GPT-4 misses the point since it does not explain how the specific line the poster asks about follows from the assumptions. we observe a pattern representing how GPT-4-generated answers can guide the retrieval system in the wrong direction. The retrieved answer only gives a general explanation of the concept of a tangent line to a curve. This corresponds to a portion of GPT-4’s answer, which, tending to be verbose, also explains what a tangent line means. This shows that GPT-4 cannot answer the questions about complex interactions between mathematical concepts.

6.2. Tora-7b-Code

The answers obtained by all the smaller models (basically every model except GPT-4) are of quite low quality. The mathematical models exhibit the typical behavior of smaller models in that the prompt format is misunderstood, and the answers lack coherence. Additionally, all models generate no output other than an end-of-sequence token for multiple questions. (6 in the case of Tora-7b-Code). In contrast, Mistral is much better at keeping a conversational tone than Tora. Additionally, the reasoning appears to be more structured. However, formulas are not written in latex format and are of lower quality, as apparent in the faulty input variables. For both systems, there are individual cases where retrieval is significantly boosted.

6.2.1. A.309 - Tora-7b-Code boosts retrieval

Figure 4. Answer generated by ToRA where it boosts precision.

Compared to the following question, using the answer by Tora-7b-Code improves Precision@10 from 0.5 to 0.8. Instead of an actual answer, ToRA simulates a thread that is typically present at the MSE as seen in Figure 4. As an answer, it is quite incoherent and logically false.

7. Conclusion

In this study, we explored the capabilities of Large Language Models (LLMs) in addressing questions from the Mathematics Stack Exchange (MSE) through a two-fold approach. Initially, we assessed the performance of state-of-the-art language models, known for their proficiency on established Math Question Answer (MathQA) datasets, against the diverse and open-ended questions found on MSE. Our findings indicate that GPT-4, with an nDCG score of 0.48 and a Precision@10 (P@10) of 0.37, surpassed its peers and showed good results despite domain-specific training. GPT-4 outperformed the current best approach on ArqMATH3 Task1, considering P@10. At the same time, the rest exhibited relatively inferior results. Subsequently, we conducted a detailed case analysis to evaluate GPT-4’s effectiveness in generating accurate answers, thereby shedding light on its potential and limitations. It was observed that LLMs previously performing well on MathQA datasets frequently produced inaccurate responses. Conversely, GPT-4 demonstrated a potential in formulating appropriate answers for straightforward mathematical inquiries. Nonetheless, its accuracy degraded with more intricate questions demanding specialized knowledge. For the benefit of the research community, we have publicly shared the answers generated by the LLMs and the code utilized in our experiments, enabling further investigation and analysis.