Large Language Models for Mathematical Problem Solving: Applications, Challenges and Future Directions

Abstract

Large language models (LLMs) have made remarkable strides in natural language tasks, but mathematical problem solving remains a significant challenge. This paper surveys recent methods developed to enhance the mathematical reasoning abilities of LLMs. Techniques such as chain-of-thought prompting, self-consistency decoding, retrieval-based augmentation, and tool use have markedly improved performance on math benchmarks. Despite progress, the LLMs still struggle with reasoning faithfulness, generalization to new problems, and arithmetic accuracy. This paper identifies three key challenges and discuss corresponding future directions, including structured reasoning formats, automatic verification, and targeted training. Bridging the gap between current LLM performance and human-level mathematical proficiency will not only advance AI’s capabilities in math but also improve the transparency and reliability of reasoning in AI systems more broadly. Ultimately, strengthening LLMs in mathematics represents a step toward Artificial Intelligence (AI) systems that can provide trustworthy, interpretable, and high-value support in education, scientific research, and other domains requiring precise reasoning.