"Enhancing Mathematical Reasoning in LLMs with Background Operators"

Converting math problems into Prolog predicates makes LLM reasoning transparent and verifiable.

This paper enhances mathematical reasoning in LLMs by introducing background operators and Prolog-based solutions. It creates a MATH-Prolog corpus from counting and probability problems, using cross-validated self-training to generate diverse solutions with high accuracy.

-----

https://arxiv.org/abs/2412.04110

🤔 Original Problem:

LLMs struggle with mathematical reasoning due to non-computable verbal steps and procedural programming limitations. Current approaches lack standardized, verifiable solutions for complex math problems.

-----

🔧 Solution in this Paper:

→ The paper introduces background mathematical operators as fundamental building blocks for solving math problems.

→ It develops Prolog solutions that combine problem-specific predicates with intermediate predicates derived from background operators.

→ The MATH-Prolog corpus is created from counting and probability categories of the MATH dataset.

→ A 5-fold cross-validated self-training approach incrementally generates new Prolog solutions.

→ The findall(.) predicate represents constraints to narrow down the search space.

-----

💡 Key Insights:

→ Background operators in prompts enhance solution coverage and learning trajectory

→ Prolog's declarative approach provides better logical reasoning than procedural programming

→ Cross-validated self-training effectively discovers diverse solution strategies

-----

📊 Results:

→ Achieved 84.6% accuracy on cross-validated set

→ Reached 84.8% accuracy on test set using Meta-Llama-3.1-8B-Instruct model

→ 26% of training solutions utilize findall(.) predicate