A team of Apple researchers has questioned the formal reasoning capabilities of large language models (LLMs), particularly in mathematics.
They found that LLMs exhibit noticeable variance when responding to different instantiations of the same question.
Literature suggests that the reasoning process in LLMs is probabilistic pattern-matching rather than formal reasoning.
Although LLMs can match more abstract reasoning patterns, they fall short of true logical reasoning. Small changes in input tokens can drastically alter model outputs, indicating a strong token bias and suggesting that these models are highly sensitive and fragile.
“Additionally, in tasks requiring the correct selection of multiple tokens, the probability of arriving at an accurate answer decreases exponentially with the number of tokens or steps involved, underscoring their inherent unreliability in complex reasoning scenarios,” said Apple researchers in their paper titled “GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models.”
The ‘GSM8K’ benchmark is widely used to assess the mathematical reasoning of models on grade-school level questions.
While the performance of LLMs …