Machines That Can't Count but Guess Numbers

The article explores why large language models (LLMs) can predict numeric quantities despite not truly understanding numbers. Unlike traditional regression tools like XGBoost, which use a decision-tree approach, LLMs process numbers as strings of characters, splitting them into tokens arbitrarily. In a 2024 experiment by researchers from the University of Arizona and Technical University of Cluj-Napoca, LLMs like GPT-4 and Claude 3 were given a table of examples and asked to predict a missing value. Without any fine-tuning, they performed comparably to specialized programs, and on the Friedman #2 function, Claude 3 even surpassed five such tools. Interestingly, increasing the number of examples improved accuracy, which cannot be attributed to luck. This in-context learning is remarkable because it doesn't require additional training. However, LLMs are unreliable: their predictions vary with the order of rows and the precision of numbers, and they lack a ceiling, sometimes giving wildly incorrect answers. Despite being order-of-magnitude more expensive per prediction than classical tools, LLMs shine when data is scarce, contains text, or when column names carry meaning. New specialized models like TabPFN, published in Nature in early 2025, outperform LLMs on pure numerical tables with small data, taking less than three seconds to beat a combination of classical methods that ran for four hours. The recommendation is to use LLMs only when there is little data or meaningful text, and to consider them as a complement rather than a replacement for classical tools.