Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality

Google Research scientists Nitay Calderon and Gal Yona propose a behavioral framework called knowledge profiling to distinguish whether factual errors in LLMs stem from missing knowledge (encoding failure) or inaccessible knowledge (recall failure). They introduce WikiProfile, a benchmark of 2,150 Wikipedia-derived facts, each paired with questions probing encoding, recall, and recognition. Evaluating 13 LLMs with and without thinking, they show that frontier models like Gemini-3-Pro and GPT-5 encode 95–98% of facts but fail to directly recall 26–34%, even with thinking failing on 11–12%. Scaling in the Gemma 3 family reduces encoding failures but recall failures persist, indicating recall is the bottleneck. Recall is affected by training context, long-tail facts are encoded but hard to recall, and the reversal curse is reframed as a recall problem, since recognition in multiple-choice tasks is not impaired. Thinking improves recall most for rare and reverse facts, recovering 40–65% of encoded-but-not-directly-known facts, acting as a recall-facilitation mechanism rather than pure reasoning.