LLM Hallucinations: How AI Models Invent Non-Existent Packages
OpenAI
Meta
DeepSeek
Mistral
A new study reveals that large language models frequently hallucinate package names, posing a supply-chain attack risk. The researchers tested 16 models, generating 576,000 code samples, and found that on average 19.7% of package recommendations were fictitious. They also propose mitigation strategies.
The study 'We Have a Package for You!' analyzed package hallucinations by code-generating LLMs. The researchers tested 16 models, including GPT-3.5, GPT-4, GPT-4 Turbo, CodeLlama, DeepSeek Coder, and others, generating 576,000 code samples for Python and JavaScript. They found that 440,445 recommendations (19.7%) were hallucinations, with commercial models hallucinating at least 5.2% and open models 21.7%. The models were more likely to hallucinate on recent topics, and lower temperature reduced but did not eliminate the risk. The study also found that models often repeat their own hallucinations, and that most hallucinated names are not simple typos of real packages. The research won a Distinguished Paper Award at USENIX Security 2025.
- Abbreviations
- LLM = Large Language Model — большая языковая модель
- PyPI = Python Package Index — индекс пакетов Python
- npm = Node Package Manager — менеджер пакетов Node
Source: Habr — хаб ИИ —
original