Prompt, RAG, or fine-tuning: what actually makes your LLM learn

Sergey Proshchaev, a Tech Lead in FinTech & E-commerce, conducted an experiment to compare three ways to teach a local LLM domain-specific knowledge: prompt with context, RAG (retrieval-augmented generation), and fine-tuning with QLoRA. He used the Qwen3-8B model and a single RTX 4090 with 24 GB VRAM. The domain was internal payment documentation: about 120 pages of regulations, including acquirer response codes, retry rules, payment routing, limits, and exceptions. He created 30 questions of three types: factual, synthetic (requiring synthesis across sections), and format (following an internal incident report template). For prompting, he put the entire regulation into the context (128K tokens) and got 20 correct answers overall, but suffered from 'Lost in the Middle' and high latency. For RAG, he used chunking (500 tokens with overlap), BAAI/bge-m3 embeddings, and Qdrant as the vector database, retrieving top-5 chunks. RAG achieved 9/10 on factual questions and provided source references, but failed on synthetic questions and could not learn internal terminology. For QLoRA, he generated about 800 question-answer pairs with a stronger model, cleaned them manually, and trained a LoRA adapter for about 40 minutes. The fine-tuned model excelled at format questions (10/10) and improved on synthetic questions (8/10), but it failed on updated facts, answering with old limits from the dataset. The author concludes that these approaches are not competitors but layers: start with prompting, then add RAG for factual accuracy and source references, and use fine-tuning when you need the model to adopt a specific style or terminology. The study also mentions that Unsloth Studio and H2O LLM Studio offer GUI-based fine-tuning, and the LoRA Land research by Predibase showed that fine-tuned models can surpass GPT-4 on narrow tasks.