Why an AI Agent Is Actually Cheaper Than RAG

The article argues that the naïve cost comparison between a RAG assistant and an agent is misleading. A classic RAG assistant makes one LLM call and relies on the user to refine the query if the answer is wrong. In practice, users often give up after 2–3 attempts, escalating to a human operator who spends 10–15 minutes solving the case. This adds up to far more than a single LLM call. An agent based on the ReAct loop autonomously iterates over search tools, hypothesizes, retrieves chunks, and synthesizes a final answer without user intervention. The author's ISTOK agent uses five tools: VectorSearch (hybrid dense+BM25 via Milvus), Search (PostgreSQL full-text), OpenChunk to read full document chunks, GetDocumentChunks for table of contents, and CallOperator as a last resort after 2–3 own attempts. To manage context window limits, the agent first tried LLM summarization but found it expensive and unreliable; it now uses a sliding window (keep last 6 messages, drop oldest tool messages) with summarization as a fallback (max 2 per session). LLM calls are also differentiated: cheap embeddings and reranking, light generation for simple steps, medium for typical reasoning, and heavy only for final synthesis or summarization. In a real pilot with ~1200 documents and 1800 monthly requests, the assistant averaged 1 LLM call, 2.3 user follow-ups, 34% escalation rate, 27 minutes to close, and ~18,000 tokens per closed question. The agent averaged 6.4 LLM calls, 0.3 follow-ups, 11% escalation, 4 minutes to close, and ~15,500 tokens. In direct cost, the assistant's closed question cost ~52 RUB (including operator cost), while the agent's cost ~17 RUB, a threefold reduction. The agent's advantage disappears for simple factual questions or when the knowledge base is empty, as it wastes iterations. The agent logs every call's token usage and tool history, with tracing via Arize Phoenix for detailed iteration breakdown.