An employee asks the company assistant how much notice a supplier is owed before a contract ends. The assistant answers at once, cites a document, and is wrong. The document it cited is a 2022 PDF. The current policy is a wiki page from 2024, and the contract itself says something else again.
Nothing in that failure is a reasoning problem. Everything the model did was sensible given what it was handed. That is the case we wanted to test against the fashionable fix, which is to wrap the retrieval step in an agent that checks its own work and searches again. We did it on paper, stage by stage, using the published literature and the way these systems are built. We ran no benchmark of our own, so what follows is analysis and not a measurement.
What we looked at first
Classic retrieval-augmented generation (RAG) embeds the question, fetches the nearest chunks from a vector store, puts them in the prompt and generates. It retrieves once and never reconsiders. Agentic RAG wraps an agent around that: it decides whether to search, picks a source, checks whether the result answers the question, and searches again if not. A survey from Rutgers and industry researchers lays the pattern out, and McKinsey's explainer is a good plain account of why grounding in your own documents beats an ungrounded model.
The loop earns its place on questions that span several documents. If you ask what was agreed with a supplier and whether it matches policy, one pass of retrieval will rarely carry both halves. That much held up.
We also checked whether anyone is asking for it. Google Ads data pulled through DataForSEO on 11 October 2026 (US, English) reports no measurable monthly search volume for "agentic rag", one of eleven terms with no volume among the fifteen AI-agent terms we tried. Buyers rarely use our jargon, so that is a limit on what we can see and not proof of absence. The plain version of the question, why does the assistant answer confidently from the wrong document, is common enough.
Where the failures turned out to live
Walk the opening failure through the pipeline and ask, at each stage, whether a loop could have caught it. Three corpus problems come up, and in each the loop is helpless.
Several versions, no winner. The leave policy exists as a 2022 PDF, a 2024 wiki page and an email attachment someone saved. The embeddings are nearly identical, so retrieval returns whichever ranks highest. An agent that re-queries gets the same three candidates and has no means of knowing which is current. Someone has to retire the old ones or mark status in metadata, and that is an editorial job.
Documents that disagree. If the contract says 60 days and the policy page says 30, retrieval surfaces both, faithfully. The agent can notice the conflict, which is a real gain. Resolving it is a business decision.
Permissions applied late. If the agent searches everything and filters afterward, or does not filter, it can show a junior employee a board paper. Retrieval has to run as the requesting user with the same document-level access the source systems enforce. Helpfulness is no reason to grant a bypass.
The finding that changed the shape of this piece is what the loop does to these failures. Without it, a wrong answer from a stale document is at least a single, inspectable step. With it, the agent retries, reconciles and writes a smoother paragraph, so the same bad source arrives wrapped in more apparent diligence. None of this appears in a demo with ten clean documents. It appears in the third month of real use.
A fourth mistake sits earlier than all three: putting structured data into the vector store. Invoices, orders and stock levels exported so the agent can "search" them are exact data, and similarity search is the wrong tool for exact data. The open orders for one customer is a query with a precise answer, and the ERP gives it directly and currently, where an embedded copy goes stale. ThinqHub implements Odoo and ERPNext, and our view is that the useful meaning of "agentic" in RAG is routing. The agent decides whether the question belongs to documents or to records. Policies, contracts and manuals go to retrieval. Balances, statuses and quantities go to a permissioned query against the system of record. A single question often needs both, and handling that combination is what justifies the extra machinery.
What the loop leaves you to decide
The loop costs time. A single-pass pipeline answers in one round trip, while an agent that checks and retries might make four or five. Set a step budget and a time limit, and decide beforehand what happens when the budget runs out. The right outcome is "I could not find a reliable source", and the system must be able to say it. A poorly built agent falls back on the model's memory and produces something that sounds grounded and is not, which is the exact failure retrieval was meant to prevent. Log each retrieval decision so you can later show which source supported which claim, as in our post on AI agent observability.
For a single-turn FAQ over a well-kept knowledge base, plain RAG is the right choice. Reach for the agent when a wrong or incomplete answer is costly and the question spans sources.
A first step can be narrow. Choose one high-stakes question that crosses sources. Clean the documents it depends on, retire the stale versions, apply real permissions and wire the ERP query for the structured half. Then run the agentic version and the single-pass version against a fixed set of test questions, using the method in AI agent evals, and keep the agent only if wrong answers drop. To see where retrieval breaks in your own systems, talk to us about a scoped assessment, or read how our AI services are scoped.
