Vector RAG is for finding text. GraphRAG is for reasoning across records. If most of the questions your people ask can be answered from one passage of one current document, vector retrieval is quick to build and good enough. If your important questions span customers, contracts, suppliers and systems, and a wrong answer costs money, you need a knowledge graph underneath. Most businesses end up needing both, with each question routed to the right one. This page compares the two in plain English, shows what the evidence says, and gives you a way to decide in an afternoon.
Vector RAG, in one paragraph
Vector retrieval-augmented generation cuts your documents into passages and places each one on a map of meaning, so similar passages sit close together. When a question arrives, it finds the passages closest to it and hands them to the model to answer from. It needs no schema, which is why it is quick to set up. Modern versions add keyword search, re-ranking and metadata filters, which close part of the gap described below. It is excellent at "find me the bit that says X". It has no idea that two passages describe the same customer, that one contract supersedes another, or that a supplier in an email is the one in your finance system. For those, it relies on the model to work it out from whatever passages happened to be retrieved, which is where the missing structure shows, as I explain in why your RAG system keeps missing what is in your documents.
GraphRAG, in one paragraph
GraphRAG grounds the model in a knowledge graph: a structured map of the entities your business deals with and the relationships between them. The customer is one node, whatever it is called in each system. It connects to its contracts, its orders, its open issues and the people who work with it, and each fact can carry its source and date. The term is used in two ways. Microsoft uses it for a specific technique that builds a graph of entities and topic summaries from a body of text; it shines on broad questions across a whole collection, such as the main themes in a year of complaints. More broadly, it means retrieval over a graph of resolved entities and relationships, which shines on precise questions across records. Either way, the model is no longer guessing which passages matter. It follows the connections to the facts it needs.
Side by side
| Vector RAG | GraphRAG | |
|---|---|---|
| Best at | Finding relevant passages of text | Reasoning across entities and relationships |
| Typical question | "What does our returns policy say?" | "Which suppliers in a live dispute are on contracts we are about to extend?" |
| Setup effort | Low: index the documents | Higher: extract entities, resolve duplicates, define relationships |
| Knows what is current | Only if you add it as metadata | When you model it (effective dates, supersedes links) |
| Same entity, different names | Treated as different text | Resolved to one record, if you invest in entity resolution |
| Counting and comparing across records | Weak | Strong |
| Shows where each fact came from | Per passage | Per fact, if provenance is stored |
| Cost | Low and predictable per query | Varies by method; building the index is the big cost |
| Typical failure | Plausible answer from the wrong or stale passage | Wrong or missing answer where extraction or entity matching went wrong |
Note the last row. Both can fail confidently. A graph's advantage is that you can inspect the path it followed to an answer, so errors are easier to find and fix. For anything important, that inspectability is worth a great deal.
When vector RAG is enough
Be honest about your questions before you invest in anything more elaborate. Vector retrieval is enough when most questions are lookups answered by a single passage, when your document set is reasonably tidy and current, and when the cost of an occasional wrong answer is low.
Internal knowledge bases, product documentation, HR and IT help, and first-line customer queries often fall into this category. So do many early pilots, which is one reason vector RAG gets a good reputation in demos.
Even here, two cheap disciplines make a large difference. Record which documents are current and filter on it, so superseded versions never reach the model. And measure accuracy on a set of real questions rather than trusting a usage dashboard. If those two are in place and the answers are right, you do not need a graph for this job. Do not build a graph to win an architecture argument.
When you need a graph
You need a graph when the questions that matter most are not lookups. The signs are recognisable.
The question spans several systems: the CRM, the contract repository and the finance system must agree on who a customer is. It needs several steps: find the customer, then its policies, then the complaints raised against those policies. It asks you to count or compare: how many, which ones, which is largest. The same entity appears under different names in different places. Or you must show exactly where each fact in the answer came from, because a regulator, an auditor or a court may ask.
These are typically the questions attached to money and risk: contract obligations, claims, compliance, supplier exposure, pricing. They are also the questions a vector system is least equipped to answer, because the pieces of the answer live in different documents and systems that nobody connected. If the data is already structured in one database, a well-built text-to-SQL layer may be enough; a graph earns its place when the answer spans sources.
What the evidence says
The research is encouraging, but it pays to read what each study actually compared.
The data.world benchmark (2023) asked GPT-4 business questions over an enterprise database. Answering over the raw database, it scored 16.7%; answering over a knowledge graph of the same data, 54.2%, with the most schema-heavy questions going from zero to answerable. That compares a graph with a raw database, not with vector search, but it shows how much explicit structure helps a model reason about business data.
LinkedIn's customer support team is the strongest production case. Adding a knowledge graph built from past support tickets improved retrieval ranking (mean reciprocal rank) by 77.6% in offline testing and, after around six months in production, cut median per-issue resolution time by 28.6% (LinkedIn, SIGIR 2024).
On cost, Microsoft found that answering broad questions from graph summaries used between 26% and 97% fewer tokens than summarising the source text directly (Microsoft Research, 2024). Building the graph costs more up front. The running cost matters more every month, as AI moves from per-seat to per-use pricing, a point I develop in why grounded agents are cheaper to run.
The honest costs of a graph
A knowledge graph is not free, and anyone who tells you otherwise is selling one.
You have to decide which entities and relationships matter, extract them from documents and systems, resolve duplicates into single records, and keep all of that current as the business changes, which means re-extracting when documents change and paying for the language-model compute that building the index takes. The old generation of knowledge graphs did most of this by hand, with teams of specialists, and many of them collapsed under their own upkeep.
This is where language models earn their place. Extraction and entity resolution that once needed people for every record can now be largely automated, with people reviewing only the exceptions. The other lever is scope. You do not build a graph of your whole company. You build one for the process where wrong answers are expensive, prove it pays, and extend it one process at a time. The central piece is an entity registry, and it is less exotic than it sounds.
The hybrid most businesses end up with
In practice the choice is rarely either-or. Mature systems use both, and the best ones decide per question.
A common pattern works in two stages. The graph identifies the right entities, filters out anything superseded and follows the relationships, which narrows the field to the records that matter. Vector search then finds the most relevant passages within that narrowed set. The graph provides precision and structure; vector search provides reach into the text.
The reverse also works: vector search finds the starting entities for a question, and the graph expands outward from them to gather everything connected. Microsoft calls this local search.
A simpler version routes questions. Lookup questions go straight to vector retrieval. Questions that mention several entities, ask for counts or comparisons, or need provenance go to the graph. Either way, the entity registry is the bridge, because it is what lets a passage of text and a record in a system be recognised as being about the same thing.
How to decide in an afternoon
Gather the twenty questions your people most need answered, including the ones they currently answer by hand with three spreadsheets and a phone call. For each, note two things: is it a lookup or a relationship question, and what does a wrong answer cost?
If nearly all are cheap lookups, invest in vector retrieval, document hygiene and accuracy testing. If a meaningful share are relationship questions carrying real money or risk, build a graph for that process, and keep vector retrieval for the rest.
That exercise is the heart of a knowledge audit, and it is the practical starting point for the argument I make in your AI does not need a bigger model, it needs to know your business. Choose the right structure for the question, not the fashionable one.
If you are deciding between approaches for a real system and want a second opinion from someone who has built retrieval systems that run in production, like the one behind 67% autonomous case resolution at a European insurance brokerage, I am glad to help. Let's talk.
Related: Why your RAG system keeps missing what's in your documents · What is an entity registry, and why does your AI need one? · Company brain products are useful. Most are not a brain yet · You're not talking to an LLM. You're talking to a system
