Companies keep upgrading their AI models the way people upgrade cameras, and keep taking the same bad photos. For most businesses the model is no longer the constraint. What holds AI back is what it knows about your business: a structured, current, trustworthy picture of your customers, projects, contracts and processes. Give a modest model that picture and it performs. Deny the best model on the market that picture and it will answer confidently from last year's documents.
The bottleneck moved from the model to the knowledge
Two years ago the question was whether the models were good enough. For the everyday work of a business, that question has been settled. The frontier models can read, summarise, reason and write better than most of the processes they are dropped into.
What has not been settled is what they are reasoning about. Gartner expects that through 2026 organisations will abandon 60% of AI projects unsupported by AI-ready data (Gartner, February 2025). The data.world benchmark makes the same point with a sharper edge: a model answering business questions directly over an enterprise SQL database scored 16.7%, and 54.2% when it worked over a knowledge graph of the same data, and scored zero on the most schema-heavy questions without one (data.world, 2023). Same model, different knowledge. Models have improved a great deal since, but better reasoning cannot supply facts the model was never given.
As Eric Seiberling, VP of Sales and Marketing at MXD Process, a US maker of industrial mixing equipment, put it on LinkedIn in August 2026: "Nobody has ever fixed dirty data by upgrading to a better model." That is the whole argument in one line.
Three signs your AI does not know your business
You do not need a benchmark to spot the problem. It shows up in three recognisable ways.
- Confident answers from stale documents. The assistant quotes a policy that was replaced last spring, because the old version and the new one sit side by side in the same drive and nothing marks which is current.
- Different answers to the same question. Ask twice, phrased slightly differently, and get two versions, because a different handful of documents comes back and nothing reconciles them. I explain why in why your RAG system keeps missing what is in your documents.
- No answer across systems. "Which of our key accounts also buy through a partner we are in dispute with?" needs the CRM, the contracts folder and the finance system to agree on who that partner is. They rarely do.
None of these is a model failure. Each is a knowledge failure, and each is fixable.
Every vendor now ships an agent on its own data
Something changed in summer 2026 that makes this more urgent, not less. Xero, HubSpot and Sage were among the many vendors that shipped AI agents working on the records they already hold, and almost every accounting platform, CRM, ERP and helpdesk now has one. Many of them are good.
But look at what each one knows. The CRM agent knows the CRM. The finance agent knows the ledger. Meanwhile Sage's survey of 1,500 UK small businesses found the average one juggling 52 digital tools and 27 separate logins (Sage, September 2026). Every vendor now ships an agent that knows its own data. Almost nobody ships one that knows yours across all of them.
That cross-system picture is where the expensive questions live, and it is the part no single application vendor is positioned to build, because none of them sees it.
What a knowledge layer actually is
Strip away the vocabulary ("company brain", "context engineering", "GraphRAG") and a knowledge layer does four plain things.
- It resolves entities. "Smith Manufacturing" in the CRM and "Smith Mfg Ltd" on an invoice become one company with one record, and "Dave at Smith's" in an email is recognised as working there. I explain how in what is an entity registry.
- It maps relationships. Who supplies what to which customer, under which contract, with which obligations.
- It knows what is current and where it came from. Superseded documents are marked as superseded. Every fact carries its source, so an answer can be checked.
- It lets agents traverse it. Instead of guessing which documents might be relevant, an agent follows the relationships to the facts it needs.
It sits between your systems and your AI. It reads from what you already run, including the sources most projects ignore, like email, the most underrated data source in your business.
I have been building this for 25 years. We just did not call it that
One of the first systems I built at scale fused radar, vessel tracking and sensor feeds into a single live operating picture that maritime authorities in more than 30 countries could act on. One client later called it "the first and only system they had ever used that never crashed", a story I tell in full in the quality you can't see. The hard part was never the display. It was deciding, continuously and reliably, which signals described the same ship.
Years later, as an expert witness in the Hong Kong High Court, I wrote software to reconstruct what had really happened from raw radar data, and that reconstruction survived six days of cross-examination. Same problem: messy signals in, one trustworthy picture out.
And when we built agentic AI for one of Europe's largest insurance brokerages, resolving 67% of customer service cases without a human, the hard work came before the agents. Before building any models, the data layer had to work. It is the step most AI projects skip, and it is usually why they fail.
Three objections, answered honestly
"Microsoft tried this and gave up." Microsoft retired Viva Topics in February 2025. It was an automatic topic-card feature with no agents on top and no joining-up of entities across systems. Its successor strategy, searching existing content with a strong model, is the approach that struggles most on relationship-heavy questions.
"Vector search is good enough." For most questions, it is. Roughly speaking, lookup questions ("what is our refund policy?") are well served by search. The knowledge layer earns its keep on the minority of questions that span entities and systems, where a wrong answer costs money. I compare the two approaches in GraphRAG vs vector RAG.
"Knowledge graphs cost a fortune to maintain." The hand-curated generation did. Language models have changed the economics: extraction and entity resolution that once took a team of specialists can now be largely automated, with people reviewing the exceptions. Scope it to one process, prove it, then grow it.
It is cheaper as well as better
There is a cost argument here, and in a world where AI is increasingly billed by use rather than by seat, it matters. Stuffing large amounts of loosely relevant text into every request costs tokens on every call, forever, and long contexts degrade recall even on simple tasks (Chroma, 2025). Microsoft's GraphRAG research found that answering broad questions from a graph of summarised entities took between 26% and 97% fewer tokens than summarising the source text directly (Microsoft Research, 2024). The details are in why grounded agents are cheaper to run.
The best production evidence comes from LinkedIn's customer support team, which grounded retrieval in a knowledge graph, improved retrieval quality (mean reciprocal rank) by 77.6% in offline testing and, after around six months in production, cut median per-issue resolution time by 28.6% (LinkedIn, SIGIR 2024). Better answers, and faster.
Where to start
You do not start with a platform. You start with one process.
- Find where the knowledge for that process actually lives. Not where the org chart says it lives. Inboxes, spreadsheets and people's heads included. This is what a knowledge audit is for.
- Pick a process where wrong answers are expensive. Contract obligations, claims, quotes, compliance. Somewhere accuracy is worth money.
- Resolve the handful of entities it depends on. Usually customers, suppliers, projects and documents. Not everything, just what this process needs.
- Ground one agent in that picture and measure it. Against a baseline, on real cases, with a human reviewing the exceptions.
- Expand only when it has earned it. Each process you add enriches the picture the next one uses. That compounding is the real prize.
If you have already been through a model upgrade that did not move the needle, you are not behind. You have simply found the real constraint. It is worth looking at the companies that are getting returns from AI: in my experience, they did this work first, whether or not they called it a knowledge layer.
If your AI keeps answering from what your business knew last year, a Knowledge Audit is the fastest way to find out why and where to start. Or simply get in touch.
Related: Company brain products are useful. Most are not a brain yet · What I look at in a knowledge audit · You're not talking to an LLM. You're talking to a system · How to unlock AI ROI: what the 20% do differently
