An AI agent aced its demo, answered every revenue question the CFO threw at it, and got the
green light to go live. Then, in production, it told the finance team Q4 revenue was $12
million. The real number was $8.4 million. The model had not made a math error. It pulled
revenue_recognized instead of revenue_net_of_returns, because nobody had told it which
field carried the authoritative definition (Atlan, 2026).
That gap, between having data and understanding what the data means, is why a growing number of enterprise AI teams are shifting their attention from prompt engineering to context engineering. Prompt engineering optimizes the question you ask a model. Context engineering optimizes everything the model can see before it answers: which systems it can query, which definitions are current, and which entity records actually match across departments (Salesforce, 2026).
Why this matters more than model choice
Most agent failures get blamed on the model. The data says otherwise. Only 7% of enterprises report their data is completely ready for AI use, according to a March 2026 study by Cloudera and Harvard Business Review Analytic Services. And per DataHub's State of Context Management Report 2026, 88% of enterprises claim their context is "operational," yet 61% still delay AI initiatives because that context is not usable in practice (Atlan, 2026).
The trust numbers are moving in the same direction. Capgemini Research Institute found that trust in fully autonomous AI agents dropped from 43% to 27% in a single year, and 80% of organizations still lack mature AI infrastructure to support them (Capgemini, 2025). Stronger models do not fix this. A more capable model just states its wrong answer more convincingly.
| Metric | Value | Source |
|---|---|---|
| Enterprises with fully AI-ready data | 7% | Cloudera / HBR Analytic Services, Mar 2026 |
| Enterprises claiming context is operational | 88% | DataHub State of Context Management Report 2026 |
| Initiatives delayed by unusable context | 61% | DataHub State of Context Management Report 2026 |
| Organizations lacking mature AI infrastructure | 80% | Capgemini Research Institute, 2025 |
The four failure modes behind most agent mistakes
Enterprise agents built without a context layer tend to fail the same four ways, and none of them are model problems:
- Meaning fragments across systems. "Customer" in the CRM, billing, and support desk often carries three different identifiers with no mapping between them. The agent joins the tables anyway and produces a number that looks right but isn't.
- Business definitions live as tribal knowledge. If nobody encodes which metric is certified, the model fills the gap with a plausible guess.
- Entity identity resolves differently everywhere. A question spanning CRM, billing, and support requires the same real-world customer to be recognized across all three, which most schemas were never built to guarantee.
- Answers can't be traced to a source. Without provenance, a wrong number is indistinguishable from a right one until someone downstream notices the damage.
What a context layer actually does
A context layer sits between your business systems and any agent that queries them. It translates raw metadata (table names, columns) into governed business meaning: certified metric definitions, cross-system entity mapping, access policy, and an audit trail for every answer (Atlan, 2026). This is distinct from the interoperability layer that protocols like MCP provide (see our piece on AI agent interoperability) — MCP lets an agent reach a tool or data source at all; context engineering governs what the agent understands once it gets there.
In practice, building this layer looks less like a model upgrade and more like a data program:
- Certify metric definitions once, in one place, and require agents to read from that source rather than re-deriving definitions per query.
- Resolve entity identity centrally (a canonical customer or product ID), so every system-specific identifier maps back to one record.
- Log provenance on every agent answer, so a finance team can trace a number back to the query and table that produced it.
- Scope what each agent can see to the systems and permission set relevant to its job, not blanket read access to the estate.
This is also where deterministic guardrails matter. A reasoning model cannot reliably guarantee that a banking agent verifies identity before discussing an account balance; that sequencing has to be enforced outside the model, in explicit workflow logic (Salesforce, 2026). The lesson generalizes: the parts of an agent's behavior that must always be correct belong in deterministic code, not in a prompt.
Where this fits for SMEs
Most SMEs do not need a full enterprise data graph to start. They need the same discipline at a smaller scale: pick the two or three metrics your team actually argues about (revenue recognized vs. booked, active customer vs. registered account), write down the one true definition, and point any AI agent or ERP integration at that definition instead of letting it infer one. This is the same groundwork we lay before any agentic AI deployment we run for clients, and it is usually the difference between an agent that survives contact with production data and one that gets quietly switched off after its first wrong number.
If you are scoping an AI agent project and want to know whether your data is ready for it before you commit budget to a model, our approach starts with exactly that audit. Get in touch to talk through where your context gaps are before you build on top of them.
