Trending Articles
Trending Topics
Trending
This article is part of our Opinions section, where we invite industry professionals to share their views on the most pressing technology questions of our time.
Every few months, a new governance framework lands. Another report on AI risk gets published. Another LinkedIn thread erupts about whether enterprise AI can be trusted. Yet after all that debate, the core problem remains unchanged.
Having spent the past six months connecting AI to real enterprise systems, including engineering platforms, support environments, sales data and customer records, I’ve reached a conclusion that runs counter to much of the current conversation. The question isn’t whether you can trust the AI. It’s whether you can trust the foundation beneath it.
It’s a not-so-subtle difference that changes everything, especially where you should look for the solution. Ultimately, trust is not something we grant because an answer sounds convincing. It is something AI systems earn by making their reasoning visible and their foundations verifiable.
That’s becoming important as AI moves into everyday workflows. The largest global study of AI trust to date, conducted by KPMG and the University of Melbourne across 47 countries, found that 57% of employees rely on AI-generated output without evaluating its accuracy. In other words, trust is already being granted without it necessarily being earned.
When organisations talk about AI trust, they almost always end up at “hallucinations.” Hallucinations matter. I’m not suggesting otherwise. But in enterprise environments, they are often the most visible symptom of a deeper problem: systems trying to generate confident answers from weak, disconnected or inferred foundations.
Fixating on hallucinations is like worrying about which brand of flour you used when your bread didn’t rise, and you didn’t proof your yeast.
Think about what a large language model actually is: a next-word generator. If I start that generation process with garbage, whether that’s vague prompts, poorly connected data or relationships that have been inferred rather than mapped, I get garbage out. The difference is that today’s models produce garbage that sounds remarkably convincing.
Prompt engineering doesn’t fix this. Telling people to write better prompts is like telling them to brief their junior employee more clearly. The junior employee is eager to please and will try their best to give you an answer regardless of whether the data supports it. The problem isn’t the request. It’s the foundation they’re working from.
This is where the architecture matters. Many AI systems infer relationships from names, patterns, documents or semantic similarity. That can be useful, but it is not the same as reading a relationship that already exists in the underlying data structure.
Here’s a real enterprise question: “For each high-priority engineering issue open right now, which customer accounts have open support tickets in the same product area?”
A VP of Support or a CSM could easily ask that on any given Monday morning. To answer it correctly, the AI needs to connect engineering issues, product areas, support tickets and customer accounts. It can’t simply summarise documents about those areas of the business. It needs to read the relationships between them from authoritative data.
I’ve run this kind of query with a general-purpose AI connected to live systems, and watched it return a confident, well-formatted product hierarchy that was entirely inferred from naming patterns rather than read from the actual data model. It looked right. It wasn’t. The only way I caught it was because I already knew how the data was structured.
If you need to already know the answer to verify the answer, you’re doing the hard part yourself.
That is the uncomfortable reality in many enterprise AI deployments: the user becomes the verification layer.
The right mental model here isn’t a technology audit. It’s a new employee.
When someone joins your team fresh from university, you don’t hand them a critical deliverable and assume they’ll nail it. You give them a task, ask them to show their work, and check. Not because you assume incompetence, but because they haven’t earned your trust yet. Over time, if they consistently walk you through their reasoning and cite their sources when questioned, you start to trust them. You stop checking every output. Because ultimately, trust builds on evidence.
AI should work the same way. What does “showing work” look like for an AI system? It means being able to ask: how did you get that answer? Which records did you use? Can you trace the logic? If the system can walk back through a deterministic query and show that here’s the data it returned and here’s the relationship it covered, you have something you can actually verify. The proof is in the provenance.
Now compare that with a system that delivers a response with no traceable path back to its sources. A wrong answer looks identical to a right one. No signal. No disclaimer. That’s not a foundation for trust. That’s a coin flip with better formatting.
Not every question deserves the same level of trust. If I ask which customers have open support tickets connected to a high-priority engineering issue, I should expect a traceable answer. If I ask what our three-year strategy should be, I am asking for judgment and opinion. That output may still be useful, but it should be evaluated differently.
If the trust problem is really a data foundation problem, then the criteria by which you evaluate it need to change. The question to ask of any AI system you’re deploying isn’t which model it runs. For most enterprise use cases, the major models available today are broadly capable.
Gartner recently captured this challenge succinctly: “Without trust in the data, outputs and decisions of AI models and agents, there is no value from AI.” That’s because value isn’t created by the model alone. It emerges when organisations can trust the information retrieved, the relationships interpreted, and the decisions informed by those outputs.
I’ve seen this firsthand. Give multiple models access to the same grounded data, and they’ll often produce different-looking responses. One may favour prose, another a table, another a different structure altogether. But the underlying facts remain consistent because the foundation remains consistent. That’s why I keep coming back to the same point: the model isn’t the variable. The foundation is.
The questions worth asking sit one layer beneath the model. Are the relationships in your data explicit or inferred? Can the system retrieve information directly from systems of record, or is it simply reasoning over disconnected fragments? When it produces an answer, can you trace it back to the underlying records? And can it do all of this while respecting the existing permissions and governance structures within the business?
These aren’t abstract architectural questions. They determine whether the trust you build in an AI system is real or illusory.
Earned trust grows over time. When you’ve asked “how did you get that answer?” enough times and the answer is consistently traceable, you stop checking every response. That’s completely natural. It’s how we trust people, and it’s how we’ll eventually trust AI systems.
But it cuts both ways. If the foundation occasionally infers rather than reads, and looks right every time, the trust you’ve built is sitting on sand. Worse, if those wrong answers feed into recurring decisions, the errors multiply. You don’t just get one bad answer. You get a feedback loop built on one.
The problem is that many organisations are trying to skip that step. They’re focusing on the model and hoping trust follows. It doesn’t. That’s why I think we’re looking in the wrong place when we talk about AI trust. We’ve spent two years debating models. The harder question is whether we can explain how an answer was produced in the first place. All the while knowing that the model isn’t the variable, the foundation is.
