What regulated industries already understand about AI risk and what they don’t


This article is part of our Opinions section, where we invite industry professionals to share their views on the most pressing technology questions of our time.


Healthcare, financial services, and public sector organisations are often described as slow adopters of AI. And from the outside, it is easy to see why. A new system has to move through security reviews, compliance checks, approval chains, and governance processes before it can touch production.

Because of the high-stakes nature of these industries – where AI could potentially influence a healthcare decision, a loan outcome, or someone’s eligibility for a government benefit – “How fast can we deploy?” isn’t the right question to ask when determining AI readiness. 

Regulated industries have spent decades building the infrastructure to answer harder questions: What is an acceptable level of risk for this decision? Who is accountable for the outcome? What data can be accessed? Where does a human need to intervene?

That instinct is valuable. In many ways, it puts regulated organisations ahead of the market.

The challenge is that the systems underneath that instinct have not always caught up with AI.

AI’s hardest problems live at the data layer

Most AI conversations still focus heavily on the model: how large it is, how well it benchmarks, how accurately it reasons.

But when I think about deploying AI in an enterprise environment, the model is only one piece of the problem.

I want to know:

  • Can we trace which data the system accessed to produce an output?
  • Does the AI have the same permissions a human user would have, or has it inherited broader access because that was easier to configure?
  • If the output is wrong, can we reconstruct what happened?
  • Can we prove that the data was authorised for the use case in the first place?

The current data suggests that most organisations cannot answer these questions consistently.

Kiteworks’ 2026 Data Security and Compliance Risk Forecast found that 78% of organisations cannot validate data before it enters AI training pipelines, while 77% cannot trace where their training data came from.

That is not fundamentally a model problem – it’s a governance problem. And it is one that regulated organisations already know how to solve for other technologies. Access controls, data lineage, audit logging, separation of duties, and evidence collection are not new concepts. The technical challenge is extending that discipline to AI.

Risk has to be tiered at the use-case level

One of the mistakes I see most often is applying a single set of guardrails to every AI use case regardless of what the system is doing. While AI guardrails are critical, they need to match the use case’s context.

A model summarising internal meeting notes is not equivalent to a system influencing a treatment path or a financial decision. The potential consequences, data sensitivity, and need for human intervention are all different.

Regulated organisations already understand this concept outside of AI. A system handling highly sensitive information has different permissions, logging requirements, approval thresholds, and review cycles than a system handling routine business data.

AI should work the same way. When building your AI guardrails, I recommend to start by looking at three things: the sensitivity of the data, the reversibility of the decision, and the level of human oversight between the AI output and the action it triggers.

That creates a more practical risk model. A low-risk use case should not have to navigate the same process as a high-consequence one. If we build risk tiering into the architecture, organisations can move quickly where the risk is low while applying stronger controls where the consequences demand them.

That is how regulated industries can move beyond the perception that governance automatically means slower innovation.

“Human in the loop” is a systems-design problem

An AI governance policy is necessary, but it isn’t sufficient on its own. While almost every governance document mandates that a human reviews AI outputs, far fewer systems are actually built so that meaningful review is possible.

Making “human in the loop” work is fundamentally a systems-design problem, not just a policy statement. For a review to be real, the reviewer needs full visibility into the process: What data did the model access? What did it produce? What assumptions did it make? What happened before the output reached them? Without that context, human review is little more than a rubber stamp.

This distinction matters far more than standard governance frameworks acknowledge. Data from IBM’s Cost of a Data Breach Report 2025 shows that 63% of organisations that experienced an AI-related breach either had no policy or were still drafting one – and for the remaining 37% that did have policies, written documentation alone clearly failed to stop the breach.

Regulated organisations already have the operational infrastructure to fix this elsewhere in the enterprise. Accountability is made real through concrete mechanics: separation of duties enforced at the permissions layer, approval thresholds that block a transaction rather than just flagging it, escalation paths that route decisions to a specific accountable person, and audit trails that preserve evidence.

The next step is extending those mechanisms to AI outputs.

Where governance policy meets its limits

The gap between policy and infrastructure is becoming harder to ignore. According to Prefactor, 46% of enterprise leaders identify governance and oversight as a top AI risk. 

That concern makes sense. You cannot enforce governance effectively if the underlying environment does not have reliable data lineage, consistent metadata, clear access controls, and a trustworthy source of truth.

Consider something as basic as document versioning. If the same policy document exists in four systems, with four different versions and unclear ownership, adding an AI agent does not solve the problem. It potentially increases the ambiguity.

The most important AI governance work is the most basic – ensuring clean data, clear ownership, consistent permissions, traceable workflows, and reliable audit trails. These are the fundamental components that allow governance policies to become technical controls rather than documents sitting in a compliance repository.

And there is increasing pressure to get those things right. Gartner’s 2026 Audit Plan Hot Spots research, based on a survey of 160 chief audit executives, found that 94% of respondents planned data-governance coverage in their 2026 audit activities, while 97% planned coverage for regulatory compliance. 

In other words, the question is increasingly not whether organisations will be asked about AI governance. It is whether they will be able to show the evidence behind it.

The advantage of regulated industries

I don’t think regulated industries need to win the race to deploy AI fastest. Their advantage is different.

They already understand that accountability has to be designed into a system. They understand that permissions need to be enforced, approvals need to be meaningful, sensitive data needs to be controlled, and important actions need to leave evidence behind.

The opportunity now is to apply that same discipline to AI. For me, that starts with three practical moves.

First, build data lineage and access controls into the architecture before scaling an AI use case. Know what the system can see, why it can see it, and where that data came from.

Second, design human review into the workflow itself. Don’t just document that someone is supposed to approve an output. Build the permissions, context, escalation, and auditability that make that review meaningful.

Third, stress-test the environment before expanding into higher-risk use cases. Ask a simple question: What did the AI see, what did it do, and who signed off?

Then ask whether you can answer it on demand. If you cannot, that is the work that comes before the next AI deployment. AI will continue to get faster, more capable, and more autonomous. The organisations that can take advantage of that progress responsibly will not necessarily be the ones with the most sophisticated models.

They will be the ones with the strongest systems of accountability underneath them. And regulated industries already know what that looks like. The next step is making sure their technology does, too.

More from our Opinions section

About The Author

Avatar photo
Anand Narasimhan

Bringing over 20 years of technical leadership experience to S-Docs, Anand Narasimhan oversees the Product, Engineering, and Professional Services departments to drive long-term innovation and deliver the latest technologies to customers.

Read more from this author.

We take journalism seriously. To learn more on why you should trust us, head to our editorial guidelines page or meet our team.