Healthcare AI must run on trust. Governance is how you earn it.

As part of our new “The future of life sciences R&D” series, exploring how emerging technologies – from AI and digital trials to synthetic biology and advanced data platforms – are reshaping the future of drug discovery, clinical research, and scientific innovation, we invited Jessica Finn, Health and Life Sciences Industry Lead ANZ, Cognizant, to share her views.

Here, Jessica argues that trust is the foundation on which healthcare AI must be built. Drawing on her experience helping healthcare organisations navigate digital transformation, Jessica examines why governance should be viewed not as a barrier to innovation, but as a strategic enabler that allows organisations to deploy AI safely, responsibly, and at scale while maintaining the confidence of clinicians, patients, regulators, and leadership teams.

Behind every AI decision in healthcare is a real person. A patient waiting for care. A clinician trying to make the right call under pressure. That needs to be the starting point of every conversation about AI in this sector.

When you strip away the hype, the jargon and endless proof of concepts, healthcare is ultimately about trust. People need care that feels safe, human and designed around them, not around the technology itself.

And the stakes here are incredibly high. A misdiagnosis can directly impact someone’s life. A privacy breach can erode trust overnight. A biased algorithm can deepen existing disparities in access and outcomes. That’s why healthcare cannot afford AI that operates without strong governance, accountability and clinical oversight built in from day one.

The biggest mistake healthcare organisations are making right now with AI adoption is focusing on proving the model works at a pilot level. But proving a model works in a controlled environment is the easy part. 

63% of enterprises report moderate-to-large gaps between their AI ambitions and current capabilities, and regulatory and compliance concerns rank as the single biggest barrier to AI adoption, ahead of ROI, talent shortages and data readiness.

The challenge is engineering AI into a controllable, auditable and operational capability that can be deployed safely and consistently across healthcare systems. That means policy, clinical accountability, risk management, and workforce readiness to show that the organisation is ready to use it responsibly at scale.

The governance misconception

The governance needed to enable responsible AI is still too often seen as the barrier that slows everything down. Historically, governance has been tied to layers of hierarchy, long approval cycles and processes that can feel disconnected from the pace and reality of delivery teams on the ground.

But healthcare cannot afford to treat governance as a blocker. Done well, governance should be an enabler of AI at scale. It should create the guardrails that allow organisations to move faster with confidence, knowing privacy, safety, fairness and clinical accountability have been built in from the beginning rather than added retrospectively when things go wrong.

While the idea of governance being the “department of no” is a familiar, and at times fair perception, it doesn’t mean governance itself is the problem.

All too often, governance is treated as an add-on, which creates friction, breeds frustration, encourages workarounds and leaves accountability unclear when something goes wrong. Ultimately, this slows innovation down. The opportunity is to modernise governance so it’s embedded in delivery from day one, with clear roles, guardrails and proportionate oversight, so teams can move into real-world deployment with confidence. 

If governance continues to be seen purely as a barrier, separate to delivery, innovation and clinical outcomes, organisations will struggle to scale AI in any meaningful or sustainable way. Organisations that fail to evolve risk being left behind, stuck in cycles of pilots and proof of concepts while others realise the full value of AI across the enterprise.

What good governance actually looks like

Good governance in practice means having clear, consistent answers to questions that shouldn’t be guessed at or improvised. And not every use case should be treated in the same way.

Take something relatively straightforward, like an AI tool that auto-fills clinical documentation. On the surface, it sounds low risk. But there are still important questions that need to be answered. What happens if the tool records something incorrectly and a clinician doesn’t pick it up? Who is ultimately accountable? Is the output auditable? How are errors monitored and corrected over time?

Now compare that to an AI model designed to identify deteriorating patients in an ICU setting, where the output could directly influence clinical decisions and patient outcomes. In that scenario, the level of evaluation, oversight, governance and accountability needs to be far more rigorous.

These are two completely different risk profiles, so it makes little sense to force them through the exact same governance process.

That’s where modern AI governance needs to evolve. It should be risk-based, practical and proportionate, applying the right level of oversight to the right use case. Otherwise, organisations end up creating unnecessary friction for low-risk innovation, while potentially missing the depth of scrutiny required for high-risk clinical applications.

Behind every AI tool is someone who has to trust it

Another common mistake is assuming AI adoption is mostly a technology problem. In my experience working with healthcare organisations, the technology is rarely what derails a deployment.

In reality, the success of healthcare AI is balanced on how well it fits clinical workflows, whether healthcare workers know when to trust it, and how effectively leaders can prove that the safeguards in place work. In fact, across the board, enterprises are consistently rejecting off-the-shelf AI solutions due to poor fit with existing workflows and lack of industry-specific expertise. Healthcare is no different, except the consequences of poor fit are considerably higher.

The hardest questions in AI adoption are human ones. What decision is the tool supporting? Who is accountable if it goes wrong? How are staff trained to use it, and when to trust their own judgement? How are patients informed?

Trust depends on human oversight, transparency, accountability and continuous evaluation in the real environment where the technology is used, beyond the controlled conditions of a pilot.

When those questions aren’t answered upfront, governance fails in predictable ways. Risk hides in untested assumptions, monitoring gaps go ignored, and when something goes wrong, accountability becomes unclear.

The consequence is a loss of clinician trust. Often, they already sensed something wasn’t right, but no one clearly explained what the tool was meant to do, where its limitations were, or what happens if it gets it wrong. And in healthcare, once trust is lost, adoption quickly stalls.

Clinicians can’t be blamed for being sceptical of tools they do not understand, with safeguards that are unclear. However, in a recent implementation within the aged care sector, we were able to build trust through clinician-led benchmarking where a senior panel defined ‘gold standard’ answers and the model’s summaries were tested against a registered nurse baseline (with ongoing re-testing and zero tolerance for hallucinations). This approach helped drive strong adoption and significantly reduced handover review times. Because clinicians can see exactly what the tool is designed to do and the rules of engagement were clear from the start, there is less friction, more clinician trust and ultimately willingness to engage.

Modern governance is how you create trust, and trust is what allows you to scale.

From pilot to practice

It is critical that healthcare organisations treat governance as a strategic capability, building it deliberately, resourcing it properly and embedding it across every function, from legal to clinical to executive leadership. Everyone should adopt a governance mindset.

Organisation-wide adoption of AI in healthcare requires trust more than any industry, because the stakes are that much higher, and that cannot be built with technology alone.

Patients need to know the tools supporting their care are safe and fair, clinicians must be assured they retain meaningful oversight and boards and regulators must ensure safeguards are proportionate to the risk.

Trust is earned. That means building the appropriate structures and accountability frameworks that make adoption sustainable.