Human-centred AI in healthcare: why clinicians must stay in the loop


This article is part of our Opinions section, where we invite industry professionals to share their views on the most pressing technology questions of our time.


Artificial intelligence in healthcare has moved swiftly from conceptual technology to operational reality, offering transformative potential to enhance decision-making, reduce administrative burden, and streamline patient pathways. However, despite its promise, adoption across the NHS has been measured and cautious, reflecting the high stakes of clinical practice and the need for trust in safety-critical environments.

In 2025, conversations around AI matured beyond theoretical debate, highlighted by the recent NHS trial of Microsoft 365 Copilot. The results were striking. Staff saved an average of 43 minutes per person per day, which, if scaled across the system, could free up an estimated 400,000 hours of clinical and administrative time each month โ€“ a clear demonstration of AIโ€™s practical, admin-relieving potential when applied thoughtfully. The real barrier to AI adoption is therefore not capability, but trust. 

Clinicians are increasingly asked to rely on tools that make complex recommendations or automate critical administrative tasks, often without full visibility into the reasoning behind them. In an environment shaped by workforce pressures and rising patient complexity, this lack of transparency can quickly erode confidence and limit uptake, regardless of technical sophistication.

2026 must mark a turning point. Healthcare organisations cannot be expected to simply layer AI onto existing workflows with transformational results. Success depends on embedding AI in a human-centred way, where healthcare teams remain firmly at the decision-making helm. By prioritising transparency, explanation, and integration into real-world workflows, AI can become a trusted partner โ€“ crucially, augmenting professional expertise rather than replacing it โ€“ and delivering tangible improvements in efficiency, accuracy, and patient outcomes.

Augmenting experience, not replacing it

AI is changing every industry, but one principle remains consistent โ€“ subject matter expertise (human intelligence) is essential at every stage of its use. A human-in-the-loop approach underpins sustainable adoption, ensuring clinicians remain final decision-makers and AI is positioned as an augmenting tool instead of an authority. This approach addresses the core challenge identified in recent NHS trials โ€“ while AI can automate routine tasks and analyse complex datasets, clinicians will only ever trust recommendations they can understand, interrogate, and contextualise.

Embedding AI successfully requires more than technical integration. Interfaces must be intuitive, outputs explainable, and insights genuinely actionable at the point of care. Critically, staff need time and training to develop digital confidence and literacy. When these elements align, AI can relieve cognitive and administrative burdens, freeing clinicians to focus on tasks that require professional judgment, empathy, and nuanced decision-making.

Balancing automation and accountability in clinical practice

For AI to meaningfully support clinical practice, the balance between automation and accountability must be carefully calibrated. Automation has clear value โ€“ it can process information at a scale no human could achieve, triage administrative tasks, identify patterns, and surface critical insights within a matter of seconds. But in healthcare, speed is only useful if it is coupled with clarity. Clinicians remain accountable for decisions, and any tool that influences those decisions must therefore provide transparent, interpretable outputs that can be scrutinised and validated.

This is where the design of AI models and interfaces becomes pivotal. Tools that show their workings โ€“ highlighting which data points informed a recommendation, where uncertainties lie, and how confidence levels are weighted โ€“ enable clinicians to combine machine-generated insights with their own expertise and contextual understanding. When AI acts as an intelligent assistant, clinicians can make faster, more informed decisions with confidence, without relinquishing professional oversight.

From raw data to real value-add insight 

Across the NHS, healthcare teams contend with patient records that are longer, more fragmented, and more complex than ever before. In this environment, the value of AI lies not in simply retrieving information, but in interpreting it in a way that reflects clinical reasoning. Frontline care teams need AI assistants that can recognise terminology variations, correlate symptoms with medications, and draw connections across years of clinical encounters. The most useful tools are those that anticipate what a clinician might need next, surfacing relevant comorbidities, highlighting potential contraindications, and pointing to overlooked elements of the record.

Combined, these capabilities make AI a catalyst for sharper decision-making. Instead of trawling multiple systems for medication histories, procedure notes, adverse reaction logs, or previous admissions, clinicians can access a coherent, consolidated view within seconds. This level of contextual intelligence supports safer choices, whether medication adjustments or procedural planning, and ensures decisions are grounded in the full clinical picture. For patients, the result is shorter waiting times, earlier intervention, and better overall experiences and outcomes. 

Healthcare professionals will not defer to systems they cannot interrogate or fully understand. Transparency is imperative, as is the requirement for clinician involvement at every stage of the integration process. Building this understanding requires structured onboarding, ongoing training, and a baseline level of AI literacy across all roles, ensuring staff feel equipped not only to use these tools, but to question them and have the confidence to implement them into their everyday practice.

Why AI alone cannot fix a broken pathway

Trust is equally dependent on usability, but usability cannot exist in isolation from the realities of NHS workflows. Even the most sophisticated AI model becomes irrelevant if it sits outside the rhythm and pace of frontline care. 

Todayโ€™s patient pathway is characterised by a compounding cycle of delays. Bottlenecks at access lead to late diagnostics, incomplete context at the point of decision-making, fragmented handovers, and missed follow-up actions โ€“ all of which contribute to growing backlogs. AI applied on top of this, without addressing workflow fundamentals, risks accelerating inefficiency rather than resolving it.

To deliver true impact, AI must be embedded directly into clinical workflows, drawing from existing systems and presenting insights precisely when decisions are made. This requires close collaboration with trust teams (and crucially, clinicians) from the earliest stages of development. Suppliers, vendors, and all NHS collaborators must listen, design around real needs, and iterate based on frontline feedback. Responsible adoption is as much about implementation culture as it is about technology. AI should streamline the pathway, not reshape it, and break the cycle of clinical drag rather than add another layer of complexity. 

What happens if humans are removed from the loop

Removing healthcare professionals from the loop โ€“ and with them human oversight โ€“ could introduce risks to an already fragile care system. AI tools have been known to unintentionally reinforce automation bias and may overlook subtle cues buried in patient records. Clinical nuance, patient context, and, importantly, professional judgment will likely never be modelled to the point of 100% accuracy, and decisions made without these elements risk compromising patient outcomes and safety.

Furthermore, when clinicians, particularly, do not understand how an AI tool reached its conclusion, trust erodes. If the result is staff spending time on second-guessing outputs, the irony is that tools designed to speed up care will slow it down instead, and staff will revert to manual processes. 

Human-centred design provides the answer to this. By involving clinicians from the outset โ€“ addressing their pain points and providing rigorous training during implementation โ€“ AI becomes something they can contextualise, validate, and confidently use. In this model, the technology can do what it does best: reduce burden, surface timely critical information, and support quicker decision-making. 

The future of AI in healthcare will be defined less by technological capability and more by the choices made about how it is used. Systems designed to think for clinicians risk undermining trust, safety, and adoption. Those designed to think with them, however, have the potential to transform care. The most effective AI will be the most accountable โ€“ embedded into workflows, guided by clinical expertise, and transparent by design. Finally, progress will be measured not by how much AI can achieve alone, but by how well it strengthens human judgment where it matters most.

Jon Pickering
Jon Pickering

Jon Pickering is the CEO of Mizaic, a company that built an Electronic Document Management System for the NHS. He has contributed to TechFinitive under its Opinions section.