Copilot AI risks run deeper than Microsoft’s confidential email debacle

You know that AI thing that’s knocking around, that’s meant to make our working lives so much easier? There’s a reason I flag the cybersecurity and privacy risks of using the likes of OpenClaw AI agents – and employees using shadow AI. As the name suggests, this is AI usage you don’t know about, let alone have approved of.

But what about Microsoft 365 Copilot? Surely that gets the corporate all clear and is out of reach my “just wait a cotton-picking minute” cybersecurity slings and arrows? Well, wait a cotton-picking minute, apparently not.

Microsoft itself has confirmed that the Microsoft 365 Copilot Chat work tab has, since at least 21 January, been able to read and summarise email messages that are marked as confidential and supposedly protected by data loss prevention tools.

“Users’ email messages with a confidential label applied are being incorrectly processed by Microsoft 365 Copilot chat,” Microsoft stated, adding that the work tab chat is “summarizing email messages even though these email messages have a sensitivity label applied and a DLP policy is configured.”

But don’t worry, a fix has already started rolling out, so that’s OK then. Right?

Well, no, not really. As Yagub Rahimov, CEO of Polygraf AI, pointed out, “DLP policies were designed for a pre-AI world, and adding them to systems that were never built with AI access patterns in mind creates exactly these kinds of gaps.”

Rahimov also warned that every CISO reading this should start to realise that they likely can’t answer one simple question: what does our AI actually have access to right now? AJ Grotto, a cybersecurity expert and former Senior White House Director for Cyber Policy, is harsh but fair when he warns that “security problems like these risk giving enterprise AI a bad name and deterring organisations from adopting AI.”

Copilot AI at work in Outlook
Copilot AI at work in Outlook (GIF created by Microsoft)

Copilot AI risks… and AI risks in general

I’m not sure that’s such a bad thing, to be honest. At least from the cybersecurity perspective. I mean, if you don’t have a handle on your security, a proper grasp of the risk, then should you be jumping into something just to keep up with the competing keeping up with the competing Joneses?

Yeah, I know, it’s not that black and white, nothing in business ever is. But it is a question that needs to be taken seriously. Especially as a new report from Check Point Research suggests that tools such as Microsoft Copilot can be “quietly exploited by cybercriminals as part of their attack infrastructure”.

The issue is, Check Point said, that trusted web-based AI assistants can be “abused as covert communication channels between malware and attackers, effectively turning AI services into an invisible command-and-control layer”.

Yes, that is as worrying as it sounds. Not least, as it means that criminal communications can hide inside the standard AI traffic your business is already generating and trusts accordingly.

“To stay safe, organisations must monitor AI traffic with the same scrutiny as any other high-risk channel, enforce tighter controls around AI-powered features,” said Eli Smadja, Head of Research, Check Point Research.

In particular, they should “adopt security measures that understand not only what AI is doing, but why, leveraging agentic AI capabilities to inspect and contextualise traffic to and from AI services and block malicious communication attempts before they can be abused as covert channels.”

Avatar photo
Davey Winder

With four decades of experience, Davey is one of the UK's most respected cybersecurity writers and a contributing editor to PC Pro magazine. He is also a senior contributor at Forbes. You can find him at TechFinitive covering all things cybersecurity.