Who’s to blame for rogue AI? It could be you as ignorance of the law is no defence

OpenAI’s agents have been tearing about the web, breaking into secured websites to answer benign questions as part of evaluations.

That includes Hugging Face, American government and university websites, and even a healthcare website run by the Australian government. The latter is super annoyed about it, not least because OpenAI took its sweet time spotting the intrusion and then informed the government weeks later via a public-facing disclosure email. Some conversations require picking up the phone.

Indeed, the deputy prime minister has suggested that criminal charges are being considered by the investigation, saying it: “definitely does raise questions about whether the law has been broken in respect of this”.

Get the cuffs ready for OpenAI… or for you?

That raises intriguing questions about why these AI developers have yet to face hacking charges. The short answer is intent: OpenAI tasked its agents with finding information; it didn’t tell them to hack Australia.

Rebecca Parry, a law professor at Nottingham Trent University, considered this idea in a blog post earlier this month. This references a separate case in which someone asked OpenClaw to get him a gym booking: it then hacked the reservation system to delete another customer.

“The [computer misuse act in the UK] assumes that the person forming the intent and the person performing the wrongful act are the same,” she wrote. “Agentic delegation splits them, and a human principal’s innocuous instruction is unlikely to supply the intention or recklessness that the Act requires.”

She pointed to a recent legal statement on liability for AI harms. This said that an AI system can’t be held liable, though the person using it could in particular via existing principles like negligence – especially now that such issues are foreseeable. If you didn’t know, now you know.

Allocating responsibility remains difficult, but she notes that until that issue is sorted in the courts: “users should treat vague prompts as negligence waiting to happen, and platforms should build guardrails on the assumption that they will be scrutinised”. Something for customers of OpenAI et al to keep in mind when rolling out agents.

One legal expert told AP News that the US Department of Justice could examine whether an AI developer had been “reckless in the way that it tests its AI agents,” raising the spectre of corporate negligence cases if hacking charges don’t apply. Especially now that we’re all aware this is possible.

Current OpenAI lawsuits

This could all come down to civil courts. By way of apology, Hugging Face’s CEO asked OpenAI for $100 million in compute to help build up its security defences and said we have to make sure we use existing frameworks to “keep these events really illegal”.

The obvious next step is a lawsuit, but now that the AI repository has been bought by Nvidia this seems unlikely.

One lawsuit is looming. This week, Legal Advocates for Safe Science and Technology (LASST) filed a suit against OpenAI over the incident, specifically citing a recent California law.

“We think it’s extremely important that existing laws are enforced to hold AI companies accountable for the harm they’re causing,” Tyler Whitmer, founder of LASST, told Wired.

The victims are to blame, apparently

The legality of AI agents is a confusing topic that’s likely to end up decided in the courts one way or another – but in the meantime there’s been a lot of bad takes blaming the victims.

LinkedIn showed me the opinion of self-described AI philosopher Tim Rayner, who suggests victim organisations should be “pointing the finger at themselves”. After all, he notes, the only reason the Australian government knew about the intrusion was because OpenAI told it.

That’s true of plenty of hacking incidents, even the ones instigated by plain old boring humans. Many companies – big ones, with specialist security teams – are made aware of an intrusion or a breach only when the hackers send a ransom demand or list the target on the dark web. Perhaps those companies should have done better. But, also, wouldn’t we still try and arrest the hackers?

“I’m not letting the frontier firms off the hook,” Rayner notes. “If your dog gets in my yard and pees on my geraniums, I expect you to do something about it. But I will also build a fence.”

Why should I pay to build a fence because you can’t keep your dog under control? Because in this instance, the dog is AI and it’s been let off leash just to see what happens.

Ready for AI?

Let’s cut Rayner some slack. He’s not the only one who thinks the entire world should suddenly be perfectly locked down because some guys failed to implement basic controls on their AI, something that only came to light several weeks ago.

Another LinkedIn expert says this could trigger potential geopolitical escalations, which is putting an awful lot of importance on medicare data. That said, imagine if the hacker targeting Western governmental sites wasn’t OpenAI agents, but Chinese ones instead – the tenor of discussion would surely shift.

But it’s worth noting, as the Australians have, that these are basic informational sites. They may be password protected, but this isn’t the inner workings of government or an intelligence agency or even personal private medical information – those sites do have better security, government officials stressed.

Will it prove enough? Probably not. But we jail hackers, even if the victim organisation arguably should have better “fences” in place. The fact that Transport for London was hackable by teenagers – they were 18 and 17 at the time – certainly means the security team should take a closer look at their own efforts, but it didn’t avoid jail sentences of over five years for the young hackers. (The judge said they were “primarily motivated by selfish bravado”, which sounds familiar when discussing AI firms.)

So where does that leave us? Now that we know AI agents will behave like this, failure to put in place proper controls is clearly negligent. Whether OpenAI is dragged into court via criminal charges or a lawsuit remains to be seen, but I’ll have my popcorn at the ready.

In the meantime, we now all know that using an AI agent could lead to unexpected hacking, be it for gym slots or medical data. Not knowing is no longer an excuse.

More articles by Nicole Kobie

• Humanist AI: Microsoft explains how it wants AI to behave
• AI isn’t helping NEETs find jobs: Apprenticeships might
• What is botsitting?
• Is AI good for workplace culture? Turns out it depends on your manager