Trending Topics

New technology, old tricks… AI hasn’t changed the security game that much
Companies are making the same security mistakes with AI that were made in the 1990s – opening access without thought for the consequences.
Speakers at the [un]prompted AI security conference, set up by Gadi Evron (who we interviewed last month) and the legendary Heather Atkins, Google’s VP of Security Engineering, said repeatedly that the industry was making the same mistakes with access to critical systems that were made in older networking days.
“Are the mitigations turned on in the system is Address Space Layout Randomization (ASLR) enabled?” asked Tim de Waal, a technical staffer at OpenAI.
“Do you have mandatory access control? Is your service running as root, which it shouldn’t be? All of these things are part of an exploitation chain that needs to be figured out.”
The bottom line is that AI has been allowed access to systems and credentials that it should have. It’s a similar situation to when Wi-Fi first became popular people would set up access points that weren’t locked down but were convenient, leaving companies open to easy hacking cases.
New techniques, old problems
In terms of identifying AI attacks some old techniques still work. Back in the day viruses were identified by heuristics, tracking behavior, and methods to send up red flags. The same holds true for AI attacks,
“AI attacking agents have some sort of an accent,” Sergej Epp, creator of the Zero Day Clock, and advisor to Twine Security told the conference. He referenced the biblical term shibboleth, a call sign used to identify allies and enemies by showing the right characteristics. AI attacks, just as with older virus attacks, had their own identifiers, he said.
“We can ask first of all, who are you? And there’s already a lot of shibboleth types of you know probing we can do there. The second one is the accent. Try to review how it’s writing text, how it’s speaking, and the third one is behavior. With Shibboleth, we are trying really to provoke AI agents to give us more information, so we can fingerprint all the aspects I’ve been talking about.”
Fingerprinting such attacks were going to be crucial for future security, he argued. There’s nothing magical about AI in some regards, it’s just a question about how to identify agentic attacks by their footprint.
CISA is on the game, sort of
While the US government hasn’t been the best of late at combatting attacks, its Cybersecurity and Infrastructure Security Agency (CISA) is on the game when it comes to AI threats Kevin Donovan, the agency’s deputy associate director for vulnerability management argues.
“We saw the opportunity for these frontier models and this technology for defenders, but we knew threat actors and our adversaries were going to have them as well,” he said. “So we had to work and change and adapt and scale how we did things at CISA.”
The agency has deployed large-language models and agents to block such attacks and mitigate them but the future is uncertain. “We don’t have any idea where this is all going,” he admitted.
It’s tricky because so much of government software is open, but by harnessing AI agents within a rigid framework the agency is looking to find new ways to block AI attacks and make sure that intruders are stopped.
Part of this is CISA’s Gold Eagle program, enacted via President Trump’s Executive Order in June. This aims to spread the word about vulnerabilities to government IT admins who can fix serious problems that are being exploited – although given the pressures they are under this may be wishful thinking.
“We want to make sure we are good partners to maintainers, whether they are regular open source or our federal agency partners, and we want them to send them quality, machine-readable reports so that they can validate, deduplicate, and prioritize on their own,” he said. “We need to get to a place where any system doing this type of vulnerability research outputs a truly high-quality report each and every time.”
The proof is in the eating of the pudding
Ultimately, the defence against AI attacks is going to be down to protectors using the technology faster than attackers get new ideas.
This is essentially problematic, since the number of attackers is larger than the defenders. AI is also making hacking much simpler for poorly defended systems.
When hacking kits first started to be sold online we saw an explosion of script kiddies, who could buy an automated system to generate ransomware or other criminal tools and spam them out for profit. AI is only going to make that worse, opening up the hellscape of random folks deploying the technology.
If the [un]prompted conference showed anything it was that the industry is saddling up to deal with this threat. If it will be successful depends on smart, ethical minds and the urge to make sure that we do the best we can.
