The cost of AI agents is spiralling. Here’s how to approach LLM cost optimisation

This article is part of our Opinions section, where we invite industry professionals to share their views on the most pressing technology questions of our time.

Google is planning to open a data centre in Essex that’s projected to emit 570,000 tonnes of CO2 a year. According to advocacy group Foxglove, that’s several times higher than an international airport. And that’s just one facility.

AI infrastructure costs are spiralling. And every enterprise running AI agents picks up a slice of that bill. The question worth asking before you sign up for any AI tool is: Do you know what you’re paying for at a granular level?

The unit of cost most businesses don’t understand

The thing that AI users often forget is that tokens aren’t free. Every time a large language model processes something, the count goes up. Retries and failed loops count too, which means the meter keeps running whether or not anything useful gets done. That’s a big reason AI infrastructure is projected to consume up to 1% of global electricity demand by 2030, according to research by Cornell University.

What I rarely see mentioned is that large foundation models charge significantly more per token than smaller, task-specific ones. Recent research found that small language models matched the performance of larger models on some business tasks despite being up to 300 times smaller.

But organisations are being pushed to pay premium rates to route work through large, general-purpose models that are often overkill for the task at hand. Hyperscalers and foundation model providers need you to burn through tokens to make a return on everything they’ve invested in infrastructure. Their business model depends on you routing as much as possible through their largest, most expensive models.

Kevin McGrath, CEO of AI startup Meibel, put it plainly: “The biggest problem that we’re working with in AI right now is that companies are routing everything through large language models without stopping to ask whether the task warrants it.” He urges companies to be more deliberate when deciding which tasks are best suited for AI agents.

What all-in-one agents do to that cost

Most enterprises default to a single AI system handling everything end-to-end. But when that system gets stuck, it loops. Each loop burns more tokens, and since these systems are opaque by design, there’s no easy way to see where the waste is happening before the bill arrives.

Gartner predicts more than 40% of agentic AI projects will be cancelled by the end of 2027, partly due to the cost of AI agents. And overreliance on large language models with a tendency to get stuck in expensive loops only raises those costs.

Organisations are busy splurging on the “best” AI models in a rush to have an edge on the competition, but they’re not realising that the amount of their budget going on AI tokens every quarter goes up and up.

With AI, it seems that organisations are prepared to open the chequebook for a system that retries without limit and charges for every failed attempt. And with no monitoring to show where the waste is happening and no way to intervene, you have no idea how much of your monthly bill is going towards paying for pointless loops. 

Smaller agents, better results

The alternative to a cookie-cutter approach to AI is to build a swarm of micro-agents. These are small, task-specific components with a defined role and a measurable output.

A 2026 benchmarking study found that smaller models consistently outperformed larger ones on economic efficiency, including time-to-ROI. Smaller components also mean shorter feedback loops. You can measure what a single agent costs to run and optimise it without touching anything else.

When an end-to-end system breaks, you’ve got to pull the whole thing apart to find out where. When a micro-agent fails, the problem is contained and easy to spot. You fix one component and move on.

What makes micro-agents easy to fix also makes them easier to measure. ROI on your AI spend comes from measuring token consumption, latency, failure rates and task completion costs at the agent level. Once those metrics are visible, teams can decide where a large model is worth it and where a smaller one will do.

But that’s rarely the setup big vendors will sell you on. In fact, they actually benefit from you skipping the architecture questions, so you end up using as many tokens as possible (known as tokenmaxxing) to get the job done. The result is you’re often sold a black box you can’t manage yourself.

The core principle behind LLM cost optimisation is matching model size to task complexity, rather than defaulting to the most powerful option available.

The operations problem hiding inside the AI problem

Most organisations will scrutinise every line of their budget. From headcount to software licences, office costs, etc., all of it gets picked apart in quarterly reviews. AI spend is different. It’s abstract, it’s technical, and most finance teams don’t yet have the language to interrogate it properly. They just know they ‘should’ invest. So it goes unchallenged.

All that waste soon adds up. Nobody is asking what a failed loop costs per hour, or what percentage of the monthly bill is going on retries that produced nothing. And without monitoring in place that makes those numbers transparent, there’s no way to find out the cost until you get the invoice through.

The shiny capability claims that vendors make are easy to get distracted by. But the more useful questions are often operational ones. What does the monitoring actually show? Where are the loops happening? Which agents are running up costs without delivering output? If your team can’t answer those, the whole thing needs a rethink.

Agentic AI ROI is a measurement problem before it’s anything else. Get visibility into what each component costs to run, and the decisions about where to cut and where to invest will become much clearer.

More from our Opinions section:

• Lessons from the boardroom: How CISOs should approach compliance
• What the EU age verification app controversy teaches us about digital identity and trust
• Fintech’s petri dish: what the rest of the world can learn from Latin America
• Why the next wave of AI is about trusted action, not text generation