Trending Articles
Trending Topics
Trending
In the boardrooms of Silicon Valley, AI ethics debates usually revolve around copyright infringement, deepfakes, surveillance and data scraping.
However, thousands of miles away, the ultimate consequence of possible AI algorithmic error is being tallied in civilian lives.
On 28 February, a missile struck the Shajareh Tayyebeh primary school in the Iranian town of Minab. As survivors sheltered, a second missile hit. These combined ‘double-tap’ attacks killed around 175 people. Some were teachers and parents, but most were young girls.
Whilst families grieve the loss of life, questions must be asked about this war’s high civilian losses. Are innocent people dying because of poor military tactics or an over-reliance on flawed AI systems?
The US has confirmed the use of sophisticated AI tools in its war against Iran. Admiral Brad Cooper, the head of CENTCOM (US Central Command) said in a video message:
“Our war fighters are leveraging a variety of advanced AI tools. These systems help us sift through vast amounts of data in seconds so our leaders can cut through the noise and make smarter decisions faster than the enemy can react.”
Cooper was also keen to emphasise that the decision to act was approved by a human.
“Humans will always make final decisions on what to shoot and what not to shoot and when to shoot.”
Cooper’s pride that the US’s military AI deployment can work at such speeds that it can compress the kill chain is worrying. Anyone who has used AI is aware of its tendency to hallucinate, err and lie. The humans at the end of this process, sifting through rapid data requests, are susceptible to making mistakes. We know this because it has happened before.
A landmark investigation by +972 Magazine revealed that an Israeli AI system called Lavender was used during the Gaza conflict. It generated kill lists of thousands of suspected targets.
It was known that Lavender’s error rate was around 10%. However, the rapidity of the information flow meant that the human overseer could only spend an average of 20 seconds reviewing data before making a life or death decision.
Sadly, humans are prone to a phenomenon known as ‘automation bias’. It’s the psychological trap where a human defaults to trusting a machine’s output. It’s the reason HGVs get wedged in country lanes because the Sat Nav said it was okay and the driver believed a box of electronics rather than their own eyes.
Under high-stress, high-volume conditions, a human operator is often stripped of the ability to exercise meaningful judgement. The consequences of making a mistake – or many mistakes – during warfare are tragically obvious.
In other words, Admiral Brad Cooper’s assurance that humans decide when to pull the trigger is a convenient illusion. There should be genuine concern that when a data reviewer is asked to vet hundreds of AI-generated targets an hour, they are not simply clicking ‘approve’ to satisfy an algorithm.
Questions are rightly being asked about the sites being targeted in Iran. In a letter to US Defense Secretary Pete Hegseth, over 120 Democratic members of Congress have asked why the US and Israel have targeted schools, hospitals, gymnasiums and other public spaces.
The Iranian Red Crescent Society, part of the world’s largest humanitarian network, reported that the ongoing bombardment campaign has damaged nearly 20,000 civilian buildings and 77 healthcare facilities in Iran.
In addition, the Democrats have also asked the following question about the use of AI in
Operation Epic Fury:
What is the role of artificial intelligence, if any, in selecting targets, assessing intelligence, and making legal determinations during Operation Epic Fury? If artificial intelligence is used, is it subject to human review and at what point? Was artificial intelligence, including the use of Maven Smart System, used to identify the Shajareh Tayyebeh school as a target? If so, did a human verify the accuracy of this target?
In simpler terms, the question is: Why was the Shajareh Tayyebeh school targeted?
A report in The Guardian holds a possible answer. The Shajareh Tayyebeh school was close to buildings used by the Islamic Revolutionary Guard Corps (IRGC). The building was once part of a larger IRGC complex but it was walled off at least nine years ago.
Investigations by Al Jazeera also reveal that a medical clinic on the IRCG complex, only opened in January2025, was not targeted.
So how did the US military hit a school but not a clinic on the same site? How old is the data used to train the AI? What is the risk that further atrocities could be caused by screen-fatigued humans green-lighting AI targets generated from obsolete data?
Humanity must look to history and learn.
Following the horrors of the First World War, the international community established the 1925 Geneva Protocol to ban the use of chemical and biological weapons.
Academics and research experts are proposing that the use of AI in combat demands the same diplomatic urgency and consideration.
TechFinitive has already reported on the war of words between Anthropic and Donald Trump. As a result of Pete Hegseth labelling Anthropic as a supply-chain risk, various tech companies protested that levelling such a designation could threaten their products and services from being used by the military. It’s impossible to ignore the cold-heartedness of this positioning. I wonder if those burying the dead will spare a moment to think about the bottom lines of the AI companies involved in the extermination of their loved ones.
As technologists, we know that this early evolutionary stage of AI is capable of great things. Yet we can’t overlook its ability to regurgitate falsehoods and re-enforce biases of its training data. Humans then compound the problem by accepting the error. Therefore, should we rely on the flimsy reassurances of a human-in-the-loop acting as a safeguard?
If Silicon Valley truly wants to build an AI future, it must demand a legal framework that prevents its code from becoming this century’s weapon of mass destruction.
As buyers and users of AI, we must confront a profound moral dilemma: Do the ethical implications of using a specific AI vendor change when that same technology is deployed in an active war zone?
