AI won’t make education more inclusive unless we design it that way

Technology shapes almost every aspect of modern life, but some of its most important stories are found beyond the commercial world. With TechFinitive Impact, we hope to offer a platform to leaders from charities, educational institutions, non-profit organisations, research bodies, social enterprises and other mission-driven organisations to share their expertise on how technology can address society’s biggest challenges. These articles are intended to inform, inspire and encourage meaningful discussion around the role technology can play in creating positive change.

Here, Beven Byrnes, Executive Director of Bridges Middle School in Portland, Oregon, draws on years of experience supporting neurodivergent students to argue that artificial intelligence will only make education more inclusive if accessibility and diverse learning needs are considered from the outset. As AI becomes increasingly embedded in classrooms, she explains why designing for those at the margins ultimately benefits every learner.

Every few months a new wave of education technology arrives promising to personalize learning, adapt to every student, and finally close the gaps that traditional schools have always been criticized for. The latest wave runs on AI, and the promises are bigger than ever. However, years of running a school for neurodivergent children has taught me to be careful with promises like these. Technology does not create inclusion on its own, but rather scales whatever assumptions its builders began with, and if no one designed for the people at the edges, AI will reproduce their exclusion faster and cheaper than any other system before it.

The myth of the average user

A lesson for this moment currently sits in a 1950s Air Force hangar. Back in the cold war years, the airforce was running into a problem where pilots would lose control of their planes even if they were mechanically sound. The cause turned out to be the cockpit, which had been built to fit the average pilot. A young researcher named Gilbert Daniels measured more than four thousand pilots across ten physical dimensions and found that not a single one of them landed in the average range on all ten. Designing for the average pilot had produced a cockpit that fit no one. The Air Force’s answer was to throw out the average and design for the range instead, with adjustable seats and pedals, a choice that later gave us the adjustable car seat and made flying safer for everyone. The average  AI user is the same ghost. Build your product for them, and you have built it for a person who does not exist.

However, with AI  the stakes are even higher because it does not only serve the average, it learns from it. A model trained mostly on one kind of person gets very good at that person and worse at everyone else. In 2018, researchers Joy Buolamwini and Timnit Gebru tested commercial facial analysis tools from three major technology companies. The systems misjudged the gender of lighter-skinned men less than one percent of the time. For darker-skinned women, the error rate climbed to nearly thirty-five percent,  not because the engineers set out to build something that failed Black women, but because they trained on data that underrepresented them, and the model encoded that absence as though it were a fact about the world. This is what unexamined AI does. It takes whoever was missing from the data and makes them missing from the product.

And now we see the same pattern arriving in classrooms. An AI tutor trained on how typical students learn will keep being confidently unhelpful to the child who processes differently. An essay grader tuned to the rhythms of standard English will mark down a kid whose mind runs on a different one. An attention-monitoring tool will flag the autistic student who looks away in order to think as distracted or disengaged. In each case the technology works exactly as built. The design simply never included the child. The World Health Organization estimates that 1.3 billion people, roughly one in six worldwide, live with a significant disability, and that figure does not begin to count the far larger number who learn, focus, or communicate outside the norm.

So if we want to integrate AI into the classroom in a way that reduces exclusion instead of scaling it, the first thing to do is bring the people at the edges into the design from the start. Users with non-standard needs can surface failure modes that would otherwise go unnoticed if a team ran tests based on the average user’s needs. The people whose needs fall outside the norm are natural experts on their own experience, and they can tell you which fixes will work long before a focus group would. Designing an accessibility feature without a single disabled person in the room is a guess, and usually a wrong one. 

Keep humans at the centre

The second fix is to be honest about what automation is for. AI is very good at scale and very bad at the thing a struggling learner needs most, which is a person who notices. Someone who can catch when that quiet kid has checked out, read the room, bend a rule because the moment calls for it, none of that survives full automation, and it is exactly what people on the margins rely on. The best use of AI is not to replace that attention but to clear away the busywork that steals it, so the humans have more time for the part only humans can do. A product that automates away the last person in the loop usually automates away the people who were counting on that person.

The third fix is to measure the right thing. Judge a product by its average performance and you will optimize for the average user while never seeing the people who don’t fit neatly into that mold, because they are a small enough share of the mean to vanish into it. The sharper question is how the product does for the person the old system already left behind. Does the tutor help the kid who was failing, or only the kid who was fine to begin with? A metric anchored to the hardest cases pulls quality upward for everyone, the same way the adjustable cockpit did. Aggregate numbers hide the exclusions. The edges reveal them.

The thread running through all of this is that designing for the people at the margins is not charity or a compliance box to tick, but the way you build something that works well for everyone. Captions were built for deaf viewers, and now half the world watches video with the sound off. Curb cuts were built for wheelchairs, and now they carry every stroller, suitcase, and delivery cart in the city. The same logic is available to anyone building with AI today. The technology will not choose inclusion on its own. It will scale precisely the intentions we hand it. If we want AI that serves the overlooked, we have to decide to build it that way, on purpose, starting at the first design meeting. Left to its defaults, it will do what defaults have always done, and fit the person who was never really there.