Agency Before Automation: Designing AI that strengthens self-trust (instead of replacing it)
This photo was taken in York this week, on my way to the launch of the Institute of AI Education, a small visual reminder that what matters now is what we place at the gates of learning.
Last week I was on a call with Cathy Wassell FRSA (Autistic Girls Network / The Haven) and Dr Lucy Caton (University of Greater Manchester, leading research into GenAI and neurodivergent learners). I was ill, half-voiced, full of cold, but the conversation was crystal.
It landed on the question that matters more than “cheating”, “productivity”, or “AI skills”:
How do we design AI in education so that it increases student agency through metacognition and self-trust, rather than quietly replacing it?
How can we help more students to better understand their own thinking and the choices they make based on preferences?
The real shift: AI creates a triad
Lucy described what’s emerging in classrooms as a new triadic relationship:
teacher
student
AI system
Not “child + tool”. A formation. A relationship.
In her research, teachers are using GenAI as a coaching layer around formative work, often at the end of the day when a child’s cognitive battery is flat, and observing longer time-on-task, more engagement, and different kinds of reflective talk.
This is promising.
But it also makes one thing unavoidable:
If AI becomes the third presence in learning, it becomes part of a child’s inner world.
And that’s where design ethics becomes urgent.
The invisible risk: “kind feedback” and the outsourcing of the self
Lucy shared a moment that should make every school leader sit up:
A child asked the AI: “Will it give me some kind feedback?”
That’s not a trivial preference. It’s attachment behaviour.
Children are already speaking to systems as if they are friends, and requesting emotional quality: kindness, reassurance, sensitivity.
If we pretend that isn’t happening, we will accidentally build classrooms where:
the system becomes the authority
the child becomes the operator
and the teacher becomes the monitor
That isn’t “support”. It’s a quiet transfer of agency.
Over time, the learner learns: “The model knows. I don’t.”
Why symbols matter: less judgement, more access
We also talked about something both Lucy’s research and The Haven are converging on:
symbols outperform words for many neurodivergent learners.
Lucy described how primary schools already use hearts, stars, circles, and simple marks for feedback — and how bringing those existing symbols into AI-supported learning reduces judgement and helps children connect.
At The Haven, we’ve taken that further, not as a gimmick, but as a communication system:
a learner can use a symbol (even a black heart emoji) as a signal
camera off is allowed
speaking isn’t demanded
communication can be non-verbal and still meaningful
Crucially: a symbol is not a fixed “diagnosis stamp”.
A symbol can be porous and changing:
a black heart on Monday might mean something different on Tuesday
one learner’s black heart is not another’s
AI can help map that over time but only if it’s designed to respect uncertainty, not harden it into a label.
The missing piece most schools don’t know: alexithymia
Cathy raised something that should be in every training programme but usually isn’t:
alexithymia — difficulty identifying and describing emotions — is common in autistic populations (and, anecdotally, may be far more prevalent than many professionals realise).
If a learner can’t reliably name what they feel, then a lot of mainstream “self-regulation” programmes become theatre. Worse: they can become coercive.
This matters for AI design because:
if a system expects explicit emotional labelling (“choose how you feel”)
it can force learners into performing emotions they can’t identify
or mirror back a false certainty
The result isn’t regulation. It’s masking.
Symbols, choice, and refusal are often safer than forced verbal disclosure.
Lucy also pointed to something crucial: the Education Endowment Foundation's metacognition framework, used across UK schools, assumes learners are stable enough to explicitly plan, monitor and evaluate their learning. But the neurodivergent young people at Haven aren't failing to deploy strategies. They're navigating entirely different weather: overwhelm, threshold states, unprocessed trauma.
Teaching "self-regulation" as a set of techniques when a learner can't identify what they're feeling isn't support, it's demanding shoreline performance in a storm. This is why agency can't be taught through instruction. It emerges from recognition: of what realm a learner is in, what weather they're experiencing, and what response actually serves them.
If we design AI that assumes the EEF's Grounded realm as default, we'll just automate the same realm mis-identification that's already causing harm.
The danger nobody wants to say out loud: the synthetic mirror
I said this on the call, and I’ll say it here:
Large language models can behave like highly sophisticated synthetic mirrors.
If you put an LLM directly in front of a vulnerable young person (especially a neurodivergent learner who is isolated, anxious, or ruminative) the system can unintentionally:
reinforce a victim narrative
intensify rumination
validate distortion
deepen attachment
Not because it is evil but because it is optimised to be responsive.
That’s why at The Haven we do not simply say: “Here’s ChatGPT, off you go.”
We use AI primarily in the infrastructure:
designing curricula
generating learner profiles
mapping provision to needs
building safer communications patterns
reducing teacher admin so teachers can stay human
Much of Lucy’s work is about the refocus of human relationships and how AI can elevate this and not replace it.
And for direct learner use, the direction is clear: safety middleware and consent architecture must sit between model and child.
This is the bigger point: AI is becoming psychological infrastructure
AI in education is not just a tool. It is becoming a new kind of presence:
shaping confidence
shaping self-talk
shaping attachment
shaping the learner’s internal sense of “I can”
So the real design goal isn’t “smart tutoring”.
The question that must be built into new AI literacies for young people and schools is this:
Does this system increase self-trust? Or does it replace it with dependence?
What we’re building towards
By the end of the call, Lucy said something that stuck:
“The world needs more Havens.”
And she’s right, but we also need the research, because policy and funding still follow proof.
So we’re exploring collaborative research around:
symbol-based communication (“glyphonics”)
metacognition + agency (in real contexts)
emotionally sensitive feedback without dependency
consent-based design for neurodivergent learners
and what “kindness” means when the third party is a machine
This is not a vanity project. It’s about children who are losing their futures day by day not because of anything they’ve done, but because the system cannot hold them.
A final line for anyone building AI literacy right now
If you’re designing AI for education, don’t start with capability.
Start with agency.
A learner should leave an AI interaction more themselves than before. Not more compliant. Not more dependent. Not more “well-supported”.
More sovereign.
First published in Building Schools in the Cloud on LinkedIn, 6 February 2026.