AI will get things wrong.
When it does, will anyone spot it and know how to fix it?
Companies are giving work to AI. Writing, coding, analysing data, and answering customers.
Every time AI does a job for us, we lose the chance to practice doing it ourselves.
What pilots learned the hard way
Modern airliners use autopilot for much of a flight. Automation improves safety, precision and reduces workload.
But it also brings a risk. If pilots rely too heavily on automation, their manual flying skills can fade.
In 2009, Air France Flight 447 crashed after its airspeed sensors became temporarily obstructed and the autopilot disconnected. The crew suddenly had to take manual control with confusing information. The aircraft entered a stall and did not recover. All 228 people on board died.
Crashes like Flight 447 led regulators to put more emphasis on practising manual flying and preparing pilots to take over when automation fails.
The lesson is simple. Make sure someone can still fly the plane.
Losing expertise
As experienced staff leave and junior employees increasingly rely on AI, we risk losing more than knowledge. We risk losing the way people become experts in the first place.
In their blog post Genie coefficient, Bruce Schneier and Barath Raghavan compare AI to a genie.
A genie can give you exactly what you ask for and still get the outcome completely wrong.
There is a long history of stories warning us to be careful what we wish for.
King Midas wished that everything he touched would turn to gold. At first it seemed like a gift, until food, drink and even the people he cared about turned to gold too.
Thousands of years later, The X-Files used the same idea. In season 7 episode 21, Mulder wishes for ‘peace on earth’. The genie grants it by making everyone else disappear.

Different story. Same warning.
Getting exactly what you ask for is not always the same as getting what you want.
If people lose the knowledge needed to ask good questions, judge the answer and spot what is missing, unexpected results become much harder to catch.
Who will spot the mistakes?
If AI does all the work, who gains the experience to spot mistakes, catch unintended consequences and fix problems?
If people stop doing hard tasks, they may lose the skills to check AI when it gets things wrong.
So the question is not just, ‘What can we give to AI?’ It is, ‘What skills can we never afford to lose?’
