Computer scientist, Professor Harold Thimbleby, shares the story of the Post Office scandal and his involvement in campaigning for a better understanding of technology in the law. Through his insights, we see the need for SHAPE and STEM to work together, balancing our humanity with our desire for new and improved technologies.
I was brought up not to make a drama out of a crisis. But, as the 2024 ITV series Mr Bates vs the Post Office shows, sometimes making a drama out of a crisis is exactly the right thing to do. The drama converted a boringly detailed technical story of computer problems into a very compelling human story. The tragic story had been simmering for over two decades but hardly getting any attention. That has now changed. and the thousands of victims (plus spouses and children) are finally getting the attention they deserve.
While SHAPE helps us make sense of the human world, to value and express the complexity of life and culture, the story in this blog is one about how technology messed things up and how the human world responded inadequately — until a tear-jerking human TV docudrama made sense of it.
The Post Office scandal started around 1999. In the period to 2015 nearly 1,000 subpostmasters were convicted of theft, fraud, and false accounting. In addition, nearly 3,000 subpostmasters were affected but did not receive criminal convictions — many would have settled out of court, paying back money they were accused of stealing to avoid prosecution.
The trouble was that the Horizon computer system that runs the Post Office accounting had flaws, bugs in the system, and also allowed remote access to change figures. The Post Office, however, behaved as if the Horizon system was infallible.
When the Post Office took subpostmasters to court, it relied on the Common Law presumption that computer evidence is correct (see Harold’s Blog 1 for an NHS example of this presumption). Essentially the Post Office could say: we have this overwhelming computer evidence against you. If you plead not guilty, things will go badly for you, so why not plead guilty?
Some subpostmasters discovered that when they pleaded guilty to the lesser charge of false accounting, their well-intentioned attempts to repay the money that the Post Office claimed they had stolen was taken as confirming the computer evidence of their guilt. Some subpostmasters committed suicide. Many ended up in prison, some with broken marriages. Declared bankrupt, many lost their homes. It had a devastating impact on their children too.
We now know this was a huge pernicious scandal.
In 2019, Mr Justice Fraser approved a settlement for more than 550 subpostmasters, and in 2021 the Court of Appeal quashed 39 convictions. The Court of Appeal said the Post Office’s prosecutions were an affront to the conscience of the court.
The Post Office scandal is without doubt the largest British miscarriage of justice ever. Horrible stories continue coming to light in the Post Office Inquiry. Jason Beer KC, counsel to the Inquiry, called the serious failures by the Post Office a scandal within a scandal within a scandal.
It is crucial to restore justice, but the prosecutions and court cases are just steps in a long process of deep failures and misunderstandings.
So, how did we end up here? Let’s go back to the beginning.
Computers are exciting and offer all sorts of amazing solutions to everyday problems. Lots of people are excited to see how AI will solve our future problems. Not only can computers offer amazing solutions, but these solutions are apparently easy: it’s often said that even children can program — so programming must be easy. It follows that anyone caught up in this sort of thinking, like the Post Office, will think they only need to get a new up-to-date IT system, and all their accounting systems will work well without any further thought.
Unfortunately, this line of thinking is magical, and not realistic. It is like saying it’s amazing that children can do brain surgery, while completely ignoring that you wouldn’t want children to actually do brain surgery! We all know that bad brain surgery is very easy. Real, safe brain surgery, though, is extremely challenging and should be left to highly-qualified professionals. It’s obvious with brain surgery — if you see a toddler wearing a stethoscope you can encourage them to become a brain surgeon in the future, but you don’t want them to start brain surgery tomorrow!
Somehow, computer excitement doesn’t benefit from this sort of clear thinking.
The consequence is that we allow people who may be no more qualified than children to program complex IT systems like Horizon. But if we tried to tighten up the regulations so that only properly qualified competent people could program, manufacturers would start talking about “regulatory burden” — they’d say they wouldn’t be able to innovate if such regulations were imposed on them.
No doubt the Post Office thought Horizon would be magic, fast, and efficient, and would sort out their business, but it was not well-designed. It turned out to have serious problems.
To me, as an engineer, each Horizon problem feels a bit like seeing a bald tyre on a car. A car with a bald tyre isn’t safe to drive. The car might have nice bits, but if a tyre is bald, the car is unsafe to drive — there’s not much to debate. Moreover, if you bought a car that came home with a bald tyre, you would also worry what else might be wrong with it!
The code in Horizon that’s been disclosed to the official Inquiry has code that looks like bald tyres — clear warnings Horizon is poorly designed. Click to see some Horizon code problems.
This is the reality where STEM meets SHAPE. The STEM is the Horizon code’s engineering, and the SHAPE is that the code is humanly hard to understand and error-prone. It’s also a SHAPE issue how this happened and how we might avoid such problems in the future. Horizon has had an awful impact on the people who tried to use it, which has also created a SHAPE-shaped legal minefield.
While children can program, they can’t become doctors without training and passing exams. In 1858 the Government passed the Medical Act to register qualified doctors. The Government wanted to end the scourge of quack doctors, and the Act required doctors to be qualified and registered in order to practice. We now think having qualified, competent doctors is obvious. Indeed, we also have legal requirements on electricians, pharmacists, drivers, caterers, lawyers, and many more professions. Yet, we put no requirements on software developers, such as those who developed the Post Office’s Horizon.
Ironically, to press a button on a medical device like a ventilator, you need to be qualified with many years of education and have passed lots of medical exams. But to write the program to make the ventilator work, to decide exactly what that button does, you need no experience or qualifications whatsoever. Moreover, if something goes wrong, the doctor, not the programmer is blamed.
Most of the time computers are amazing, and why should we hold up progress with tighter regulation? But occasionally something as terrible as Mr Bates vs The Post Office wakes us up. This is the thin end of a serious and widespread problem we urgently need to fix.
The details need working out, of course, but having no registry of competent programmers, and no effective qualifications to easily — and legally — distinguish cowboys from professionals is a recipe to repeat disasters like the Horizon scandal both on the small scale, as well as on the large scale of the national IT projects we all depend on.
Further reading
The Post Office Horizon IT Inquiry website - https://www.postofficehorizoninquiry.org.uk
N Wallis, The Great Post Office Scandal, Bath Publishing, 2021.
About the author
Prof Harold Thimbleby
I am a professor of computer science, but I’ve moved into legal activism, via this story in the NHS and the Post Office. I am now working with barristers, and I am proud my work in the NHS has led to me being elected honorary fellow of the Royal College of Physicians and of the Royal College of Physicians Edinburgh. I’ve published 345 refereed papers and been invited to give over 300 conference keynotes and over 600 presentations and workshops around the world.