Saturday, August 8

Risky Talk review: How to protect yourself from dodgy statistics

Portrait of woman looking on blue screen lit with binary code

Understanding statistics is a vital tool for life in the 2020s

Stanislaw Pytel/Getty

“I’ve got a lower than average risk of developing coronary heart disease and Alzheimer’s and a higher risk of getting psoriasis and coeliac disease. And I have got absolutely no idea what to do with this information,” says David Spiegelhalter, in , a monthly podcast about statistics that debuts on 2 March.

For Spiegelhalter, an academic who has spent years trying to improve the public understanding and image of statistics, this kind of information illustrates why he has had his work cut out – and why genetic risk is a good topic for the first episode.

“We are all bombarded by misleading anecdotes and biased information every day, whether it’s people trying to sell us stuff or persuade us to vote for them. But to make important everyday decisions, from the personal to the political, we need a clear view of the best evidence about risks and benefits,” he says.

A few years ago, Spiegelhalter sent saliva to a consumer DNA testing company, which resulted in those perplexing findings. We can all pay to have this type of genetic test, but they are also being rolled out on a large scale. The US National Institutes of Health plans to sequence a million people’s genomes and the National Health Service in the UK has a 10-year plan to expand the use of genetic sequencing.

The big question will be knowing what to do with all that data. So far, tests flag a few genetic conditions where you can reduce your risk of them developing by adjusting your lifestyle or through a medical intervention. But in most cases, there is very little you can do.

Saskia Sanderson, a behavioural scientist at University College London and a panellist in this episode of the podcast, says she has found an extra factor. When people are properly informed about the possible outcomes of genetic testing, they tend to be good at opting out of taking a test if they think they won’t be able to handle the news.

However, few doctors are trained to explain the implications of genetic testing, and an NHS source told last year that there were only 200 genetic counsellors in the whole of England.

Flaky immigration stats

In episode 2, moves on to other hot topics: crime, immigration and opinion polls. This time the problem is a shock: it isn’t that there is lots of good data but no one to interpret it, it is more that some of the key stats are, well, flaky.

Take immigration. Spiegelhalter discovers a stat that says immigration has fallen in the UK by 13,000 people, but when he downloads the relevant table from the UK’s Office for National Statistics (ONS), he finds that the margin of error is plus or minus 73,000. “This seems rather large,” he says.

But it gets worse. The way these numbers are generated is even flakier. Ed Humpherson, of numbers watchdog the UK Statistics Authority, says that the immigration estimate primarily comes from an “intention survey”. This is a questionnaire the ONS uses for people entering and leaving the country about the length of their trip, whether they intend to stay long, and so on. No one is obliged to reply.

Millions of people come in and out of the country during the year, but the ONS gets just 5000 responses. Its final estimate is a result of combining this tiny dataset with some other data, such as the number of visas awarded.

The UK Statistics Authority awards a mark of quality for stats, and the authority has recently removed one of its marks from the immigration figures. Immigration is one of the hottest of political potatoes, so regardless of your political leanings, surely getting better numbers should be a no-brainer?

Not so simple. “Statistics aren’t just numbers, they are numbers with a social life,” argues Humpherson. They are sought by people, governments, and corporations all with agendas who want the appearance of cold facts to justify their arguments. Small wonder people feel they can’t trust statistics, or the experts who cite them.

In that light, Spiegelhalter’s attempts to sift the good from the bad and generally improve the dialogue can only be a good thing.