8 out of 10 cats...believe statistical modelling should be treated with caution.

Over the last months throughout the pandemic we have heard a lot about statistics and statistical modelling. Statistical results are often held up to be absolute and are often reported in the media without the levels of uncertainty that naturally accompany them. 

A good example of this was the statistic being banded around that bald men were more likely to become hospitalised with Covid-19 than men with a full head of hair. This would have caused a degree of alarm for the follicly challenged members of the population. It would suggest that there is a causal link between hairloss and Covid-19. Now perhaps there is I have not accessed the original information from where this came perhaps to do with male hormones?. However it is more likely that this is just a correlation and that the cause is age. Older men are more likely to have complications with the virus and statistically more older men are bald compared to their younger counterparts. Notice that I used the term statistically in my argument against statistics!  Interesting. 

The point, the statistic still tells you something it just needs to be treated with caution. 

Consider a game of dice where the aim of the game is to throw two dice and get a total of 7. I throw the dice, what are my chances of getting a 7? The probability is 16.67% (6 combinations out of a possible 36). The chance of throwing a 2 is just 2.78%. Now try throwing two dice 36 times. Do you throw a 7 six times? Do you throw a double 1 once?  Probably, but not definitely. The more you throw, the bigger the sample size and the more likely your actual results fit the statistical model. 

The important difference between a correlation and a causal relationship is demonstrated when looking at data surrounding the occurrence of people seeking medical support for hay-fever in the UK and the number of ice cream sales from ice cream vans. Both sets of data show spikes in early April and again in June, July and August. If one was careless with this data one might suggest that having hay-fever increased the desire for an ice cream. Or that in some way buying an ice cream brought on the symptoms of watering eyes and sneezing etc. Of course this is ridiculous. It just so happens that at these times of the year the release of pollen accompanies nice weather when children are in school holidays. This causes both these factors at the same time. 

Opinion polls are another example of statistical modelling that recently have shown that this uncertainty can make predicting events very difficult especially when these events are full of complex variables. How many times has a political election gone against the information provided by the polls?  I’m sure someone can provide a statistic on that. 

Used properly statistics are important. They help us make judgements that keep us alive. They are part of the decisions we make all the time when we consider the risk of a particular action. It’s just that we don’t actively ‘do the maths’. Statistics are everywhere. They can be used to predict and can be used very successfully. But predictions are not always correct they have a degree of uncertainty and this is where the flaw lies. 

I believe that the big issue with statistics is that it generalises and although tries to account for outliers these are difficult to deal with when trying to model complex human behaviours. The outliers are people not just numbers. 

So, it was inevitable, when applying a statistical model to exam grading in the UK the model would be flawed. The data used would not be able to take into account all factors about individual students and their schools. 

Remember the outliers are real people with their futures ahead of them.

The outliers are students who have seen their results drop by 2, 3 or more grades

The outliers are schools that have cohorts that would have performed much better than in previous years. 

The statistical process has apparently failed these individuals by applying generalised criteria. 

The use of this mystical statistical algorithm has led to real sadness and anxiety for a large number of our 18 year olds that will be hard to rectify. A group of young people who have already been significantly compromised by the pandemic. 

And it’s set to repeat next week. 

As Mark Twain wrote, ‘facts are stubborn things but statistics are pliable,’. Perhaps the government will learn and allow more pliability in the statistical modelling?  I hope so for the sakes of all those young people.