I came across a paper (of sorts) recently, written by three editors of the American Statistical Association that I thought was worth writing a short post about.
Here’s a link to that paper.
It’s not the shortest read, but the TL;DR gist of it is that they argue it might be time to retire the old p<0.05 ‘rule’ that has been relied upon as the gold-standard litmus test for “true” statistical significance.
It’s something I’ve often thought about in reading countless academic papers and research works over the years.
The authors argue for taking a more holistic approach to research findings, which considers p-values (among other statistical tests) within their context, as opposed to somewhat blindly relying on the p<0.05 rule, which is just an entirely arbitrary ‘line in the sand’.
I’m sympathetic to this view, as I think there have been many research findings put forth as ‘fact’ simply because they passed a p<0.05 significance test, when in reality, they’re probably mostly just picking up on noise.
The world is a messy, complicated place, and the ‘truth’ is often fleeting.