Making mountains out of scientific mole hills

Richard Horton, Editor of the Lancet speaking at the Global Health Metrics & Evaluation conference 2011. (Photo credit: Vimeo.)

The controversial Lancet editor, Richard Horton, has produced an opinion piece which some are interpreting as an attack on medical science, if not science in general. His article, What is medicine’s 5 sigma?, is being touted by websites like Collective Evolution as authoritative “evidence,” or justification, for their attempts to manufacture doubt on scientific issues (see Editor In Chief Of World’s Best Known Medical Journal: Half Of All The Literature Is False). And, of course, these messages are spread far and wide in social media like Facebook and Twitter by activists for various anti-science causes.

Horton, of course, exaggerates. He calls a spade a shovel – or even a giant earth moving machine. That is music to the ears of propagandists and manufacturers of doubt in the anti-vaccination, anti-fluoridation or even climate change denial movements. But, putting aside the damage such exaggeration causes for a moment, I do sympathise with some of Horton’s claims.

Provisional nature fo scientific knowledge

Horton begins with the statement “A lot of what is published is incorrect.”

While he appears to think this claim has shock value it is hardly news to scientific researchers. By its very nature, scientific knowledge is both provisional and incomplete. In the real world, no scientific idea or theory can accord completely with the true objective reality. We are always dealing with just a part of that reality. And our theories are always being replaced by new, updated and more complete theories which give better explanations of reality.  Let’s be clear, though, in most cases this is not a simple mechanical replacement but usually a modification to, or improvement of, existing theories.

So, yes, published science is “incorrect” in that it is always incomplete and provisional. It is always open to sceptical consideration and improvement – or even rejection.

However, Horton has really brought in the earth-moving machinery when he advances “the idea that something has gone fundamentally wrong with one of our greatest human creations.” 

Modern science is indeed “one of our greatest human creations” – despite the fact that it is by its very nature incomplete and imperfect.  Despite all the human problems influencing science it has still enabled us to solve many problems, to provide a more comfortable and safer existence for much of the world’s population and provided us with amazing technology (which critics of science enthusiastically use without being aware of the irony involved).

Sure, we still have many pressing problems to solve but no-one can seriously believe that science cannot contribute to the solution of these problems. In the end, despite all the human frailties inherent in such human endeavours, no other approach to obtaining knowledge and solving problems can seriously compete with science.

Some real problems

Horton gets specific:

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”

He also refers to those papers relying on correlations without showing causation, saying:

“Our love of “significance” pollutes the literature with many a statistical fairy-tale.”

But these problems are, in essence, not new and do not justify his purple prose about “a turn towards darkness.” That prose is rather hypocritical considering his own role in the publication of, and resistance to the retraction of, Andrew Wakefield’s article claiming a causal link between standard childhood vaccinations (measles, mumps and rubella) and autism (see  Why Is Richard Horton Still The Editor Of The Lancet?).

The fact is that poor quality research does get published – even by reputable journals like The Lancet (see for example Repeating bad science on fluoride). Poor quality research is not always knocked back, or improved, by the peer review processes journals use. These peer review processes themselves can be very flawed and even suffer from cronyism (see Poor peer review – and its consequences) and Poor peer-review – a case study) .

Many studies are poorly designed, report tiny effects and use small sample sizes. Many rely on statistically significant correlations which may be meaningless without any evidence for causation. Peer reviewers and journal editors, if they are actually conscientiously doing their jobs, are forced to make judgment calls. There is an argument for sometimes getting such studies into the literature where they can be critically examined and discussed. (Horton himself justified his decision to publish Andrew Wakefield’s article, which he acknowledged was an inferior study by claiming it would generate debate on the autism/vaccine issue).

But this backfires when uncritical journalists report the studies as scientifically credible, even gospel truth, when they are far from it. This is compounded by propagandists for activist groups who, confirmation bias in full flight, latch on to such studies to give “scientific authenticity” to their unscientific claims. And then promote them far and wide.

Recently we saw this with published papers claiming a link between fluoridation and thyroid problems (Peckham et al., 2015 – see Paper claiming water fluoridation linked to hypothyroidism slammed by experts) and ADHD (Malin and Till, 2015 – see ADHD linked to elevation not fluoridation). These papers were prime examples of Horton’s “statistical fairy tales.”

Reader beware

So, I am repeating a theme I often promote here. When it comes to the scientific literature it really is a matter of “reader beware.”

The reader must approach this literature carefully – intelligently and critically. The reader has the task of identifying “statistical fairy tales,” poor study designs and problems of tiny effects or small sample sizes. If the reader does not have the ability to do so they need to seek the opinions of qualified experts – and I don’t mean the self-appointed “world experts” leading activists groups or the google-informed commenters who seem to dominate social media and the internet.

After all, it is the real expert, many of whom are active researchers, who critically assess the scientific literature on a daily basis. And if they are participants in an active scientific community problems of confirmation bias are reduced.

The “reader beware” approach is even more necessary with the “scientific” claims often bandied about in the popular news media – mainstream media and especially the ideologically motivated “alternative” media.

I am an avid reader of the NZ Listener – which I consider a reputable mainstream journal. But every week I am annoyed by the small snippets reporting some new scientific claim (usually related to popular health issues) relying on individual scientific papers which I would place in Horton’s group of “statistical fairy tales.” I hope most readers are intelligent enough to seek further advice before taking such reports seriously.

But this annoyance is minor compared with what I feel about the rubbish I see daily on the internet daily. Ideologically motivated activists dominate social media here. They opportunistically link to such media reports, and even the original scientific papers, to give “scientific justification,” and confirmation bias for their unscientific messages.

When it comes to the internet one cannot repeat often enough – reader beware.

Similar articles