Citing scientific studies and the arrogance of ignorance

Image credit: Why Citing a Scientific Study Does Not Finish An Argument

One thing that gets me going (as readers here probably know) is the way scientific citations are cherry-picked and dragged in to support biased positions. It’s a common form of confirmation bias in the fluoridation debate. And I get even angrier when the perpetrators of this thoughtless and cherry-picking citations will then claim they “have science on their side.”

Parroting citations and (often unconnected) claims are not what science is about!

So I was very pleased to see this article “Why Citing a Scientific Study Does Not Finish An Argument“ by Jonny Anomaly and Brian Boutwell at Quillette. They point out that throwing down a gauntlet like “actually, studies show . . . “ often kills the discussion:

“It’s hard to know what to say when people cite scientific studies to prove their point. Sometimes we know the study and its relative merits. But most of the time we just don’t know enough to confirm or refute the statement that the study is supposed to support. We are floating in a sea of information, and all we can do is flounder around for the nearest buoy to support a view that’s vaguely related to the conversation.”

I think this is why anti-fluoride propagandists rarely get challenged when they come out with their misinformed claims that fluoridation causes IQ loss, ADHD, hypothyroidism, etc. Their discussion partners are often not familiar enough with the scientific literature to challenge the claims. Of course 99 times out of 100 the propagandist is also completely unfamiliar with the literature and is simply parroting a claim they saw in one of their “nature news” newsletters, or similar. And surely throwing out citations one has never read is a clear example of the arrogance of ignorance.

The fact is: “All of us lack the time to understand more than a small fraction of scientific research.” But when the discussion partner is familiar with the cited studies the propagandist quickly lurches into a Gish Gallop – or deletes the online discussion and bans the person. Those of us who have entered into this debate with the motivation of clarifying the science will know what I am talking about.

We should be wary of arguments relying on citations even in cases where the proponent has read the literature. Citing an individual study is really meaningless:

“Of course, that’s not always how science works, or how knowledge is spread. A single study is rarely anything more than suggestive, and often it takes many replications under a variety of circumstances to provide strong justification for a conclusion. And yet, poorly supported studies often make their way into newspapers and conversations as if they are iron clad truths.”

That’s another thing that angers me – even fairly reputable magazines will report individual studies as if we should take the results as “gospel truth,” without even considering the quality of the research – let alone any supporting research.

The problem of correlations

Often such poorly supported studies rely on correlations – and the way commenters and the media cite such correlations as “evidence” is another bugbear of mine. The authors rip into this problem:

“Correlations are everywhere, and given enough data from enough studies, we will find correlations that are surprising and interesting. But . . .  causation is difficult to infer, and some correlations are flukes that don’t admit of a common cause, or that can’t be consistently replicated.

“We are pattern-seeking creatures, and correlations are patterns that cry out for explanation. But sometimes our political views infect our prior beliefs, and these beliefs lead us to look for patterns until we find them. Given enough tests and time, we will find them.”

I am amazed at how studies relying on the poor use of correlations often make it into scientific journals. I have written about one example in “ADHD linked to elevation not fluoridation.” There is a similar situation for the recent paper of Hirzy et al. (2016) which I discuss in Anti-fluoride authors indulge in data manipulation and statistical porkies. Although I understand that particular paper was rejected by several scientific journals before it ended up in Fluoride which accepts anything that is anti-fluoridation.

Frankly, I think more papers like this should be challenged and that journals have an ethical responsibility to publish critiques of such papers. Unfortunately, I think I am being a bit idealistic here as many editors have their own biases.

When it comes down to it I think even with the scientific literature it is a case of reader beware. One should never take citations at face value – especially when used to confirm a biased argument. Rather than accepting such arguments we should follow them up, read the cited paper – and other papers in the research area of we have time. We should approach all such claims using citations critically and sensibly.

This is in line with the conclusion the authors make to their article:

“We’re not advising you to commit social suicide by interrupting every conversation with a demand for more evidence. But we do think the phrase “studies show…” should be met with cautious skepticism, especially when the study supports the politically-motivated preconceptions of the person who’s talking.”

Similar articles