Data dredging, p-hacking and motivated discussion in anti-fluoride paper

Image credit: Quick Data Lessons: Data Dredging

Oh dear – another scientific paper claiming evidence of toxic effects from fluoridation. But a critical look at the paper shows evidence of p-hacking, data dredging and motivated reasoning to derive their conclusions. And it was published in a journal shown to be friendly to such poor science.

The paper is:

Cunningham, J. E. A., Mccague, H., Malin, A. J., Flora, D., & Till, C. (2021). Fluoride exposure and duration and quality of sleep in a Canadian population-based sample. Environmental Health, 1–10.

Data dredging

This study used data from a Canadian database – the Canadian Health Measures Survey. Databases with large numbers of variables tempt researchers to dredge for data or relationships which confirm their biases. Despite the loss of statistical significance in this approach data dredging or data mining is quite common in epidemiological studies.

Cunningham et al (2021) looked for relationships using two separate measure of fluoride exposure and four different measures of possible sleep disturbance. They found a “statistically significant (p<0.05) relationship between lower sleep duration and water fluoride. But no relationships for higher sleep duration, trouble sleeping or daytime sleepiness with either water fluoride or urinary fluoride. Their results for logical regression analysis are summarised in this figure. (Error bars crossing an Odds Ratio value of 1.0 indicate that the relationship is not statistically significant and p<0.05).

Of the 8 relationships investigated only 1 was statistically significant.

p-hacking

I discussed the problem of p-hacking in Statistical manipulation to get publishable results.

With a large dataset, one can inevitably find relationships that satisfy the p<0.05 criteria – because this p-value value is meaningless when multiple relationships are considered. One can even find such “statistically significant relationships” when random datasets are investigated (see Science is often wrong – be critical,  I don’t “believe” in science – and neither should you, The promotion of weak statistical relationships in science  and Can we trust science). Once multiple relationships are investigated the chance of finding accidental relationships is much greater than 1 in 20 signified by the p<0.05 value.

So, one of the 8 relationships above satisfied the p<0.05 criteria when considered alone. But as part of multiple investigations, the chance of finding such a relationship by chance is much greater than 1 in 20.

Motivated reasoning

Image credit: Xkcd. 2167: Motivated Reasoning Olympics

This paper smacks of motivated reasoning. The authors obviously have a commitment to the concept that fluoride causes problems with the pineal gland and drag up anything they can find in the literature to support this – without critically assessing the quality of the cited work or even mentioning the fact that the cited studies were made at much higher fluoride concentration on non-human animals. In effect, they are attempting to convert very weak results, obtained by data dredging and p-hacking, to a fact. They are attempting to make a purse out of a sow’s ear.

This research group is not new to this game. I commented on this in my critique of another sleep disorder paper from the group (see Statistical manipulation to get publishable results).

Many of the same researchers are listed as authors on both papers – yet Cummingham et al (2021 ) cite the previous paper as if it was an independent study. They say “As far as we are aware, this is only the second human
study investigating the effects of fluoride exposure on sleep outcomes” which is simply disingenuous considering the involvement of the same researchers in both papers.

Both these papers were also published in the same journal – Environmental Health – a pay-to publish-journal that is known to be friendly to anti-fluoride researchers and uses very sympathetic peer reviewers (see  Some fluoride-IQ researchers seem to be taking in each other’s laundry). The Chief editor, Philippe Grandjean, is well known for his opposition to fluoridation. I commented on his refusal to consider a paper of mine that critiqued an anti-fluoride paper published in his journal (see Fluoridation not associated with ADHD – a myth put to rest).

Conclusion

Yet another very weak study, published in an anti-fluoride friendly pay-to-publish journal with poor peer review. Despite the weaknesses due to data dredging, p-hacking and motivated reasoning, anti-fluoride activists will cite the single “statistically significant” result as gospel and ignore the 7 relationships that are not significant. As for inadequate consideration of confounders or other risk-modifying factors, this study ignores completely the fact that city size and geographic factors have a strong effect on both sleep patterns and water fluoride concentrations (see Perrott 2018). Such inadequate consideration of confounders is another common problem in epidemiological studies.

Oh, well, we are not a rational species. More a rationalising one. And in such areas motivated rationalisation and confirmation bias is rife.

Similar studies