Finding reality needs more than wishful thinking. The problem is that statistical arguments often provide a jargon to confirm biases. Image credit: Accurate Thinking Versus Wishful Thinking in Gambling
I worry at the way some scientists use statistics to confirm their biases – often by retrieving marginal relationships from data that do not appear to provide evidence for their claims. This seems to be happening with the recent publication of a study reporting on maternal urinary fluoride-child IQ relationships in Canada (see If at first you don’t succeed . . . statistical manipulation might help).
Now we have a new paper from this group of researchers that seems to be repeating the pattern – this time with fluoride- attention deficit hyperactivity disorder (ADHD) relationships. The paper is:
Riddell, J. K., Malin, A., Flora, D., McCague, H., & Till, C. (2019). Association of water fluoride and urinary fluoride concentrations with Attention Deficit Hyperactivity Disorder in Canadian Youth. Submitted to Environment International, 133(May), 105190.
At first sight, the data does not seem promising for the fluoride-ADHD story. Compare the values of some of the factors they considered for Canadian youth which have been diagnosed with ADHD with values for youth not diagnosed with ADHD (From Table 2 in the paper).
It seems that being a male and exposure to smoking in the home are two factors predisposing youth to ADHD (already known) but the fluoride in tap water and fluoride intake (indicated by urinary F) have no effect. Although the data suggest that residence in sites where F is added to tap water may reduce the chances of ADHD diagnosis.
But the authors actually conclude that fluoride does increase the chance of an ADHD diagnosis. So it seems, once again, statistics appear to have been used in an attempt to incriminate fluoride – to make a silk purse out of a pig’s ear.
In effect, the paper is reporting three separate studies:
- They looked for a relationship of ADHD diagnosis with urinary fluoride;
- They checked if there was a difference in ADHD prevalence for youths living in fluoridated or unfluoridated areas, and
- They looked for a relationship of ADHD diagnosis with F in tap water.
No relationship of ADHD with urinary fluoride
SDQ hyperactive/inattentive subscale scores were obtained using a Strengths and Difficulties Questionnaire. Information about ADHD diagnosis and SDQ ratings were provided by parents of children aged 6 – 11 years and from a questionnaire completed by youth aged 12 – 17 years.
The paper reports:
“UFSG [urinary fluoride] did not significantly predict an ADHD diagnosis (adjusted Odds Ratio [aOR]=0.96; 95% CI: 0.63, 1.46, p=.84) adjusting for covariates.”
Similarly:
“UFSG did not significantly predict SDQ hyperactive/inattentive subscale scores
(B=0.31, 95% CI=−0.04, 0.66, p=.08).
So no luck there (for the authors who appear to be wishing to confirm a bias). The tone of the discussion indicates the authors were disappointed as they considered urinary fluoride has “has the advantage of examining all sources of fluoride exposure, not just from drinking water.” However, they did discuss some of the disadvantages of the spot samples for urinary fluoride they used:
“. . . urinary fluoride levels in spot samples are more likely to fluctuate due to the rapid elimination kinetics of fluoride. Additionally, urinary fluoride values may capture acute exposures due to behaviours that were not controlled in this study, such as professionally applied varnish, consumption of beverages with high fluoride content (e.g., tea), or swallowing toothpaste prior to urine sampling. Finally, the association between urinary fluoride and attention-related outcomes could be obscured due to reduced fluoride excretion (i.e., increased fluoride absorption) during a high growth spurt stage.”
We should note the WHO recommends against using urinary F as an indicator of F intake for individuals, and certainly against using spot samples (see Anti-fluoridation campaigner, Stan Litras, misrepresents WHO). They recommend 24-hour collections (see the WHO document “Basic Methods for Assessment of Renal Fluoride Excretion in Community Prevention Programmes for Oral Health”). I really cannot understand why these researchers chose spot sampling over 24-hour sample collection – although this would have not overcome the problem that urinary F is not a good indicator of fluoride intake at the individual level.
While it is refreshing to see the disadvantages of spot samples for urinary fluoride discussed, this probably would not have happened if they had managed to find a relationship. Neither Green et al., (2109) or Bashash et al., (2017) considered these problems – but then they managed to find relationships (although very weak ones) for spot samples.
Relationship of ADHD diagnoses with fluoridation
While this paper reports a significant (p<0.05) relationship of ADHD diagnosis and SDQ ratings with community water fluoridation (CWF) this really only applies to older youth (14 years). The relationship is not significant for younger youth (9 years).
However, the relationship is rather tenuous – this effect of age for ADHD diagnosis was seen only for “cycle 3” date (collected from 2012 to 2013) and was not seen for “cycle 2” data (collected from 2009 to 2011). The confidence intervals for Odd Ratios are also quite large – indicated the high variance in the data.
I think their conclusion of an effect due to fluoride and their lack of consideration of the poor quality of their relationships and alternative explanations for their results smacks a bit of straw clutching. The authors appear too eager to speculate on possible mechanisms involving fluoride rather than properly evaluating the quality of the relationships they found.
Relationship of ADHD diagnoses with tap water F
The paper reports a statistically significant (p<0.05) relationship of ADHD diagnosis with tap water fluoride. While the reported Odds Ratio appears very large (“a 1 mg/L increase in tap water fluoride was associated with a 6.1 times higher odds of ADHD diagnosis”) the 95% confidence interval is very large (1.60 to 22.8) indicating a huge scatter in the data. Unfortunately, the authors did not provide any more information from their statistical analysis to clarify the strength of the relationship.
Again, there was a significant relationship of SDQ score with tap water fluoride concentration but in this case, it was only significant for older youth and the CI was also relatively large.
So again the relationships with tap water F are tenuous – influenced by age and with large confidence intervals indicating a wide scatter in the data.
Problems with the paper’s discussion
Of course, correlation by no means implies causation. But there is always the problem of confirmation bias and special pleading where a low p-value in a regression analysis gets construed as evidence for the preferred outcome.
There are problems with relying only on p-values – which is why I have referred to confidence intervals and would prefer to actually see the actual data and full reports of the statistical analyses. The confidence interval values indicate that the data is highly scattered and the reported models from the regression analyses in this paper probably explain very little of the data. In such cases, there is a temptation to dig deeper and search for significant relationships by separating the data by sex or age but the resulting significant relationships may be meaningless.
And the “Elephant in the Room” – the relationships themselves say nothing about the reliability of the favored model. Nothing at all. A truly objective researcher would recognize this and avoid the staw clutching and rationalisation of evidence in the paper’s discussion. For example, the author’s considered another Canadian study which did not find any relationship of ADHD to fluoride in drinking water and argued the difference was solely due to deficiencies in the other study, not theirs.
The authors also seem not to recognise that any relationship they found may have nothing to do with fluoride but could be the result of other related risk-modifying factors they did not include in their statistical analysis. Worse, the argue their results are consistent with those of Malin and Till (2015) without any acknowledgment that that specific study is flawed. Perrott (2108) showed that the relationship reported by Malin & Till (2015) disappeared completely when the altitude was included in the statistical analysis. This is consistent with the study of Huber et al., (2015) which reported a statistically significant relationship of ADHD prevalence with altitude.
Conclusion
I think the Riddell et al., (2109) paper presents problems similar to those seen with a previous paper from this research group – Green et al., (2019). I have discussed some of these problems in previous articles:
- If at first you don’t succeed . . . statistical manipulation might help
- Politics of science – making a silk purse out of a sow’s ear
- More expert comments on the Canadian fluoride-IQ paper
- An evidence-based discussion of the Canadian fluoride/IQ study
- Fluoridation – A new fight against scientific misinformation
- Biostatistical problems with the Canadian fluoride/IQ study
Others in the scientific community have also expressed concern about the problems in that paper and a recent in-depth critical evaluation of (see CADTH RAPID RESPONSE REPORT: Community Water Fluoridation Exposure: A Review of Neurological and Cognitive Effects) pointed to multiple “limitations (e.g., non-homogeneous distribution of data, potential errors and biases in the estimation of maternal fluoride exposure and in IQ measurement, uncontrolled potential important confounding factors).” It urged that “the findings of this study should be interpreted carefully.”
More significantly widespread scientific concern about weaknesses in the Green et al., (2019) paper has led 30 scientific and health experts to write to the funding body involved (US National Insitute of Environmental Health Science – NIEHS) outlining their concern and appealing for the data to be made public for independent assessment (see Experts complain to funding body about quality of fluoride-IQ research. Download their letter). Last I was aware the authors were refusing to release their data – claiming not to own it!
We could well see similar responses to the Riddell et al., (2109) ADHD paper.