We all know the phrase “Lies, damned lies, and statistics.” If nothing else, this should warn us not to take on faith arguments which rely on statistical analysis for their credibility. Wikipedia uses this phrase to illustrate the “persuasive power of numbers, particularly the use of statistics to bolster weak arguments.”
Unfortunately, the scientific literature is full of weak arguments bolstered by statistics. It’s another case of “reader beware.” Do the statistical analyses used really support the argument? And how good was the statistical analysis anyway?
Unfortunately, scientific papers with poor or inappropriate statistical analyses often get used to bolster arguments in the political field. Anti-fluoride campaigners do this all the time. I illustrated this for the “fluoridation caused ADHD” argument in my articles ADHD linked to elevation, not fluoridation and ADHD link to fluoridation claim undermined again.
Another paper often used by anti-fluoride campaigners is that of Takahasi et al., (2001). They cite this to support their “fluoridation causes cancer” argument. For example, the prominent anti-fluoride activist Karen Favazza Spencer did this recently in a Facebook post quoting from Tkahashi et al., (2001):
“Cancers of the oral cavity and pharynx, colon and rectum… were positively associated with ‘optimally’ fluoridated drinking water.”
Well, how justified is that quote? How reliable was the statistical analysis used by these authors to arrive at that claim?
Takahashi et al., (2001)
In fact, their statistical analysis was poor. They considered only fluoridation as a factor. When we consider other likely factors the statistical analyses show no significant association between these cancers and fluoridation.
Let’s have a look at the paper and the statistical analysis.
The paper is:
Takahashi, K., Akiniwa, K., & Narita, K. (2001). Regression Analysis of Cancer Rates and Water Fluoride in the USA based Incidence on IACR / IARC ( WHO ) Data ( 1978-1992 ). Journal of Epidemiology, 11(4), 170–179.
Briefly, it searched for possible statistically significant associations between the incidence rates for a whole range of cancers and the extent of fluoridation. It used fluoridation extent and cancer incidence data for three US states and six US cities. Other factors were considered only for lip cancer where sunshine extent was included in the analyses.
I set out to repeat their statistical analysis, including some other relevant factors. However, the data they used for cancer incidence in 1978-1992 is not available on-line. But there are data sets available for more recent years.
Here I use the cancer incidence data for 1993-1997 taken from the WHO, International Agency for Research on Cancer publication Cancer Incidence in Five Continents Vol. VIII. This lists cancer incidence for 58 body sites but I restricted my analysis to eight of the body sites for which Takahashi et al., (2001) reported significant associations with fluoridation.
Are any of these cancers significantly associated with the extent of fluoridation?
Well, yes, two are at the 5% level (p < 0.05) – cancers of the rectum and bladder. The table lists values for the probability p value produced by linear regressions. The p values for cancers at all the body sites considered is also significant – but only for females.
||p – Male
||p – Female
Takahashi et al., (2001) found significant associations for rectum and bladder. But also for Colon, bone (male), oesophagus (female), prostate (male) and lip. This difference is not too surprising as I used a different, more recent, data set. Also, correlations do not mean causation, they can occur by chance (1 in 20 samples) and other factors are more than likely involved (see below).
Another difference is that I used simple linear regressions. Takahashi et al., (2001) transformed both fluoridation extent and cancer incidence to logarithms but their explanation for this is inadequate. Such transformations are not normally applied unless there is evidence that a relationship is nonlinear. Takahashi et al., (2001) did not give any evidence for this and there was no evidence for it in the data set I used. Neither was there any evidence of patterns in the residual values from the regression analysis – another sign that simple linear regression was valid.
What about the influence of other factors?
One of the biggest complaints I have about the use of regression analysis in studies like this is that very often other factors are ignored. Takahashi et al., (2001) considered only sun shine extent – and then only for lip cancer.
I think the restriction to consideration of only fluoridation is naive. In fact, probably indicating a bias and a desire to confirm it. It is extremely unlikely that all, or even most, of the specific cancers considered have a single cause – fluoride. And it is unlikely that a single factor would explain all the variability in the cancer incidence data.
Also, fluoride could be acting as a proxy for more relevant factors. The ADHD relationship with the extent of fluoridation is an example. In my paper Attention deficit hyperactivity disorder prevalence associated with altitude but not exposure to fluoridated water*, I showed that fluoridation extent is significantly correlated with mean altitude. When altitude was included in a multiple regression there was no significant association of ADHD with fluoridation. This suggests that, in fact, the fluoridation data was really a proxy for something else – in this case, altitude – which Huber et al (2015) reported is associated with ADHD prevalence.
I am not intending here to narrow down the most likely factors which are associated with cancer at all these body sites. I simply want to check how significant any association with fluoridation is when other possible factors are included.
Geographic factors are worth considering – not because they necessarily have a direct influence. But because they may act as proxies from environmental, population density and industrial concentration factors which could be important. So I included data for mean elevation, mean latitude and mean longitude together with the extent of fluoridation in multiple regressions of the eight cancers above as well as for all the body sites data.
Using adjusted R square values to test for a fluoridation contribution
Rather than attempting to identify significant correlations with different factors for different cancers, I used the method of judging what effect inclusion of fluoridation extent had on the explanatory power of regression models which included the geographic factors. Jim Frost describes this approach in his article Multiple Regression Analysis: Use Adjusted R-Squared and Predicted R-Squared to Include the Correct Number of Variables.
Briefly, he describes problems with the R squared value:
“Every time you add a predictor to a model, the R-squared increases, even if due to chance alone. It never decreases. Consequently, a model with more terms may appear to have a better fit simply because it has more terms.”
Include more factors and you could simply be modelling random noise in the data.But the adjusted R-squared overcomes this because it adjusts for the number of predictors in a model:
“The adjusted R-squared increases only if the new term improves the model more than would be expected by chance. It decreases when a predictor improves the model by less than expected by chance. The adjusted R-squared can be negative, but it’s usually not. It is always lower than the R-squared.”
These examples below of multiple regression output including fluoridation and excluding fluoridation in the models illustrate where adjusted R square values are reported:
The table below lists the adjusted R square values for multiple regressions:
- +Fl included fluoridation extent, mean elevation, mean latitude and mean longitude, and
- -F included only mean elevation, mean latitude and mean longitude.
Comparing the adjusted R square values for +Fl and -Fl tells us about the effect of including fluoridation extent on the models:
- Where the value for +Fl is larger than for -F then the extent of fluoridation improves to model more than would be expected by chance.
- Where the value of +Fl is smaller than for -F then the extent of fluoridation improves to model less than would be expected by chance.
The table shows that adjusted R square values are greater (red) when fluoridation extent is not included in the regression model for all cancer sites except the colon and female lip. That indicates that these cancers are not associated with fluoridation extent. That the simple regression results alone for fluoridation extent in the case of rectum and bladder cancer (and all sites female cancer) are misleading.
The colon and female lip cancer are exceptions – but the fact no significant association was found for fluoridation extent alone (first table) suggests something more complex is occurring here. It could be that the selected geographic factors have very little role in these cancers and inclusion of more relevant factors is needed.
The associations of fluoridation extent with various cancers reported by Takahashi et al., (2001) disappear when we consider other more relevant factors. Therefore, the use of this study by anti-fluoride campaigners to claim fluoridation is responsible for cancer is misleading. Not that I expect, from their past record, they will stop doing this.
More generally this is yet another example showing that readers should beware of putting too much faith in simple statistical analyses reported in scientific papers – even those published in respectable journals. It is just too easy to use statistical analysis to confirm a bias.
We should all keep in mind the phrase “Lies, damned lies, and statistics” and treat such reports critically. If possibly checking out the extent to which other factors have been considered. Even where significant correlations are reported we should check how useful such correlations are at explaining the variations in the data.
*The full text of this paper is not yet available as it is undergoing journal peer review. However, the full text of CRITIQUE OF A RISK ANALYSIS AIMED AT ESTABLISHING A SAFE DAILY DOSE OF FLUORIDE FOR CHILDREN, the first draft from which this paper was taken, is available.