Anti-fluoride authors indulge in data manipulation and statistical porkies

Darrell Huff & Irving Geis wrote a classic book – How to Lie With Statistics. They outline various ways data can be presented to give the wrong story. However, there is an even more naive use of statistics to misrepresent data – just declare that a relationship is statistically significant, don’t show any data or statistical analysis.

Unfortunately, many people are fooled by the use of those magical words – “statistically significant.”

I suppose the lay person could be excused – although it would pay even them to be a bit more sceptical about such claims. But it seems that even some “scientific” journals, or perhaps inadequate peer reviewers, can be fooled by those magical words. Here is an example in the paper by Hirzy et al., (2015) in the journal Fluoride. (Yes, I know, this journal is well known for its anti-fluoride stance and poor scientific quality but I would have thought the editor, Bruce Spittle,  would have picked this one up – even if they do not have an adequate peer review system. Perhaps the fact Spittle is one of the authors of the paper is a factor).

I critiqued the paper in my article Debunking a “classic” fluoride-IQ paper by leading anti-fluoride propagandists and have submitted a more formal critique to the journal (see – Critique of a risk analysis aimed at establishing a safe dose of fluoride for children.) But here I just want to deal with those magical words used in the paper – “statistically significant.”

Hirzy et al (2016) rely completely on data reported by Xiang et al., (2003) and claim they “found a statistically significant negative relationship between . . . .  drinking water fluoride levels and IQ.” Trouble is – you can search through the data presented by Xiang et al., (2003) and there is absolutely nothing to indicate a “statistically significant” relationship. Sure, that paper actually claims “This study found a significant inverse concentration-response relationship between the fluoride level in drinking water and the IQ of children.”  But there is no table or graphic presenting the individual data points and no statistical analysis for drinking water F and IQ. Rather surprising because Xiang et al., (2003) did present the individual data points for urinary fluoride and did present some results for statistical analysis of other relationships.

The trick behind the misleading use of Xiang’s data

However, what Xiang et al (2003) did do was separate their drinking water fluoride and IQ data into different ranges. This is a table of their result.

While group F was data for one village (Xinhuai) and the data in the other groups were for a separate village (Wamiao), there was no explanation of the criteria used for the groups – and the numbers in each group very tremendously. Over half the children (290 of the total 512) were in Group F and the size of the other groups seem to arbitrarily vary between 8 and 111.

This manipulation produces data which can be used to imply a statistically significant relationship. Do the statistical analysis for water F and IQ in the above table and sure you get a lovely straight line, a correlation of 0.96 and very significant statistically (p=0.003). But because of the manipulation, this says exactly nothing about the original data.

I will illustrate this by taking some data which Xiang et al (2003) did provide – for urinary fluoride and IQ. The data are illustrated in the figure below from the paper.

A statistical analysis of that data did show it was statistically significant – Xiang et al. (2003) cite a “Pearson correlation coefficient –0.174 , p = 0.003.” Now, that explains about 3% of the variance in IQ and I would have liked to see a similar analysis for water F as other workers have usually found weaker relationships for water F than for urinary F.

But let’s try using the manipulation of Xiang et al (2o03) and Hirzy et al (2016) to make the relationship between urinary F and IQ look a lot better than it is. I used a software tool to extract data from the figure – it didn’t extract all the points (264 out of a total 290) because of overlaps but statistical analysis of my extracted data gave a Pearson correlation coefficient of 0.16, p=0.010. Very similar to that reported by Xiang et al., (2003).

The tricky manipulations

I have absolutely no idea why Xiang et al., (2003) used different group sizes – so, to be fair, I have divided my extracted data into 6 groups of 44 pairs each (after sorting them into order based on urinary F) to produce the following table.

	Urinary F	IQ
A	1.79	105.57
B	2.30	89.45
C	2.30	77.72
D	2.69	68.58
E	2.48	56.25
F	2.69	40.10

This produces a lovely graph:

But, just a minute, I can get a better graph if I sort  the data according to IQ instead of urinary F:

But why stop there. If I choose different group sizes – remember Xiang et al., (2003) had groups ranging from 8 to 250 in size – I am sure I can get an even better presentation of the data.

TBut these graphs look far better than the one presented in Xiang et al (2003) for urinary F. We have taken data where the urinary F data explains only about 3% of the variance in IQ and produced graphics implying it “explains” up to about 75% of the variance. And we could “explain” more with a bit of extrra manipulation.

Conclusion

Data manipulation like this doesn’t change the fact that while the relationship between urinary F and IQ is statistically significant it only explains about 3% of the variance in IQ. This means that other factors, or confounders, should be considered – and when they are it is likely the significant relationship of IQ to urinary F would disappear.

Although Xiang et al., (2003) did not provide any statistical analysis to support their claim there was a significant relationship between water F and IQ I am sure the relationship is similar to that for urinary F – maybe even worse. Manipulating the data by using a range of groups of different sizes has certainly made the data look a lot better – but it is completely misleading.

I think it shocking that the authors of the Hirzy et al., (2016) paper have used manipulated data in this way – first to claim that fluoride in drinking water has a major negative effect on IQ and secondly to use such massaged data to work out a “safe dose.”

Worse, the journal Fluoride, and its peer reviewers, should never have accepted this paper without querying the claim of a significant relationship between drinking water F and IQ.

Similar articles