Did the Royal Society of NZ and the Office of the NZ Prime Minister’s Chief Science Advisor make a big mistake in their report
This is what “Connett’s Crowd,” anti-fluoridation activists and propagandists, are saying in their attempts to discredit the review. So, did this review make the mistake its critics claim?
Well, no. It’s just a beat up. But there is a small mistake in the review’s executive summary which the anti-fluoridationists are pouncing on.
Most critics of community water fluoridation rely heavily on this paper:
Choi, A. L., Sun, G., Zhang, Y., & Grandjean, P. (2012). Developmental fluoride neurotoxicity: A systematic review and meta-analysis. Environmental Health Perspectives, 120(10), 1362–1368.
This was a metareview of mainly obscure and brief reports (see Quality and selection counts in fluoride research) indicating the possibility the fluoride intake by children living in high fluoride areas of China and Iran may suffer IQ deficits. Choi et al., (2012) used a statistical analysis to determine the possible size of the IQ drop averaged over all the studies. They found a small drop and said:
“The estimated decrease in average IQ associated with fluoride exposure based on our analysis may seem small and may be within the measurement error of IQ testing.”
Their abstract reported the:
“standardized weighted mean difference in IQ score between exposed and reference populations was –0.45 (95% confidence interval: –0.56, –0.35).”
(Their use of “standardised weighted mean difference” was poorly explained and has caused confusion with many readers. See below for a brief explanation of the term).
What did the Royal Society Review say about this?
The review discusses the question of possible neurotoxic effects on page 49-50. Their comment relevant to Choi et al., (2014) appears below (click to enlarge):
And this is what is in the executive summary (click below to enlarge). It makes a very small mistake by referring to “less than one IQ point” when it should have said “less than one standard deviation.”
So, the review reported the Choi et al., (2012) findings accurately but made a small mistake in the executive summary. This is really of no consequence because the overall message of the small size of the estimated IQ drop (described by the authors as “small and may be within the measurement error of IQ testing”) is not really altered.
What do the anti-fluoride critics say?
Such mistakes are inevitable and authors will universally say they usually find them only after publication when no correction is possible. I remember picking up 5 mistakes in one of my papers – mainly incorrect spelling of my own name several times and a mistake in the address of my institution – those were the early days of word processing! Of course no one used my mistakes to cast doubts on the scientific content of the paper.
Still, “Connett’s crowd” have been merciless in their criticism. Here is an example from the big man himself (see Water Fluoridation: The “Healthy” Practice That Has Deceived the World):
Gluckman and Skegg (sic)* mistakenly claim “a shift of less than one IQ point” in the 27 studies reviewed by Choi et al. (2012). What they have done here is to confuse the drop of half of one standard deviation reported by the authors with the actual drop in IQ, which was 6.9 points. Such an elementary mistake would not have been made by Gluckman and Skegg (sic)* if they had actually read the report, instead of relying on what fluoridation propagandists were saying about it.
* Of course Gluckman and Skegg – who Connett calls The ‘Hollow Men’ of New Zealand – did not author this review.
H.S. Micklem, in the Fluoride Free NZ report on the Royal Society review, snipes:
“It is hard to imagine how this mistake could have been made by anyone who had actually read the papers that are disparaged so casually.”
I guess critics should read carefully before indulging in such snaky comments. All they have demonstrated is that they did not read past the executive summary of the review (and certainly did not read the relevant section in the review). Or, more seriously, that they wish to misrepresent the review by highlighting the mistake and ignoring what the review actually says.
(At Least Kathleen Thiessen was more honest in her comments in the FFNZ report because she did refer to page 49 as well as the mistake. However she still concluded “The RSNZ report is not accurate in its characterization of the Choi et al. (2012) article on effects of fluoride on children’s IQ.”)
Update: One of my commenters, picker22, has brought this to our attention – it puts the mistake mentioend above into context.
“The original press release from Harvard School of Public Health News service made the same error stating that the difference was .5 IQ points. This error on the part of Harvard led to more that a couple of mis-statements by fluoridation advocates in the US.
The current web page notes that the sentence reporting the magnitude of IQ change was “updated” Sept 5, 2012. Sadly, I didn’t copy the original.
Is Choi et al (2012) relevant to fluoridation?
The only study specific to community water fluoridation (CWF) the Royal Society review mentions is Broadbent, et al., (2014). Community Water Fluoridation and Intelligence: Prospective Study in New Zealand.
The Choi et al., (2012) paper reviews reports mainly from areas of endemic fluorosis where fluoride intake is much higher than areas using CWF. Subsequently the same authors made their own measurements in a similar area of China and did not find a significant relationship of drinking water fluoride to IQ (see Choi et al., 2014. Association of lifetime exposure to fluoride and cognitive functions in Chinese children: A pilot study).
The did, however, find a relationship of IQ to severe dental fluorosis. I discuss their findings in my article Severe dental fluorosis the real cause of IQ deficits?
What is this “standardised weighted mean difference”
This term caused a lot of confusion with readers and critics. Choi et al., (2012) used this statistical device because they were attempting to estimate the average decrease in IQ associated with fluoride exposure based on the difference in IQ between children from high fluoride villages and low fluoride villages in a large number of studies. Further, different IQ scales and measurement methods were used in the different studies which had different levels of variation in the data.
They therefore standardised the differences by expressing them as a fraction of the standard deviation for each study. A mean value over all studies was determined, weighting the contribution from each study according to the precision of the IQ measurements.
The standardised weighted mean difference value of 0.45 has meaning because we know it represents less than half of one standard deviation so it gives us an indication of how it compares with measurement error. But a value of 6.9 as used by Paul Connett is meaningless – until we are told the standard deviation. Choi et al. (2012) did not report a difference of 6.9 implied by Paul Connett who appears to have obtained that value from a response to a letter to the editor where they use a hypothetical example to explain the meaning:
“For commonly used IQ scores with a mean of 100 and an SD of 15, 0.45 SDs is equivalent to 6.75 points (rounded to 7 points).”