The promotion of weak statistical relationships in science

Image credit: Correlation, Causation, and Their Impact on AB Testing

Correlation is never evidence for causation – but, unfortunately, many scientific articles imply that it is. While paying lip service to the correlation-causation mantra, some (possibly many) authors end up arguing that their data is evidence for an effect based solely on the correlations they observe. This is one of the reasons for the replication crisis in science where contradictory results are being reported. Results which cannot be replicated by other workers (see I don’t “believe” in science – and neither should you).

Career prospects, institutional pressure and the need for public recognition will encourage scientists to publish poor quality work that they then use to claim that have found an effect. The problem is that the public, the news media and even many scientists simply do not properly scrutinise the published papers. In most cases they don’t have the specific skills required for this.

There is nothing wrong with doing statistical analyses and producing correlations. However such correlations should be used to suggest future more meaningful and better-designed research like randomised controlled trials (see Smith & Ebrahim 2002 – Data dredging, bias, or confounding. They can all get you into the BMJ and the Friday papers. ). They should never be used as “proof” for an effect, let alone argue that the correlation is evidence to support regulations and advise policymakers.

Hunting for correlations

However, researchers will continue to publish correlations and make great claims for them because they face powerful incentives to promote even unreliable research results. Scientific culture and institutional pressures provide expectations demanding academic researchers produce publishable results. This pressure is so great they will often clutch at straws to produce correlations even when the initial statistical analyst produces none. They will end up “torturing the data.”

These days epidemiological researchers use large databases and powerful statistical software in their search for correlations. Unfortunately, this leads to data mining which, by suitable selection of variables, makes the discovery of statistically significant correlations easy. The data mining approach also means that the often cite p-values are meaningless. P-values measure the probability the relationship occurs by chance and often cited as evidence of the “robustness” of the correlations. But probability is so much greater when researchers resort to checking a range of variables and that isn’t reflected properly in the p-values.

Where data mining occurs, even to a limited extent, researchers are simply attempting to make a purse out of sow’s ear when they support their correlations merely by citing a p-value < 0.05  because these values are meaningless in such cases. The fact that so many of these authors often ignore more meaningful results from their statistical analyses (like R-squared values which indicate the extent that the correlation “explain” the variation in their data) underlines their deceptive approach.

Poor statistical relationships

Consider these correlations below – two data sets are taken from a published paper – the other four use random data provided by Jim Jones in his book Regression Analysis: An Intuitive Guide.

You can probably guess which correlations were from real data (J and M) because there are so many more data points All of these have correlations low p values – but of course, those selected from random data sets resulted from data mining and the p-values are therefore meaningless because they are just a few of the many checked. Remember, a p-value < 0.05 means that the probability of a chance effect is one in twenty and more than twenty variable pairs were checked in this random dataset.

The other two correlations are taken from Bashash et al (2017). They do not give details of how many other variables were checked in the dataset used but it is inevitable that some degree of data mining occurred. So, again, the low p-values are probably meaningless.

J provides the correlation of General Cognitive Index (GCI) scores in children at age 4 years with maternal prenatal urinary fluoride and M provides the correlation of children’s IQ at age 6–12 y with maternal prenatal urinary fluoride. The paper has been heavily promoted by anti-fluoride scientists and activists. None of the promoters have made a critical, objective, analysis of the correlations reported. Paul Connett, director of the Fluoride Action Network, was merely supporting his anti-fluoride activist bias when he uncritically described the correlations as “robust.” They just aren’t.

There is a very high degree of scattering in both these correlations, and the R-squared values indicate they cannot explain any more than about 3 or 4% of the variance in the data. Hardly something to hang one’s hat on, or to be used to argue that policymakers should introduce new regulations controlling community water fluoridation or ban it altogether.

In an effort to make their correlations look better these authors imposed confidence intervals on the graphs (see below). This Xkcd cartoon on curve fitting gives a cynical take on that. The grey areas in the graphs may impress some people but it does not hide the wide scatter of the data points. The confidence intervals refer to estimates of the regression coefficient but when it comes to using the correlations to predict likely effects one must use the prediction intervals which are very large (see Paul Connett’s misrepresentation of maternal F exposure study debunked). In fact, the estimated slopes in these graphs are meaningless when it comes to predictions.

Correlations reported by Bashash et al (2017). The regressions explain very little of the variance in the data and connect be used to make meaningful predictions.

In critiquing the Bashash et al (2017) paper I must concede that at least they made their data available – the data points in the two figures. While they did not provide full or proper results from their statistical analysis (for example they didn’t cite the R-squared values) the data does at least make it possible for other researchers to check their conclusions.

Unfortunately, many authors simply cite p-values and possible confidence intervals for the estimate of the regression coefficient without providing any data or images. This is frustrating for the intelligent scientific reader attempting to critically evaluate their claims.

Conclusions

We should never forget that correlations, no matter how impressive, do not mean causation. It is very poor science to suggest they do.

Nevertheless, many research resort to correlations they have managed to glean from databases, usually resorting to some extent of data mining, to claim they have found an effect and to get published. The drive to publish means that even very poor correlations get promoted and are used by ideologically or career-minded scientists, and by activists, to attempt to convince policymakers of their cause.

Image credit: Xkcd – Correlation

Remember, correlations are never evidence of causation.

Similar articles

Can we trust science?

Image credit: Museum collections as research data

Studies based simply on statistically significant relationships found by mining data from large databases are a big problem in the scientific literature. Problematic because data mining, or worse data dredging, easily produces relationships that are statistically significant but meaningless. And problematic because authors wishing to confirm their biases and promote their hypotheses conveniently forget the warning that correlation is not evidence for causation and go on to promote their relationships as proof of effects. Often they seem to be successful in convincing regulators and policymakers that their serious relationships should result in regulations. Then there are the activists who don’t need convincing but will promote willingly and tiresomely these studies if they confirm their agendas.

Even random data can provide statistically significant relationships

The graphs below show the fallacy of relying only on statistically significant relationships as proof of an effect. The show linear regression result for a number of data sets. One data set is taken from a published paper – the rest use random data provided by Jim Jones in his book Regression Analysis: An Intuitive Guide.

All these regressions look “respectable.” They have low p values (less than the conventional 0.05 limit) and the R-squared values indicated they “explain” a large fraction of the data – up to 49%. But the regressions are completely meaningless for at least 7 of the 8 data sets because the data were randomly generated and have no relevance to real physical measurements.

This should be a warning that correlations reported in scientific papers may be quite meaningless.

Can you guess which of the graphs is based on real data? It is actually the graph E – published by members of a North American group currently publishing data which they claim shows community water fluoridation reduces child IQ. This was from one of their first papers where they claimed childhood ADHD was linked to fluoridation (see Malin, A. J., & Till, C. 2015. Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association).

The group used this paper to obtain funding for subsequent research. They obviously promoted this paper as showing real effects – and so have the anti-fluoride activists around the world, including the Fluoride Action Network (FAN) and its director Paul Connett.

But the claims made for this paper, and its promotion, are scientifically flawed:

1. Correlation does not mean causation. Such relationships in larger datasets often occur by chance – hell they even occur with random data as the figure above shows.
2. Yes, the authors argue there is a biologically plausible mechanism to “explain” their association. But that is merely cherry-picking to confirm a bias and there are other biologically plausible mechanisms they did not consider which would say there should not be an effect. The unfortunate problem with these sorts of arguments is that they are used to justify their findings as “proof” of an effect. To violate the warning that correlation is not causation.
3. There is the problem of correcting for cofounders or other risk-modifying factors. While acknowledging the need for future studies considering other confounders, the authors considered their choice of socio-economic factors was sufficient and their peer reviewers limited their suggestion of other confounders to lead. However, when geographic factors were included in a later analysis of the data the reported relationship disappeared. 

Confounders often not properly considered

Smith & Ebrahim (2002) discuss this problem an article  – Data dredging, bias, or confounding. They can all get you into the BMJ and the Friday papers. The title itself indicates how the poor use of statistics and unwarranted promotion of statical analyses can be used to advance scientific careers and promote bad science in the public media.

These authors say:

“it is seldom recognised how poorly the standard statistical techniques “control” for confounding, given the limited range of confounders measured in many studies and the inevitable substantial degree of measurement error in assessing the potential confounders.”

This could be a problem even for studies where a range of confounders are included in the analyses. But Malin & Till (2015) considered the barest minimum of confounders and didn’t include ones which would be considered important to ADHD prevalence. In particular, they ignored geographic factors and these were shown to be important in another study using the same dataset. Huber et al (2015) reported a statistically significant relationship of ADHD prevalence with elevation. These relationships are shown in this figure

Of course, this is merely another statistically significant relationship – not proof of a real effect and no more justified than the one reported by Malin and Till (2015). But it does show an important confounder that Malin & Till should have included in their statistical analysis.

I did my own statistical analysis using the data set of Malin & Till (2015) and Huber et al (2015) and showed (Perrott 2018) that inclusion of geographic factors showed there was no statistically significant relationship of ADHD prevalence with fluoridation as suggest by Malin & Till (2015). Their study was flawed and it should never have been used to justify funding for future research on the effect of fluoridation. Nor should it have been used by activists promoting an anti-fluoridation agenda.

But, then again, derivation of a statistically significant relationship by Malin & Till (2o15) did get them published in the journal Environmental Health which, incidentally, has sympathetic reviewers (see Some fluoride-IQ researchers seem to be taking in each other’s laundry) and an anti-fluoridation Chief Editor – Phillipe Grandjean (see Special pleading by Philippe Grandjean on fluoride). It also enabled the promotion of their research via institutional press releases, newspaper article and the continual activity of anti-fluoridation activists. Perhaps some would argue this was a good career move!

Conclusion

OK, the faults of the Malin & Till (2015) study have been revealed – even though Perrott (2018) is studiously ignored by the anti-fluoride North American group which has continued to publish similar statistically significant relationships of measures of fluoride uptake and measures of ADH or IQ.

But there are many published papers – peer-reviewed papers – which suffer from the same faults and get similar levels of promotion. They are rarely subject to proper post-publication peer-review or scientific critique. But their authors get career advancement and scientific recognition out of their publication. And the relationships are promoted as evidence for real effects in the public media.

No wonder members of the public are so often confused by the contradictory reporting, the health scares of the week, they are exposed to.

No wonder many people feel they can’t trust science.

Similar articles

Science is often wrong – be critical

Activists, and unfortunately many scientists, use published scientific reports like a drunk uses a lamppost – more for support than illumination

Uncritical use of science to support a preconceived position is widespread – and it really gets up my nose. I have no respect for the person, often an activist, who uncritically cites a scientific report. Often they will cite a report which they have read only the abstract of – or not even that. Sometimes commenters will support their claims by producing “scientific evidence” which are simply lists of citations obtained from PubMed or Google Scholar.

[Yes, readers will recognise this is a common behaviour with anti-fluoride activists]

Unfortunately, this problem is not restricted to activists. Too often I read scientific papers with discussions where authors have simply cited studies that support, or they interpret as supporting, their own preconceived ideas or hypotheses. Compounding this scientific “sin” is the habit of some authors who completely refuse to cite, or even discuss, studies producing evidence that doesn’t fit their scientific prejudices.

Publication does not magically make scientific findings or ideas “true” – far from it. The serious reader of scientific literature must constantly remember that the chances are very high that published conclusions or findings are likely to be false. John Ioannidis makes this point in his article Why most published research findings are false. Ioannidis concentrates on the poor use, or misuse, of statistics. This is a constant problem in scientific writing – and it certainly underlines the fact that even scientists will consciously or unconsciously manipulate their data to confirm their biases. They are using statistical analysis in the way a drunk used a lamppost – for support rather than illumination.

Poor studies often used to fool policymakers

These problems are often not easily understood by scientists themselves but the situation is much worse for policymakers. They are not trained in science and don’t have the scientific or statistical experience required for a proper critically analysis of claims made to them by activists. Yet they are often called on to make decisions which rely on the acceptance, or rejection, of scientific claims (or, claims about the science).

An example of this is a draft (not peer-reviewed) paper by Grandjean et al  – A Benchmark Dose Analysis for Maternal Pregnancy Urine-Fluoride and IQ in Children.

These authors have an anti-fluoride activists position and are campaigning against community water fluoridation (CWF). Their paper uses their own studies which report very poor and rare statistical relationships of child IQ with fluoride intake as “proof” of causation sufficiently strong to advocate for regulatory guidelines. Unsurprisingly their recommended guidelines are very low – much lower than those common with CWF.

Sadly, their sciencey sounding advocacy may convince some policymakers. It is important that policymakers be exposed to a critical analysis of these studies and their arguments. The authors will obviously not do this – they are selling their own biases. I hope that any regulator or policymaker required to make decisions on these recommendations have the sense to call for an independent, objective and critical analysis of the paper’s claims.

[Note: The purpose of the medRxiv preprints of non-peer-reviewed articles is to enable and invite discussion and comments that will help in revising the article. I submitted comments on the draft article over a month ago (Comments on “A Benchmark Dose Analysis for Maternal Pregnancy Urine-Fluoride and IQ in Children”) and have had no response from the authors.  This lack of response to constructive critiques is, unfortunately, common for this group. I guess one can only comment that scientists are human.]

Observational studies – exploratory fishing expeditions

A big problem with published science today is that many studies are nothing more than observational exploratory studies using existing databases which, by their nature, cannot be used to derive causes. Yet that can easily be used to derive statistically significant links or relationships. These can be used to write scientific papers but they are simply not evidence of causes.

Properly designed studies, with proper controls and randomised populations properly representing different groups, may provide reasonable evidence of causal relationships – but most reported studies are not like this. Most observational studies use existing databases with non-random populations where selection and confounding with other factors is a huge problem. Authors are often silent about selection problems and may claim to control for important confounding factors, but it is impossible to include all confounders. The databases used may not include data for relevant confounders and authors themselves may not properly select all relevant confounders for inclusion.

This sort of situation makes some degree of data mining likely., This occurs when a number of different variables and measures of outcomes are considered in the search for statistically significant relationships. Jim Frost illustrated the problems with this sort of approach. Using a set of completely fictitious random data he was able to obtain a statistically significant relationship with very low p values and R-squared values showing the explanation of 61% of the variance (see Jim Frost – Regression Analysis: An Intuitive Guide).

That is the problem with observational studies where some degree of data mining is often involved. It is possible to find relationships wich look good, have low p-values and relatively high R-squared values, but are entirely meaningless. They represent nothing.

So readers and users of science should beware. The findings they are given may be completely false or contradictory. or at least meaningless in quantitative terms (as is the case with the relationships produced by the Grandjean et al 2020 group discussed above).

A recent scientific article provides a practical example of this problem. Different authors used the same surgical database but produced complete opposite findings (see Childers et al: 2020). Same Data, Opposite Results?: A Call to Improve Surgical Database Research). By themselves each study may have looked convincing. Both used the same large database from the same year. Over 10,000 samples were used in both cases and both studies were published in the same journal within a few months. However, the inclusion and exclusion criteria used were different. Large numbers of possible covariates were considered but these differed. Similarly, different outcome measures were used.

Readers interested in the details can read the original study or a Sceptical Scalpel blog article Dangerous pitfalls of database research. However, Childers et al (2020) describe how the number of these sort of observational studies “has exploded over the past decade.” As they say:

“The reasons for this growth are clear: these sources are easily accessible, can be imported into statistical programs within minutes, and offer opportunities to answer a diverse breadth of questions.”

However:

“With increased use of database research, greater caution must be
exercised in terms of how it is performed and documented.”

“. . . because the data are observational, they may be prone to bias from selection
or confounding.”

Problems for policymakers and regulators

Given that many scientists do not have the statistical expertise to properly assess published scientific findings it is understandable for policymakers or regulators to be at a loss unless they have proper expert advice. However, it is important that policymakers obtain objective, critical advice and not simply rely on the advocates who may well have scientific degrees. Qualifications by themselves are not evidence of objectivity and, undoubtedly, we often do face situations where scientists become advocates for a cause.

I think policymakers should consciously seek out a range of scientific expert advice, recognising that not all scientists are objective. Given the nature of current observational research, its use of existing databases and the ease with which researchers can obtain statistically significant relationships I also think policymakers should consciously seek the input of statisticians when they seek help in interpreting the science.

Surely they owe this to the people they represent.

Similar articles

No relationship of bone cancer to fluoridation – another new study the anti-fluoride brigade will attempt to ignore

Anti-fluoride activists claim that water fluoridation causes nine cancer proved wrong, yet again. Image credit: Four myths about water fluoridation and why they’re wrong

A new study confirms, yet again, that osteosarcoma, a type of bone cancer, is not associated with community water fluoridation (CWF). This the seventh such study since a 1990 report of an animal study suggested such a link.

The 199o study exposed rats to very high concentrations of fluoride so the results were not relevant to CWF. But, of course, this did not stop anti-fluoride campaigners using the study to argue that CWF causes osteosarcoma.

The citation for this new study is:

Kim, F. M., Hayes, C., Burgard, S. L., Kim, H. D., Hoover, R. N., Osteosarcoma, N., … Couper, D. (2020). A Case-Control Study of Fluoridation and Osteosarcoma. Journal of Dental Research 1.

This was a hospital-based study where patients diagnosed with osteosarcoma were compared with control patients diagnosed with other bone tumours or different conditions. This figure summarises the findings.

The only statistically significant effects show a reduced likelihood of osteosarcoma diagnosis for people living in fluoridated areas – compared with those living in non-fluoridated areas (the red triangles in the figure). These were for people who never drank water and people who had lived in fluoridated areas for 0% to 50% of their lives. It is likely the effects for people who did drink bottles water and those who had lived in fluoridated areas for 50% to 100% or 100% of their lives are not statistically significant because of the smaller numbers involved (The green circles in the figure).

It’s been a bad week for the anti-fluoride crowd – the science keeps proving them wrong. Perhaps that is why they are silent about these new studies.

Similar articles

 

New review finds fluoride is not a developmental neurotoxicant at exposure levels relevant to fluoridation

Proper consideration of the best science shows community water fluoridation does not have a negative effect on child IQ. Image credit: Africa Studio / Shutterstock.com

A new extensive review of the scientific literature has concluded that fluoride is not a human developmental neurotoxicant at the current exposure levels in Europe. This is of course just as valid for New Zealand, the USA and other countries which use community water fluoridation (CWF).

Forty-one pages long, it’s a very extensive and detailed review. The full text can be downloaded  and its citation is:

Guth, S., Hüser, S., Roth, A., Degen, G., Diel, P., Edlund, K., … Thomas, H. (2020). Toxicity of fluoride: critical evaluation of evidence for human developmental neurotoxicity in epidemiological studies, animal experiments and in vitro analyses. Archives of Toxicology. 2020 May 8.

The anti-fluoridation crowd won’t be happy with this review. They have tended to have things their own way as they have argued that fluoridation is harmful to child IQ using irrelevant studies from endemic fluorosis areas where people suffer a range of health effect from overexposure to fluoride and other contaminants. Anti-fluoride campaigners have also misrepresented and misused recent studies from areas where fluoride exposure is lower.

So this review is timely because it critically examines all the recent studies and identifies their limitations. It identified 23 relevant epidemiological studies published between January 2012 and August 2019. One of these examined an association between fluoride exposure and school performance. The other 22 examined possible relationships with IQ.

Limitations of fluoride-IQ studies

The authors reported that:

“So far, almost all studies investigating the effect of fluoride intake on intelligence were performed in relatively poor, rural communities, e.g., in China, Iran, and Mongolia, where drinking water may contain comparatively high levels of fluoride (‘exposed population’), whereas the ‘reference populations’ often had access to water that was fluoridated at the recommended level.”

Figure 1: People in endemic fluorosis area sufferer a range of health problems – studies from these areas are not relevant to CWF

This means that anti-fluoride campaigners usually rely on studies which actually show no effect at F intake levels relevant to CWF. They base their arguments on the known negative health effects at high fluoride intake (people in areas of endemic fluorosis suffer a range of health problems) but ignore, or cover-up the fact the data actually does not show any harmful effects at levels similar to that experienced by people in areas of CWF.

Figure 2: Drinking water concentrations reported by Duan et al. (2018) from “high F” and “low F” villages compared with tap water F in areas of CWF

Figure 2 above shows this using data from 26 studies reported in the review of Duan et al. (2018). Here the blue range represents the drinking water concentration range for the control groups where no health problems were reported, or it was assumed none occurred (that is why it was a control group). The green range represents drinking water fluoride concentration common in areas of CWF.

We should be drawing our conclusions about the possible effects of CWF from the blue range of data – not the red range.

Confounding effects

Guth et al (2020) stress that most studies they considered ignored many confounding effects.  For example:

” . .rural regions with unusually high or unusually low fluoride in drinking water may be associated with a less developed health-care system, as well as lower educational and socioeconomic status. Furthermore, in these regions the overall nutritional status and the intake of essential nutrients may be lower and the exposure to environmental contaminants such as lead, cadmium, mercury, or manganese may be higher—factors that are also discussed to have a potential impact on intelligence”

Only two of the studies were from areas using CWF – Broadbent et al (2015) and Green et al (2019) – and their conclusions were different. Guth et al (2020) considered these two studies in detail.

Both studies were limited by the lack of IQ data for mothers – parental IQ is a strong confounder for child IQ studies. But Guth et al (2020) are quite critical of the lack of consideration of confounders in the Green et al (2019) study:

“Green et al. (2019) did not consider breastfeeding and low birth weight as possible confounders (both factors significantly associated with IQ in the study of Broadbent); they considered some of the relevant confounders (city, socioeconomic status, maternal education, race/ethnicity, prenatal secondhand smoke exposure), but did not adjust for others (alcohol consumption and further dietary factors, other sources of fluoride exposure, exact age of children at time point of testing). Furthermore, the study (Green et al. 2019) did not include assessment of children’s postnatal fluoride exposure via, e.g., diet, fluoride dentifrice, and/or fluoride tablets, which is considered to be a noteworthy limitation.”

Problems like poor consideration of confounders, contradictory results and the vague results reported by Green et al (2019) (no overall effect of fluoridation on child IQ, a statistically significant relationship of drinking water F concentration with male child IQ but not with female child IQ) caused Guth et al (2020) to conclude:

“The available epidemiological evidence does not provide sufficient arguments to raise concerns with regard to CWF in the range of 0.7–1.0 mg/L, and to justify the conclusion that fluoride is a human developmental neurotoxicant that should be categorized as similarly problematic as lead or methylmercury at current exposure levels.”

To repeat – this review is very detailed and thorough. Unlike the recent review of Grandjean (2019) (Developmental fluoride neurotoxicity: an updated review) which was superficial and somewhat biased (Grandjean is well known for his opposition to CWF) it made a detailed assessment of problems like the poor consideration of confounders or important risk-modifying factors and the concentration on poor quality studies from areas of endemic fluorosis.

Hopefully, policymakers will read this new review and take its conclusions into account.

Similar articles

Industry-funded translation can introduce bias in selection of studies for scientific review

Image credit: Assessing and addressing bias in systematic reviews

The Fluoride Action Network (FAN), in the last decade, paid for translation of a lot of Chinese-language scientific papers linking high fluoride dietary intake to IQ deficits in children. They, of course, selected papers to fit their own ideologically-motivated bias. This is perfectly understandable for an activist group. But has this caused a bias in available English-language sources on this topic? And does this mean recent scientific reviews of this subject unintentionally suffer from selection bias?

I hadn’t considered this possibility before, but it is an issue raised in the recent US National Academies of Sciences (NAS) peer review of the US National Toxicity Program’s (NTP) review of possible neurotoxic effects of fluoride (see Another embarrassment for anti-fluoride campaigners as neurotoxic claim found not to be justified).

Use of FAN sources introduces biased study selection

The NAS peer reviewers are harshly critical of the NTP draft review. A central concern was the way the NTP evaluated the literature on the subject. The NAS peer reviewers say on page 3 of their report:

“The committee had substantive concerns regarding NTP’s evaluation of the human evidence as noted below. The strategy used for the literature search indicated that NTP used FAN as a source to identify relevant literature. The process by which FAN identified and selected studies is unclear, and that uncertainty raises the question of whether the process could have led to a biased selection of studies. Such a concern raises the need for a formal evaluation of any potential bias that might have been introduced into the literature-search process.”

OK, I am not impressed that the NTP used FAN as a source. FAN is hardly a reliable source and its “study tracker” certainly does not pick up anywhere near the full literature available (see Cherry-picking and ring-fencing the scientific literature). But, at first thought, I imagined that the FAN source simply produced a subset of anything that is picked up using a more reliable source like PubMed to do literature searches.

Injection of study bias into English-language scientific literature

But the NAS peer reviewers raise an important problem with reliance on FAN as a source and its effect on the available English-language scientific literature. On page 24 of their report they say:

“. . the process by which FAN identified and selected studies is not clear. FAN identified a number of studies published in Chinese language journals—some of which are not in PubMed or other commonly used databases—and translated them into English. That process might have led to a biased selection of studies and raises the question of whether it is possible that there are a number of other articles in the Chinese literature that FAN did not translate and about which NTP is unaware. NTP should evaluate the potential for any bias that it might have introduced into the literature search process. Possible ways of doing so could include conducting its own searches of the Chinese or other non–English-language literature and conducting subgroup analyses of study quality and results based on the resource used to identify the study (for example, PubMed vs non-PubMed articles). As an initial step in such evaluations, NTP should consider providing empirical information on the pathway by which each of the references was identified. That information would also improve understanding of the sources that NTP used for evidence integration and the conclusions drawn in the monograph.”

In a nutshell, FAN arranged and paid for translation of quite a large number of Chinese papers on this issue (fluoride intake and child IQ deficits). Naturally, they have selected papers supporting their political cause (the abolition of community water fluoridation) and ignored papers which they could not use to that end. It is therefore likely they have introduced into English-language scientific literature a biased selection of Chinese papers because FAN effectively “republished” the translated papers in the journal “Fluoride” – a well-known repository of anti-fluoride material.

Maybe I was wrong to assume anything from FAN would simply be a subset of what is available through more respectable searching sources. But, according to the peer reviewers, some of the translated papers may be picked up when FAN is used as a source of studies but not when PubMed or similar respected sources are used. A warning, though – many of the FAN-promoted translated studies have only been partly translated, maybe only the abstract is available. This is not sufficient for a proper scientific review (see Beware of scientific paper abstracts – read the full text to avoid being fooled).

I am not saying this bias introduction into the English-language scientific literature was intentional, but it is a likely end-result of their actions. Importantly, it is also a likely end-result of funding from big money sources (the “natural”/alternative health industry which funds FAN and similar anti-fluoride and anti-vaccination groups – see Big business funding of anti-science propaganda on health).

So, is this a way that big industry can inject their bias into the available scientific literature? A way to ensure that reviewers will, maybe unintentionally, convey this industry bais into their own summary of scientific findings?

Reviewers should make a critical assessment of studies

The FAN-promoted Chinese studies really do not contribute to any rational discussion of issues with CWF because they were all made in areas of endemic fluorosis. Ironically they often compare child IQ in villages where fluoride intake is high, with that in villages where the fluoride intake is low. It is the low -fluoride villages which are relevant to areas of CWF because their drinking water F concentrations are comparable.

In reality, these Chinese studies could be used to support the idea that CWF is harmless. Even if that is an inherent assumption for low fluoride intake in these studies.

So, perhaps the bias introduced to the literature by translation of the FAN-promoted studies really is of no consequence to the evaluation of CWF. However, consideration of reviews like the recent one by Grandjean (2019) indicates there is a tendency to simply extrapolate from high concentration studies to make unwarranted conclusions about CWF. In this case, the tendency is understandable as Grandjean is well known for his opposition to CWF and is often used by FAN to make press statements raising doubts about this health policy (see Special pleading by Philippe Grandjean on fluoride, Some fluoride-IQ researchers seem to be taking in each other’s laundry, and Fluoridation not associated with ADHD – a myth put to rest).

This was also a problem with the draft NTP review which produced the (unwarranted) conclusion “that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans.” The draft did actually mention that the conclusion “is based primarily on higher levels of fluoride exposure (i.e., >1.5 ppm in drinking water” and “effects on cognitive neurodevelopment are inconsistent, and therefore unclear”  for “studies with exposures in ranges typically found in the water distribution systems in the United States (i.e., approximately 0.03 to 1.5 ppm according to NHANES data).” But, of course, it is the unwarranted conclusion that gets promoted.

Conclusions

Reviewers need to be aware of this and other ways activist groups and big business can inject bias into the scientific literature.

This problem underlines the responsibility reviewers have of recognising all possible ways that biased selection of studies they consider can occur. It also means they should make every effort to include negative studies (not supporting the effect they may personally prefer) as well as positive studies. They also need to include all the findings (positive and negative) included in the individual studies they review.

In cases like the FAN-promoted Chinese studies, there is an obligation to at least note the possibility of bias introduced by activists and industry-funded translations. Even better, to ensure that the reviewer undertakes to independently search for all studies on the subject and arrange for translations where necessary.

Above all, reviewers should critically consider the quality of the studies they include in their reviews and not simply rely on their own confirmation bias.

Similar articles

Biostatistical problems with the Canadian fluoride/IQ study

There are insights in there somewhere. Image Credit: DATA ANALYTICS COMES TO THE LEGAL PROFESSION

There has been widespread scientific criticism of the recently published Canadian fluoride-IQ study of Green et al., (2019). Most recently Dr. René F. Najera (a Doctor of Public Health, an epidemiologist and biostatistician) has critiqued the statistical analysis. He finds a number of faults and concludes by hoping “public health policy is not done based on this paper:”

 “It would be a terrible way to do public health policy. Scientific discovery and established scientific facts are reproducible and verifiable, and they are based on better study designs and stronger statistical outcomes than this. “

Dr. Najera’s critiques the biostatistics is in his article The Hijacking of Fluorine 18.998, Part Three. This follows his previous critique (Part 1 and Part 2) of the epidemiological issues which I reviewed in Fluoridation – A new fight against scientific misinformation.

Dr. Najera starts by stressing the important role of biostatistics in epidemiological studies. After all the planning and measurement:

“.. we hand off the data to biostatisticians, or we do the work with biostatisticians. Doing this assures us that we are measuring our variables correctly and that all associations we see are not due to chance. Or, if chance had something to do with it, we recognize it and minimize the factors that lead to chance being a factor in and of itself.”

I agree completely. In my experience statisticians play a critical role in research and should be involved even at the planning stage. Further, I think the involvement of experienced biostatisticians is invaluable. Too often I see papers where the authors themselves relied on their own naive statistical analyses rather than calling on experience. Perhaps they are being protective of their own confirmation bias.

The specific study Dr. Najera critiques is:

Green, R., Lanphear, B., Hornung, R., Flora, D., Martinez-Mier, E. A., Neufeld, R., … Till, C. (2019). Association Between Maternal Fluoride Exposure During Pregnancy and IQ Scores in Offspring in Canada. JAMA Pediatrics, 1–9.

For my other comments on the Candian fluoride/IQ research see:

• If at first you don’t succeed . . . statistical manipulation might help
• Politics of science – making a silk purse out of a sow’s ear
• More expert comments on the Canadian fluoride-IQ paper
• An evidence-based discussion of the Canadian fluoride/IQ study
• Fluoridation – A new fight against scientific misinformation

No comparison group

One problem with this study is that a number of mother-child pairs were excluded and, in the end, the sample used was not representative of the Canadian population. Najera summarises the “main finding of the study as “that children of mothers who ingested fluoride during pregnancy had 4 IQ points lower for each 1 mg of fluoride consumed by the mother:”

“If you’re asking yourself, “Compared to whom?” you are on the right track. There was no comparison group. Women who did not consume tap water or lived outside a water treatment zone were not included, and that’s something I discussed in the previous post. What the authors did was a linear regression based on the data, and not much more.”

In fact, while the sample used was unrepresentative the study did compare the IQs of children whose mothers had lived in fluoridated and nonfluoridated areas. There was no statistically significant difference – an important fact which was not discussed at all in the paper. This table was extracted from the paper’s Table 1.

What about that regression?

While ignoring the mean values for fluoridated and nonfluoridated areas the authors relied on regression analyses to determine an effect.

But if you look at the data  in their Figure 3A reproduced below you can see problems:

“. . . you can see that the average IQ of a child for a mother consuming 1.5 mg of fluoride is about 100. You also see that only ONE point is representing that average. That in itself is a huge problem because the sample size is small, and these individual measurements are influencing the model a lot, specially if their value is extreme. Because we’re dealing with averages, any extreme values will have a disproportionate influence on the average value.”

Several scientific commenters on this paper have noted this problem which is important because it should have been dealt with in the statistical analysis:

“When biostatisticians see these extreme values popping up, we start to think that the sample is not what you would call “normally distributed.” If that is the case, then a linear regression is not exactly what we want to do. We want to do other statistical analyses and present them along with the linear regressions so that we can account for a sample that has a large proportion of extreme values influencing the average. Is that the case with the Green study? I don’t know. I don’t have access to the full dataset. But you can see that there are some extreme values for fluoride consumption and IQ. A child had an IQ of 150, for example. And a mother consumed about 2.5 milligrams of fluoride per liter of beverage. Municipal water systems aim for 0.7 mg per liter in drinking water, making this 2.5 mg/L really high.”

No one suggests such outliers be removed from the analyses (although the authors did remove some). But they “should be looked at closely, through statistical analysis that is not just a linear regression.”

This is frustrating because while the authors did not do this they hint that it was considered (but do not produce results)  when they say:

“Residuals from each model had approximately normal distributions, and their Q-Q plots revealed no extreme outliers. Plots of residuals against fitted values did not suggest any assumption violations and there were no substantial influential observations as measured by Cook distance. Including quadratic or natural-log effects of MUFSG or fluoride intake did not significantly improve the regression models. Thus, we present the more easily interpreted estimates from linear regression models.”

As Dr. Najera comments, this is “.. worrisome because that is all they presented. They didn’t present the results from other models or from their sensitivity analysis.”

Scientific commenters are beginning to demand that the authors make the data available so they can check for themselves. My own testing with the data I extracted from the figure does show that the data is not normally distributed. Transformation produced a normal distribution of the data but the relationship was far weaker than for a straight linear regression. Did the authors reject transformations simply because they  “did not significantly improve the regression models?”

That suggests confirmation bias to me.

Confidence intervals

In their public promotions, the authors and their supporters never mention confidence intervals (CIs)- perhaps because the story does not look so good when they are considered. Most of the media coverage has also ignored these CIs.

A big thing is made for the IQ score of boys dropping by 4.49 points with a 1 mg/L increase in  mother’s urinary fluoride, but:

“Based on this sample, the researchers are 95% confident that the true drop in IQ in the population they’re studying is between 0.6 points and 8.38 points. (That’s what the 95% CI, confidence interval, means.)”

In other words:

“In boys, the change is as tiny as 0.6 and as huge as 8.38 IQ points.”

For girls the change:

“is between -2.51 (a decrease) and 7.36 (an increase). It is because of that last 95% CI that they say that fluoride ingestion is not associated with a drop in IQ in girls. In fact, they can’t even say it’s associated with an increase. It might even be a 0 IQ change in girls.”

Dr. Najera asks:

“Is this conclusive? In my opinion, no. It is not conclusive because that is a huge range for both boys and girls, and the range for girls overlaps 0, meaning that there is a ton of statistical uncertainty here. “

This is why the epidemiological design used by the authors is worrying. For example:

” The whole thing about not including women who did not drink tap water is troubling since we know that certain drinks have higher concentrations of fluoride in them. If they didn’t drink tap water, what are the odds that they drank those higher-fluoride drinks, and what was the effect of that?”

This comes on top of the problems with the regression models used.

Transformation to normalise the data and inclusion of other important facts may have produced a non-significant relationship and there would be no need for this discussion and speculation.

What about those other important factors?

Green et al (2019) included other factors (besides maternal urinary fluoride) in their statistical model. This “adjustment” helps check that the main factor under consideration is still statistically significant when other factors are included. In this case, the coefficient (and CIs) for the linear association for boys was reduced from -5.01  (-9.06 to -0.97) for fluoride alone to -4.49 (-8.38 to -0.60) when other considered factors were included. In this case, the other factors included race/ethnicity, maternal education, “city”, and HOME score (quality of home environment).

Dr. Najera questions the way other factors, or covariates, were selected for inclusion in the final model. He says:

“The authors also did something that is very interesting. They left covariates (the “other” factors) in their model if their p-value was 0.20. A p-value tells you the probability that the results you are observing are by chance. In this case, they allowed variables to stay in their mathematical model if the model said that there was as much as a 1 in 5 chance that the association being seen is due to chance alone. The usual p-value for taking out variables is 0.05, and even that might be a little too liberal.

Not only that, but the more variables you have in your model, the more you mess with the overall p-value of your entire model because you’re going to find a statistically significant association (p-value less than 0.05) if you throw enough variables in there. Could this be a case of P Hacking, where researchers allow more variables into the model to get that desired statistical significance? I hope not.”

Good point. I myself was surprised at the use of such a large p-value for selection. And, although the study treats fluoride as the main factor and inclusion of the other factors reduces the linear coefficient for maternal urinary fluoride, I do wonder why more emphasis was not put on these other factors which may contribute more to the IQ effect than does fluoride.

Perhaps this paper should have concentrated on the relationship of child IQ with race or maternal education rather than with fluoride.

Padding out to overcome the poor explanation of IQ variance

Another point about the inclusion of these covariates. As well as possible improving the statistical significance of the final model they may also make the model look better in terms of the ability to explain the variance in IQ (which is very large – see figure above).

In my first critique of the Green et al (2019) paper (If at first you don’t succeed . . . statistical manipulation might help) I pointed out that the reported relationship for boys, although statistically significant, explained very little of the variance in IQ. I found only 1.3% of the variance was explained – using data I had digitally extracted from the figure. This was based on the R-squared value for the linear regression analysis.

Unfortunately, the authors did not provide information like R-squared values for their regression analysis (poor peer review in my opinion) – that is why I, and others, were forced to extract what data we could from the figures and estimate our own. Later I obtained more information from  Green’s MA thesis describing this work (Prenatal Fluoride Exposure and Neurodevelopmental Outcomes in a National Birth Cohort). Here she reported an R-squared value of 4.7%. Bigger than my 1.3% (my analysis suffered from not having all the data) but still very small. According to Nau’s (2017) discussion of the meaning of R-squared values (What’s a good value for R-squared?), ignoring the coefficient determined by Green et al (2019) (5.01) and relying only on the constant in the relationship would produce a predicted value of IQ almost as good (out by only about 2%).

That is, simply taking the mean IQ value (about 114.1 according to the figure above) for the data would be almost as good as using the relationship for any reasonable maternal urinary fluoride value and OK for practical purposes.

But look at the effect of including other factors in the model. Despite lowering the coefficient of the relationship for fluoide it drastically increases the R-squared value. Green reported a value of 22.0% for her final model. Still not great but a hell of a lot better than 4.7%.

Perhaps the inclusion of so many other factors in a multiple regression makes the final model look much better – and perhaps that perception is unjustly transferred to the relationship with fluoride.

Are other more important factors missed?

Almost certainly – and that could drastically alter to conclusions we draw from this data. The problem is that fluoride can act as a proxy for other factors. City location and size are just one aspect to consider.

In my paper Fluoridation and attention deficit hyperactivity disorder a critique of Malin and Till (2015), I showed inclusion of altitude as a risk-modifying factor completely removed any statistical significance from the relationship between ADHD prevalence and fluoridation – despite the fact Malin & Till (2015) had reported a significant relationship with R-squared values over 30%!

Malin & Till (2015) reported these relationships as statistically significant. However, when altitude was included in the multiple regressions by Perrott (2018) no significant relationships were fluoridation were found.

So you can see the problem. Even though authors may list a number of factors or covariates they “adjusted” for, important risk-modifying factors may well be ignored in such studies. This is not to say that inclusion of them “proves” causation any more than it does for fluoride. But if their inclusion leads to the disappearance of the relationship with fluoride one should no longer claim there is one (reviewers related to the group involved in the Green et al., 2019 study still cite Malin & Till 2015 as if their reported relationship is still valid).

In effect, the authors acknowledge this with their statement:

“Nonetheless, despite our comprehensive array of covariates included, this observational study design could not address the possibility of other unmeasured residual confounding.”

Summary

Dr. Najera summarises his impression of the Green et al (2019) study in these words:

“The big idea of these three blog posts was to point out to you that this study is just the latest study that tries very hard to tie a bad outcome (lower IQ) to fluoride, but it really failed to make that case from the epidemiological and biostatistical approaches that the researcher took, at least in my opinion. Groups were left out that shouldn’t. Outliers were left in without understanding them better. A child with IQ of 150 was left in, along with one mother-child pair of a below-normal IQ and very high fluoride, pulling the averages in their respective directions. The statistical approach was a linear regression that lumped in all of the variables instead of accounting for different levels of those variables in the study group. (A multi-level analysis that allowed for the understanding of the effects of society and environment along with the individual factors would have been great. The lack of normality in the distribution of outcome and exposure variables hint at a different analysis, too.)”

Pretty damning!

Similar articles

Anti-fluoridationists rejection of IQ studies in fluoridated area.

US anti-fluoride activist Paul Connett claims studies cannot detect an IQ effect from fluoridated water because total fluoride intake is the real problem – but still campaigns against community water fluoridation. Image credit: MSoF “Activist Spouts Nonsense – The Evidence Supports Fluoridation”

This is another article in my critique of the presentation Paul Connett prepared to present to a meeting at Parliament in February.

I deal with his coverage of the studies of IQ effects where community water fluoridation (CWF) is used. There are now actually three such studies (Broadbent et al. 2015, Barberio et al. 2017  and  Aggeborn & Öhman 2016), but Connett pretends there is only one – the Broadbent et al. (2015) New Zealand study.

Maybe because it was the first one to provide evidence challenging his extrapolation of the fluoride/IQ studies (see The 52 IQ studies used by anti-fluoride campaigners) results in areas of endemic fluorosis to areas where CWF is used. It is also the study which seems to have resulted in the most hostility from anti-fluoride campaigners.

So here I will just be sticking with his criticism of the New Zealand study Broadbent et al (2015):

Slide 76 from Paul Connett’s presentation prepared for his February meeting at  parliament buildings

Broadbent’s findings do not “negate all other human studies”

Paul allows emotion to get the better of him as no one is suggesting this at all. The studies Connett refers to are all from areas of endemic fluorosis (see  The 52 IQ studies used by anti-fluoride campaigners), not from areas of CWF.

Broadbent et al (2015) simply concluded that their “findings do not support the assertion that fluoride in the context of CWF programmes is neurotoxic.”  That is a modest statement and Broadbent et al. (2015) simply do not draw any conclusions about the studies Connett relies on. But, of course, Connett is upset because this and similar studies just do not support his attempt to extrapolate results from areas of endemic fluorosis to areas of CWF.

The health problems suffered by people in areas of endemic fluorosis are real and it is right they should be studied and attempts made to alleviate them. But this has absolutely nothing to do with CWF.

“Fatally flawed” charge is itself fatally flawed

Again, Paul has allowed emotions to get the upper hand. It is possible, and necessary, to critique published papers – but critiques should be evidence-based and realistic. Paul’s “fatally flawed” charge (slides 77 & 78) simply displays how much this paper has put his nose out of joint.

But let’s look at the specific “flaws” Paul (and other critics associated with the Fluoride Action Network) claim.

The two villages mindset: Paul alleges that the Broadbent et al (2015) study “essentially compared two groups.” He is stuck in the mindset of most of his 52  studies from areas of endemic fluorosis (see  Fluoride & IQ: The 52 Studies). The mindset of simply comparing the IQ levels of children in a village suffering endemic fluorosis with the IQ levels of children in a village not suffering endemic fluorosis. This simple approach can identify statistically significant differences between the villages but provides little information on causes. For example, most of these studies used drinking water fluoride as a parameter but there could be a whole range of other causes related to health problems of fluorosis.

Professor Richie Poulton, current Director of the Dunedin Multidisciplinary Health and Development Research Unit

In contrast, Broadbent et al. (2015) used “General Linear models to assess the association between CWF and IQ in childhood and adulthood, after adjusting for potential confounders.” The statistical analysis involved includes accounting for a range of possible risk-modifying factors besides CWF., This was possible because the study was part of the Dunedin Multidisciplinary Health and Development Study. This is a highly reputable long-running cohort study of 1037 people born in 1972/1973 with information covering many areas.

The fluoride tablets argument: Connett and other critics always raise this issue – the fact that “In New Zealand during the 1970s, when the study children were young, F supplements were often prescribed to those living in unfluoridated areas.” Often they will go further to claim that all the children in the unfluoridated area of this study were receiving fluoride tablets – something they have no way of knowing.

But the fact remains that fluoride tablets were included in the statistical analysis. No statistically significant effect was seen for them.  Overlap of use of fluoride tablets with residence in fluoridated or unfluoridated areas will have occurred and their influence would be reflected in the results found. Presumably, the effect would be to increase the confidence intervals. As the critics, Menkes et al. (2014), say “comparing groups with overlapping exposure thus compromises the study’s statistical power to determine the single effect of CWF.”  I agree. But this does not negate the findings which are reported with the appropriate confidence intervals (see below).

The point is that the simplistic argument that effects of fluoride tablets were ignored is just not correct. Their effect is reflected in the results obtained.

Potential confounders: Many poor quality studies have ignored possible confounders, or considered only a few. This is a general problem with these sort of studies – and even when attempts are made to include all that the researchers consider important a critic can always claim there may be others – especially if they do not like the results. Claims of failing to consider confounders can often be simply the last resort of armchair critics.

In this case, there is no actual reported association to be confounded (unlike my identification of this problem with the Malin & Till 2015 ADHD study – see Perrott 2017). However, Osmunson et al. (2016) specifically raised possibilities of confounding by lead, manganese, mother’s IQ and rural vs urban residence. Mekes et al. (2014) also raised the rural vs urban issue as well as a possible effect from breastfeeding reducing fluoride intake by children in fluoridated areas.  In their response, Broadbent et al (2015b & 2016) reported that a check showed no significant effect of lead or distance from the city centre and pointed out that manganese levels were too low to have an effect. Broadbent et al (2015b) also reported no significant breastfeeding-fluoride interaction occurred.

Numbers involved: Connett claims the study was fatally flawed because “it had very few controls: 991 lived in the fluoridated area, and only 99 in non-fluoridated” (Slide 77). But the numbers are simply given by the longer term Dunedin study themselves – they weren’t chosen by Broadbent and his co-workers. That is the real world and is hardly a “fatal flaw.”

The 95% confidence intervals

Yes, statisticians always love to work with the large numbers but in the real world, we take what we have. Smaller numbers mean less statistical confidence in the result – but given that Broadbent et al (2015) provides the results, together with confidence intervals, it is silly to describe this as fatally flawed. These were the results given in the paper for the parameter estimate of the factors of interest:

Factor	Parameter estimate	95% Confidence interval	p-value
Area of residence	-0.01	-3.22 to 3.20	.996
Fluoride toothpaste use	0.70	-1.03 to 2.43	.428
Fluoride tablets	1.55	-0.38 to 3.49	.116

Connett did not refer to the confidence intervals reported by Broadbent et al (2015). However, Grandjean and Choi (2015) did describe them as “wide” – probably because they were attempting to excuse the extrapolation of “fluoride as a potential neurotoxic hazard” from areas of endemic fluorosis to CWF.

The argument over confidence intervals can amount to straw clutching – a “yes but” argument which says “the effect is still there but is small and your study was not large enough to find it.” That argument can be never ending but it is worth noting that Aggeborn & Öhman (2016) made a similar comment about wide confidence intervals for all fluoride/IQ studies, including that of Broadbent et al. (2015).  Aggeborn & Öhman (2016) had a very large sample (almost 82,000 were involved in the cognitive ability comparisons) and reported confidence intervals of -0.18 to 1.03 IQ points (compared with -3.22 to 3.20 IQ points reported by Broadbent et al 2015). Based on this they commented, “we are confident to claim that we have estimated a zero-effect on cognitive ability.”

The “yes but” argument about confidence intervals may mean one is simply expressing faith in an effect so small as to be meaningless.

Total fluoride exposure should have been used: Connett says (slide 77) “Broadbent et al did not use the proper measure of fluoride exposure. They should have used total F exposure.  Instead, they used only exposure from fluoridated water.” Osmunson et al. (2016) make a similar point, claiming that the study should not have considered drinking water fluoride concentration but total fluoride intake. They go so far as to claim “the question is not whether CWF reduces IQ, but whether or not total fluoride intake reduces IQ.”

This smacks of goalpost moving – especially as the argument has specifically been about drinking water fluoride and most of the studies they rely on from areas of endemic fluorosis specifically used that parameter.

In their response to this criticism Broadbent et al (2016) calculated estimates for total daily fluoride intake and used them in their analysis which “resulted in no meaningful change of significance, effect size, or direction in our original findings.”

It’s interesting to note that Connett and his co-workers appear to miss completely the point about “wide” confidence intervals made by Grandjean and Choi (2015). Instead, they have elevated their argument to the claim that fluoride intake is almost the same in both fluoridated and unfluoridated areas so that any study will not be able to detect a difference in IQ. Essentially they are claiming that we are all going to suffer IQ deficits whether we live in fluoridated or unfluoridated areas.

This is the central argument of their paper – Hirzy et al (2016). However, the whole argument relies on their own estimates of dietary intakes – a clear example where motivated analysts will make the assumptions that fit and support their own arguments. This argument also fails to explain why the Dunedin study found lower tooth decay in fluoridated areas.

Last time I checked the anti-fluoride campaigners, including Connett, were still focusing on CWF – fluoride in drinking water. One would think if they really believed their criticism that they would have given up that campaign and instead devoted their energies to the total fluoride intake alone.

Conclusions

All studies have limitations and of course, Broadbent et al. (2015) is no exception. However, the specific criticisms made by Connett and his fellow critics do not stand up to scrutiny. Most have been responded to and shown wrong – mind you this does not stop these critics from continuing to repeat them and disregard the responses.

I believe the relatively wide confidence intervals could be a valid criticism – although it does suggest a critic who is arguing for very small effects. A critic who may always find the confidence intervals still exclude their very small effect – no matter how large the study is.

In effect, the narrow confidence intervals reported by Aggeborn & Öhman (2016) should put that argument to rest for any rational person.

References

Aggeborn, L., & Öhman, M. (2016). The Effects of Fluoride In The Drinking Water. 

Barberio, A. M., Quiñonez, C., Hosein, F. S., & McLaren, L. (2017). Fluoride exposure and reported learning disability diagnosis among Canadian children: Implications for community water fluoridation. Can J Public Health, 108(3),

Broadbent, J. M., Thomson, W. M., Ramrakha, S., Moffitt, T. E., Zeng, J., Foster Page, L. A., & Poulton, R. (2015). Community Water Fluoridation and Intelligence: Prospective Study in New Zealand. American Journal of Public Health, 105(1), 72–76.

Broadbent, J. M., Thomson, W. M., Moffitt, T., Poulton, R., & Poulton, R. (2015b). Health effects of water fluoridation: a response to the letter by Menkes et al. NZMJ, 128(1410), 73–74.

Broadbent, J. M., Thomson, W. M., Moffitt, T. E., & Poulton, R. (2016). BROADBENT ET AL. RESPOND. American Journal of Public Health, 106(2), 213–214. https://doi.org/10.2105/AJPH.2015.302918

Grandjean, P., Choi, A. (2015). Letter: Community Water Fluoridation and Intelligence. Am J Pub Health, 105(4).

Hirzy, J. W., Connett, P., Xiang, Q., Spittle, B. J., & Kennedy, D. C. (2016). Developmental neurotoxicity of fluoride: a quantitative risk analysis towards establishing a safe daily dose of fluoride for children. Fluoride, 49(December), 379–400.

Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14.

Menkes, D. B., Thiessen, K., & Williams, J. (2014). Health effects of water fluoridation — how “ effectively settled ” is the science? NZ Med J, 127(1407), 84–86.

Osmunson, B., Limeback, H., & Neurath, C. (2016). Study incapable of detecting IQ loss from fluoride. American Journal of Public Health, 106(2), 212–2013.

Perrott, K. W. (2017). Fluoridation and attention deficit hyperactivity disorder – a critique of Malin and Till ( 2015 ). Br Dent J.

Similar articles

 

 

Fluoridation means money in the pocket

Local researchers recently presented data showing that the ordinary person, and not the taxation financed health system, is the main financial beneficiary of community water fluoridation.

Their research confirmed that community water fluoridation in New Zealand is highly cost-effective for all but the smallest communities. This study updates previous evaluations by including data for adults – previous studies were limited to children. It also corrected for under-estimation of averted dental restoration costs in a previous study.

The authors also make the point that an update is necessary because:

“Sound public health practice requires periodic re-evaluation of interventions’ benefits and costs.”

The results are reported in the paper:

Moore, D., Poynton, M., Broadbent, J. M., & Thomson, W. M. (2017). The costs and benefits of water fluoridation in NZ. BMC Oral Health, 17(1), 134.

Community size

As with previous studies, the results confirmed that fluoridation is not cost effective for very small communities because of the capital cost of fluoridation plants and the use of sodium fluoride instead of fluorosilicic acid as the fluoridating chemical in small plants. However:

“For ‘minor’ through to ‘large’ plants, there is a net cost saving. For a ‘large’ plant supplying 50,000 people, the cost offsets are over 20 times the cost of fluoridation. The break-even point appears to be reached by ‘minor’ plants supplying a population over 500.”

National net savings from universal fluoridation

The authours estimated the national costs and saving from averted ental costs over a 20 year period. If all New Zealand reticulated water supplies serving populations greater than 500 were fluoridated costs over 20 years would amount to$177 million while the cost offset due to averted dental treatment costs would be $1578 million.

The national 20-year net saving due to such universal community water fluoridation in NZ would amount to $1401 million.

That is a nine times pay-off!

Individuals save more than the state

I hadn’t thought of this before but the data enables separate estimates of savings to the state from universal CWF through reduced costs to the health budget, and to the individual citizen through their reduced costs for private dental treatment.

In fact, the major benefit is to the individual rather than the health budget.  National savings over 20 years for reduction of private dental care expenditure would be $1428 million – 10 times the savings to the national health budget.

Perhaps this helps people understand that they, personally, have something to gain fiancially from community water fluoridation

Similar articles

 

 

Fluoridation not associated with ADHD – a myth put to rest

Fluoridated water is NOT associated with ADHD: Photo by mtl_moe

The myth of community water fluoridation causing attention deficit hyperactivity disorder (ADHD) is just not supported by the data. I show this in a new paper accepted for publication in the British Dental Journal. This should remove any validity for the claims about ADHD by anti-fluoride campaigners.

Mind you, I do not expect them to stop making those claims.

The citation for this new paper is (will be):

Perrott, K. W. (2017). Fluoridation and attention hyperactivity disorder – a critique of Malin and Till. British Dental Journal. In press.

The Background

The fluoridation causes ADHD myth was initially started by the publication of Malin & Till’s paper in 2015:

Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14.

It was quickly taken up and promoted by anti-fluoride campaigners – becoming one of their most cited papers when claiming harmful psychological effects from fluoridation. Part of the reason for its popularity is that it is the only published paper reporting an association between community water fluoridation (CWF) incidence and the prevalence of a psychological deficit. All other reports on this used by anti-fluoride campaigners are based on studies made in high fluoride regions like China where fluorosis is endemic. Those studies are just not relevant to CWF.

While many critics rejected Malin & Till’s conclusions on the simple basis that correlation does not mean causation I decided to look a bit deeper and test their statistical analyses. This was easy because they used published US data for each US state and such data is available for many factors.

I posted my original findings in the article ADHD linked to elevation not fluoridation. This showed that a number of factors were independently associated with ADHD prevalence (eg., home ownership, poverty, educational attainment, personal income, and % of the population older than 65) and these associations were just as significant statistically as the associaiton reported by Malin & Till.

However, multiple regression of possible modifying factors showed no statistically significant of ADHD prevalence with CWF incidence when mean state elevation was includedd.

The importance of elevation was confirmed by Huber et al. (2015):

Huber, R. S., Kim, T.-S., Kim, N., Kuykendall, M. D., Sherwood, S. N., Renshaw, P. F., & Kondo, D. G. (2015). Association Between Altitude and Regional Variation of ADHD in Youth. Journal of Attention Disorders.

Huber et al., (2015) did not include CWF incidence in their analyses. I have done this with the new paper in the British Dental Journal.

Publication problems

I firmly believe that scientific journals, like  Environmental Health which published the Malin & Till paper, have an ethical obligation to accept critiques of papers they publish (subject to peer review of course). Similarly, it is appropriate that any critique of a published paper is made in the journal where it was originally published. Implicit in this arrangement, of course, is that the authors of the original paper get the chance to respond to any critique and that the response be published by the original journal.

Unfortunately, this was not possible for this paper because the Chief Editor of  Environmental Health,  Prof Philippe Grandjean, simply refused to allow this critique to be considered for publication. No question of any peer reviuew. In his rejection he wrote:

“Although our journal does not currently have a time limit for submission of comments on articles published in EH, we are concerned that your response appears a very long time after the publication of the article that you criticize. During that period, new evidence has been published, and you cite some of it. There are additional studies that would also have to be taken into regard in a comprehensive comment, as would usually be the case after two years. In addition, the way the letter is written makes us believe that the letter is part of a controversy, and our journal is certainly not the appropriate forum for a dispute on fluoride policies.”

My response pointed out the reasons for the time gap (problems related to the journals large publication fee), that no other critique of the Malin & Till paper had yet been published and that any perceived polemics in the draft should normally be attended to by reviewers. This was ignored by Grandjean.

While Grandjean’s rejection astounded me – something I thought editors would consider unethical – it was perhaps understandable. Grandjean is directly involved as an author of several papers that activists use to criticise community water fluoridation. Examples are:

• Grandjean, P., & Landrigan, P. J. (2014). Neurobehavioural effects of developmental toxicity. Lancet Neurol, 13(March), 330–338.
• Choi, A. L., Sun, G., Zhang, Y., & Grandjean, P. (2012). Developmental fluoride neurotoxicity: A systematic review and meta-analysis. Environmental Health Perspectives, 120(10), 1362–1368.
• Choi, A. L., Zhang, Y., Sun, G., Bellinger, D., Wang, K., Yang, X. J., … Grandjean, P. (2015). Association of lifetime exposure to fluoride and cognitive functions in Chinese children: A pilot study. Neurotoxicology and Teratology, 47, 96–101.

Grandjean is part of the research group that has published data on IQ deficits in areas of endemic fluorosis – studies central to the anti-fluoride activist claims that CWF damages IQ.  He has also often appears in news reports supporting research findings that are apparently critical of CWF so has an anti-fluoridation public standing.

In my posts Poor peer-review – a case study and Poor peer review – and its consequences I showed how the peer review of the original Malin & Till paper was one-sided and inadequate. I also provided a diagram (see below) showing the relationship of Grandjean as Chief Editor of the Journal, and the reviewers as proponents of chemical toxicity mechanisms of IQ deficits.

So, I guess a lesson learned. But the unethical nature of Grandjean’s response did surprise me.

I then submitted to paper to the British Dental Journal. It was peer-reviewed, revised and here we are.

The guts of the paper

This basically repeated the contents of my article ADHD linked to elevation not fluoridation. However, I tried to use Malin &Till’s paper as an example of problems in ecological or correlation studies. In particular the inadequate consideration of possible risk-modifying factors. Malin & Till clearly had a bias against CWF which they confirmed by limiting the choice of covariates that might show them wrong. I agree that a geographic factor like altitude may not have been obvious to them but their discussion showed a bias towards chemical toxicity mechanisms – even though other social factors are often considered to be implicated in ADHD prevalence.

Unfortunately, Malin & Till’s paper is not an isolated example. Another obvious example of confirmation bias is that of Peckham et al., (2015). They reported an association of hypothyroidism with fluoridation but did not include the most obvious example of iodine deficiency as a risk-modifying factor in their statistical analysis

Of course, anti-fluoride campaigners latched on to the papers of Peckham et al., (2015) and Malin & Till (2015) to “prove” fluoridation was harmful. I guess such biased use of the scientific literature simply to be expected from political activists.

However,  I also believe the scientific literature contains many other examples where inadequate statistical analyses in ecological studies have been used to argue for associations which may not be real. Such papers are easily adopted by activists who are arguing for or against specific social policies or social attitudes. For example, online articles about religion will sometimes refer to published correlations of religosity with IQ, educational level or scoio-economic status. Commenters simply select the studies which confirm the bias they are arguing for.

These sort of ecological or corellations studies can be useful for developing hypotheses for future study but it is wrong to use them to support an argument and worse as “proof” of an argument.

Take home message

1. There is no statistically significant association of CWF with ADHD prevalence. Malin & Till’s study was flawed by lack of consideration of other possible risk-modifying factors;
2. Be very wary of ecological or correlation studies.Correlation is not evidence for causation and many of these sudues iognore other possible important risk-modifying factors.

Similar articles