Recently I suggested that Attention Deficit Hyperactivity Disorder (ADHD) was better correlated with elevation than with community water fluoridation (see ADHD linked to elevation not fluoridation). I criticised the study of Malin and Till (2015) for limiting their investigation to a chemical toxicity hypothesis and pointed out that once confounding factors like elevation are included their reported relationship between ADHD and community water fluoridation (CWF) disappears.
Seems I am not the only one to notice this. A new paper reports that same relationship:
They used data sets for the prevalence of ADHD in 2007 and 2010 in US states and found a negative relationship with average state elevation. Their correlation coefficients (R2 = .38, p < .001; R2 = .31, p < .001 respectively) are similar to the one I found.
“I do not think Malin and Till (2015) are justified in drawing the conclusion that CWF influences ADHD. Their mistaken conclusion has arisen from their limited choice of data considered for the exploratory analysis. That in itself seems to have resulted from a bias inherent in their hypothesis that “fluoride is a widespread neurotoxin.”
I was not advancing an alternative hypothesis but Huber et al., (2015) did suggest the hypothesis:
“As decreased dopamine (DA) activity has been reported with ADHD and hypoxia has shown to be associated with increased DA, we hypothesized that states at higher altitudes would have lower rates of ADHD.”
But the important lesson is once factors like elevation are taken into account there is no statistically significant relationship with CWF. The Malin & Till (2015) paper currently heavily promoted by anti-fluoride propagandists is flawed.
Fluoride Free NZ (FFNZ) promotes a list of “NZ Health Professionals who are calling for an end to fluoridation.” I am generally cynical about such endorsement lists, but the details in this list do give a picture of the commercial and ideological alignment of the FFNZ supporters and activists. So I did my own analysis, dividing the list into those described as “Science and Environmental PhD Professionals”, “NZ Dentists, “NZ Doctors” and Alternative health professionals (Chiropractors, naturopaths, Homeopaths, etc.). Of course, this is approximate as, for example, some listed as doctors may have specialised in one or another alternative fields. The pie chart below shows the distribution of FFNZ supporters among these groups. Clearly with such a large proportion of supporters coming from alternative health fields this above distribution is not representative of professionals in general, let alone health professionals. However, anyone who has looked at the anti-fluoride movement or debated with anti-fluoride activists would not be surprised as “natural”/alternative health arguments and sources are frequently used. I wonder, though, to what extent local body councillors are aware of this commercial and ideological orientation when considering submissions they get on the fluoridation issue. I suspect they aren’t. Yet groups like FFNZ engineer these submissions from their supporters – often providing templates for individuals to sign – and usually dominate the submission process. Personally I think this is a defect in our system of representative democracy – councils should actually insist on declarations of conflict of interest, details of employment and commercial interests from submitters. Their failure to do this explains how some local bodies, like the Hamilton City Council, have unwittingly been captured by ideological and commercial interests from the “natural”/alternative health industry during such submission processes.
Declaration of conflicts of interest and details of employment, etc., may to some extent help identify big business interests financing this sort of submission in future. At the moment, we are largely left to speculate. However, there are financial data available showing the money trail involved in at least one anti-fluoride campaign – the High Court case against the South Taranaki District Council aiming for a judicial review of a decision to fluoridate water supplies in Patea and Waverley (see Who is funding anti-fluoridation High Court action? and Corporate backers of anti-fluoride movement lose in NZ High Court). This action was taken by New Health NZ – an incorporated body set up by the NZ Health Trust – In November 2013. Statements of financial performance of these two organisations are available online and show the following movements of large amounts of money during the year to March 2014. As the NZ health Trust is a lobby group for the “natural”/alternative health industry the grants it receives must come out of the profits of this industry which is actually a big business in New Zealand. Although the financial statements do not identify sources and recipients the $100,00 grant to New Health NZ clearly came from its parent body and is included in their declared $125,ooo grants and donations. The $95,156 paid out by New Health NZ in professional and consulting fees would have covered the costs involved in their High Court action. So this is a clear example of pretty direct funding of anti-fluoride activity (the High Court action) by corporate interests – the “natural”/alternative health industry. But none of the reporting of this High Court action identified the commercial interests involved. Readers were given the impression that New Health NZ was just another one of these anti-fluoride activist groups and possibly assumed funds for the legal action came from donations. Again, this is a flaw in our representative democratic system. There should be more transparency of financial links. Corporate interests should be able to hide behind astroturf organisations and the dishonesty that their actions are the result of concerned citizens and not the ideological and commercial interests of big business. Similar articles
Anti-fluoride propagandists like Declan Waugh and Paul Connett avidly scan the scientific literature looking for anything they can present as evidence for harmful effects of community water fluoridation (CWF). Sometimes they will even do their own “research” using published and on-line health data looking for any correlations with CWF, or even just with fluoride levels in drinking water.
Several years ago an activist going under the nom de plume “Fugio” posted images showing correlations of mental retardation, adult tooth loss and ADHD with the incidence of CWF in the US. These images are simply the result of “research” driven by confirmation bias and data dredging.They prove nothing. Correlation is not proof of a cause. And no effort was made to see if other factors could give better correlations.
This figure is essentially the same as that reported by Malin & Till (2015). In fact, I wonder if Fugio (who posted December 2012) is the unattributed source of Malin & Till’s hypothesis. Fugio chose the ADHD data for 2007 and fluoridation data for 2006 whereas Malin and Till (2015) concentrated mainly on fluoridation data for 1992 which had the highest correlation with ADHD figures.
I won’t discuss this further here – my earlier article ADHD linked to elevation not fluoridation shows there are a number of other factors which correlate with ADHD prevalence just as well or better than CWF incidence does and should have at least been considered as confounding if not the main factors. I found a model using mean elevation, home ownership and poverty only (no CWF included) explained about 48% of the variation whereas their model using CWF and mean income explained only 22-31% of the variation. And when these confounder factors were considered the correlation of ADHD with CWF was not statistically significant.
In other words we could do a far better job of predicting ADHD prevalence without involving CWF.
Checking out correlations with a range of factors I found a model involving only smoking and longitude explaining about 74% of the variation. The contribution from CWF was not significant statistically – it added nothing to this model.
Water Fluoridation and Mental Retardation
Fugio found a better relationship between CWF in 1992 and mental retardation in 1993 – a correlation explaining 19% of the variation. Apparently the concept of “mental retardation” was later abandoned as there do not appear to be any more recent statistics.
But again, if Fugio had not stopped there he/she would have found a number of other factors with better correlations. I give an example in the figure where state educational level (% Bachelors Degree in 1993) explained 50% if the variation. This correlation is negative as we might expect.
Again I used multiple regression analysis to derive a model involving educational level (% with Bachelors degree in 1993), poverty in 1993 and mean state elevation which explained 69% of the variation. No statistically significant contribution from CWF occurred.
I am not suggesting here that the factors I identified have a causal effect. Simply that they give better correlations than CWF. These and similar confounding factors should have been considered by Fugio and Malin and Till (2015).
My purpose is to show that this sort of exploratory analysis of easily available data can easily produce results for anti-fluoride activists who are searching for some “sciency” looking arguments to back up their position. Provided they don’t look too deeply, stop while they are ahead and refuse to consider the influence of other factors.
Unfortunately poor peer review by some journals is allowing publication of work that is no better than this. Peckham et al (2015) did nothing to check out other factors except gender in their correlations of hypothyroidism with CWF. The glaring omission was of course dietary iodine which is known to have a causative link with hypothyroidism. (I could not find US data for hypothyroidism so was unable to check out Peckham et al’s hypothesis for the US.) Malin and Till (2015) included only socioeconomic status (as indicated by income) in their analysis despite the fact that ADHD is known to be related to a number of factors like smoking and alcohol intake.
As I keep saying, when it comes to understanding the scientific literature it really is a matter of “reader beware.” It’s easy to find papers supporting one’s pet obsession if you are not critical and sensible with your literature searches. And it is important not to take at face value the claims of activists who clearly rely on confirmation bias when they explore the literature.
IQ data for US states are not readily available but I managed to find a data set of IQ estimates by state in 2000 based on Scholastic Aptitude Test scores. The correlation of these average IQ scores with water fluoridation (1992) is not at all significant statistically. The slope of the trend line in the plot below is not significantly different to zero (-0.04 to +0.01 at the 95% confidence level as represented by the dashed lines).
This lack of correlation is not at all surprising. After all, the only published study to compare IQ and community water fluoridation (CWF) is that of Broadbent et al., (2014) – they also did not find any statistically signficant relationship.
So what are the anti-fluoride propagandists on about?
They do not rely on studies involving CWF but instead claim support in studies from areas where fluorosis due to excess fluoride is endemic – eg Choi et al., (2012). These and similar studies have reported a correlation of IQ with drinking water fluoride- but there are 2 problems:
Very little was done in these studies to consider confounding factors. There is the possibility that inclusion of these confounding factors in correlations would show that fluoride does not make a statistically signficant contribution to IQ changes.
Generally the authors have assumed a chemical toxicity explanation without any real justification. The data can be explained by other mechanisms such as the influence of the disfiguring effect of severe dental fluorosis on quality of life and learning (Perrott, 2015). In the few cases where data for severe dental fluorosis was included its relationship with IQ is statistically significant (eg Choi et al., 2015) (see Severe dental fluorosis the real cause of IQ deficits?). Severe dental fluorosis is not a problem in areas where CWF is used.
There is no need to consider confounding factors for the correction in the above figure as CWF does not explain any of the variation in IQ. But I did find statistically significant relationship for IQ with a number of factors. The plots below show the data for premature births in 1990-1991 and average percent poverty in 2002-2004. These correlations by themselves explain 50 and 63% of the variation in IQ. Combined they explain 69% of the variation.
The percentage of CWF in each state explains none of the variation.
It would be more rational for those concerned about CWF to get active on issues related to poverty and premature births.
The community water fluoridation issue is a dead duck as far as IQ is concerned.
This has been a common theme here as I have campaigned against cherry-picking research papers, relying on confirmation bias and putting blind faith in peer-review as a guarantee of research quality.
In short I have pleaded for readers to approach published research critically and intelligently.
The article The 10 stuff-ups we all make when interpreting research from The Conversation gives some specific advice on how to do this. Well worth keeping in mind when you next set out to scan the literature to find the current state of scientific knowledge on a subject that interests you.
UNDERSTANDING RESEARCH: What do we actually mean by research and how does it help inform our understanding of things? Understanding what’s being said in any new research can be challenging and there are some common mistakes that people make.
Have you ever tried to interpret some new research to work out what the study means in the grand scheme of things?
Well maybe you’re smart and didn’t make any mistakes – but more likely you’re like most humans and accidentally made one of these 10 stuff ups.
1. Wait! That’s just one study!
You wouldn’t judge all old men based on just Rolf Harris or Nelson Mandela. And so neither should you judge any topic based on just one study.
If you do it deliberately, it’s cherry-picking. If you do it by accident, it’s an example of the exception fallacy.
The well-worn and thoroughly discredited case of the measles, mumps and rubella (MMR) vaccine causing autism serves as a great example of both of these.
People who blindly accepted Andrew Wakefield’s (now retracted) study – when all the other evidence was to the contrary – fell afoul of the exception fallacy. People who selectively used it to oppose vaccination were cherry-picking.
2. Significant doesn’t mean important
Some effects might well be statistically significant, but so tiny as to be useless in practice.
Associations (like correlations) are great for falling foul of this, especially when studies have huge number of participants. Basically, if you have large numbers of participants in a study, significant associations tend to be plentiful, but not necessarily meaningful.
One example can be seen in a study of 22,000 people that found a significant (p<0.00001) association between people taking aspirin and a reduction in heart attacks, but the size of the result was miniscule.
The difference in the likelihood of heart attacks between those taking aspirin every day and those who weren’t was less than 1%. At this effect size – and considering the possible costs associated with taking aspirin – it is dubious whether it is worth taking at all.
3. And effect size doesn’t mean useful
We might have a treatment that lowers our risk of a condition by 50%. But if the risk of having that condition was already vanishingly low (say a lifetime risk of 0.002%), then reducing that might be a little pointless.
We can flip this around and use what is called Number Needed to Treat (NNT).
In normal conditions if two random people out of 100,000 would get that condition during their lifetime, you’d need all 100,000 to take the treatment to reduce that number to one.
4. Are you judging the extremes by the majority?
Biology and medical research are great for reminding us that not all trends are linear.
We all know that people with very high salt intakes have a greater risk of cardio-vascular disease than people with a moderate salt intake.
The graph is U shaped, not just a line going straight up. The people at each end of the graph are probably doing different things.
5. Did you maybe even want to find that effect?
Even without trying, we notice and give more credence to information that agrees with views we already hold. We are attuned to seeing and accepting things that confirm what we already know, think and believe.
There are numerous example of this confirmation bias but studies such as this reveal how disturbing the effect can be.
In this case, the more educated people believed a person to be, the lighter they (incorrectly) remembered that person’s skin was.
6. Were you tricked by sciencey snake oil?
You won’t be surprised to hear that sciencey-sounding stuff is seductive. Hey, even the advertisers like to use our words!
But this is a real effect that clouds our ability to interpret research.
In one study, non-experts found even bad psychological explanations of behaviour more convincing when they were associated with irrelevant neuroscience information. And if you add in a nice-and-shiny fMRI scan, look out!
7. Qualities aren’t quantities and quantities aren’t qualitites
For some reason, numbers feel more objective than adjectivally-laden descriptions of things. Numbers seem rational, words seem irrational. But sometimes numbers can confuse an issue.
For example, we know people don’t enjoy waiting in long queues at the bank. If we want to find out how to improve this, we could be tempted to measure waiting periods and then strive to try and reduce that time.
But in reality you can only reduce the wait time so far. And a purely quantitative approach may miss other possibilities.
If you asked people to describe how waiting made them feel, you might discover it’s less about how long it takes, and more about how uncomfortable they are.
8. Models by definition are not perfect representations of reality
A common battle-line between climate change deniers and people who actually understand evidence is the effectiveness and representativeness of climate models.
But we can use much simpler models to look at this. Just take the classic model of an atom. It’s frequently represented as a nice stable nucleus in the middle of a number of neatly orbiting electrons.
While this doesn’t reflect how an atom actually looks, it serves to explain fundamental aspects of the way atoms and their sub-elements work.
This doesn’t mean people haven’t had misconceptions about atoms based on this simplified model. But these can be modified with further teaching, study and experience.
9. Context matters
The US president Harry Truman once whinged about all his economists giving advice, but then immediately contradicting that with an “on the other hand” qualification.
Individual scientists – and scientific disciplines – might be great at providing advice from just one frame. But for any complex social, political or personal issue there are often multiple disciplines and multiple points of view to take into account.
To ponder this we can look at bikehelmet laws. It’s hard to deny that if someone has a bike accident and hits their head, they’ll be better off if they’re wearing a helmet.
But if we are interested in whole-of-society health benefits, there is research suggesting that a subset of the population will choose not to cycle at all if they are legally required to wear a helmet.
Balance this against the number of accidents where a helmet actually makes a difference to the health outcome, and now helmet use may in fact be negatively impacting overall public health.
Valid, reliable research can find that helmet laws are both good and bad for health.
10. And just because it’s peer reviewed that doesn’t make it right
Peer review is held up as a gold standard in science (and other) research at the highest levels.
But even if we assume that the reviewers made no mistakes or that there were no biases in the publication policies (or that there wasn’t any straight out deceit), an article appearing in a peer reviewed publication just means that the research is ready to be put out to the community of relevant experts for challenging, testing, and refining.
It does not mean it’s perfect, complete or correct. Peer review is the beginning of a study’s active public life, not the culmination.
And finally …
Research is a human endeavour and as such is subject to all the wonders and horrors of any human endeavour.
Just like in any other aspect of our lives, in the end, we have to make our own decisions. And sorry, appropriate use even of the world’s best study does not relieve us of this wonderful and terrible responsibility.
There will always be ambiguities that we have to wade through, so like any other human domain, do the best you can on your own, but if you get stuck, get some guidance directly from, or at least originally via, useful experts.
There are now over 300 blogs on the list, although I am weeding out those which are no longer active or have removed public access to sitemeters. (Let me know if I weed out yours by mistake, or get your stats wrong).
Every month I get queries from people wanting their own blog included. I encourage and am happy to respond to queries but have prepared a list of frequently asked questions (FAQs) people can check out. Have a look at NZ Blog Rankings FAQ. This is particularly helpful to those wondering how to set up sitemeters.
Please note, the system is automatic and relies on blogs having sitemeters which allow public access to the stats.
Here are the rankings of New Zealand blogs with publicly available statistics for March 2015. Ranking is by visit numbers. I have listed the blogs in the table below, together with monthly visits and page view numbers.
Meanwhile I am still keen to hear of any other blogs with publicly available sitemeter or visitor stats that I have missed. Contact me if you know of any or wish help adding publicly available stats to your bog.
You can see data for previous months at Blog Ranks
My old man used to label us kids as “fair-weather sailors” when we bitched about working outside during bad weather.
That phrase comes to my mind sometimes when I come across people who claim to be “sceptics ” (“Skeptics”) behaving very unsceptically when confronted with a claim outside their area of interest. For example, someone who can be quite objective about scientific claims but reacts quite unobjectively to political claims.
Perhaps politics is a bit like religion to some people – they line up instinctively on one side or another. However, I think a true sceptic should still be able to consider political claims according to the facts available and not just rely on instincts.
So, I am all for this image. Yes it is hard. But when you think about it what use are one’s ingrained prejudices if they do not stand up to sceptical consideration.
“Peer-review” status is often used to endorse scientific papers cherry-picked because they support a bias.
Many scientists are not impressed with the peer-review processes scientific journals use. Like democracy, this peer-review is better than all the available alternatives but it certainly doesn’t guarantee published scientific papers are problem-free.
Sure, peer-reviewed sources are better than others which have no quality control. But it is still a matter of “customer beware.” The intelligent users of scientific literature must do their own filtering – make their own critical judgements of the likely reliability of reported scientific findings.
Despite this people often use the “peer-reviewed” description to endorse published finding (especially if they confirm their own biases) without any critical assessment. This happens a lot in on-line debates of “controversial” issues.
Here I will go through the details of peer-review of a recently published paper which anti-fluoride activists are endorsing and promoting, but others are critcising. The paper is:
The authors are clearly committed to a pet theory that fluoride is a neurotoxicant which could contribute to ADHD prevalence. Nothing wrong with that – we all feel committed to our hypotheses. We wouldn’t be human if we didn’t. But the best way to produce evidence for a hypothesis is to test it in a way that could prove it wrong.
In this case the authors found a correlation between ADHD prevalence in US states and the amount of community water fluoridation in each state. Trouble is, one can find just as good a correlation, or even a better correlation, with many other criteria for which state prevalence statistics are available. I listed a few in ADHD linked to elevation not fluoridation. Some of these factors are also correlated with community water fluoridation suggesting the correlation reported by Malin and Till (2015) may be deceptive.
A proper test of the fluoridation hypothesis would include considering the effect of including such confounders together with fluoridation in their statistical analysis. Malin and Till (2015) did include one other criteria – the median household income for 1992 – but did not include any others. I find this surprising because they acknowledged ADHD results from interaction of genetic and environmental factors. While fluoridation is not usually considered a relevant factor things like smoking and premature births are and there is conflicting evidence about the role of economic factors like poverty and income.
I can’t help feeling the limited consideration of confounding factors results from a desire to protect the fluoridation hypothesis and therefore not test it properly.
Again, such a desire is only human. But reviewers should have picked this up during their own considerations.
Interestingly, only one of the two reviewers raised possibility other confounders – specifically lead levels. This is of course valid as lead is a recognised neurotoxicant – but why did none of the reviewers question why other factors like smoking, premature births and social or regional factors were not considered?
I believe that is because both reviewers had research interests directed at chemical toxicity and not ADHD or similar mental characteristics. A matter of someone with a hammer only seeing nails.
“Some examples of my current work are exploring how exposure to, e.g., lead, manganese, and air pollution affect cognitive function and psychiatric symptoms; how exposure to Agent Orange and other herbicides used in Vietnam relate to the development of PD; and how formaldehyde and lead exposure relate to the development of ALS.”
“Dr Choi’s research focuses on the effects of environmental exposures on health outcomes. She has been studying the birth cohorts in the Faroe Islands where exposures to environmental chemicals including mercury, PCBs, and PFCs are increased due to traditional marine diets. In addition, she also studies the effects of the contaminants on cardiovascular function and type 2 diabetes among the Faroese septuagenarians. She is also actively involved in the research on the impact of nutrients as possible negative confounders that may have caused an underestimation of methylmercury toxicity. Dr Choi’s other research interests include studying the adverse effects of fluoride exposure in children.”
Why were reviewers with a wider research experience not chosen? This journal allows authors to propose suitable reviewers themselves. Or the reviewers may have been chose by the associate editor handling this paper – Prof David Bellinger. His research focuses on the neurotoxicity of metabolic and chemical insults in children. So again it may just be the blinkered view of someone whose research background stressed the role of neurotoxicants rather than other factors likely to influence ADHD prevalence.
The journal’s responsibility
I noticed that one of the two chief editors (who have final say over acceptance of submitted papers) of this journal is Prof Philippe Grandjean. He himself has been actively promoting the idea that fluoride is a neurotoxicant purely on the evidence of the metareview of Choi et al (2012). Yes he is a coauthor of that review and Choi is one of the reviewers of the Malin and Till paper. The review of Choi et al (2012) related to areas of mainly China where fluoride concentrations are higher than used in community water fluoridation. Areas where endemic fluorosis is common.
I have to wonder if Grandjean’s well-known position on fluoride and community water fluoridation was a consideration in choosing this journal for publication.
Others have commented that the journal Environmental Health is considered low-quality based on its low impact factor. I do not know the area well enough to pass judgement myself. However, I notice that the journal charges authors for publishing their paper (£1290/$2020/€1645 for each article accepted for publication.) This sort of charge, associated with poor quality peer-review makes me suspicious. I have commented on these sort of journal before in my post Peer review, shonky journals and misrepresenting fluoride science.
This is one example of peer-review and paper acceptance which brings into question the idea of using publication and peer-review as endorsement of a study’s quality. I am sure this is not an isolated case. Even with the best of intentions journal editors and reviewers are limited by their own areas of expertise. Journal publication and peer-review is a far from perfect process – even if it is preferable to current alternatives.
Unfortunately activists will promote poor quality studies like this by blindly using the study’s peer-review status.
The intelligent reader should beware of such blind endorsements. Knowing the human foibles which exist in the research and publication processes such a reader will consider the contents of the paper and not rely on peer-review status. They will consider the evidence and conclusions critically. And if they don’t have enough background to make their own critical assessment they will consider the views of others with the required expertise and not blindly accepting what political activists tell them.
BioMed Central publishes the Journal Environmental Health discussed in this post. I am not suggesting this paper was part of the peer-review racket discussed in the article. But the news item does highlight the point I am making that intelligent readers need to consider published scientific papers carefully and critically and not blindly rely on “peer-review” endorsement.