Tag Archives: Calgary

Flaw and porkie in anti-fluoride report claiming a flaw in Canadian study

Anti-fluoride group, Fluoride Action Network, ironically stamps their own critique of the Calgary fluoridation cessation study as “debunked.”

Anti-fluoride campaigners have launched another attack on a Canadian fluoridation cessation study. They claim it is flawed – but there is a huge flaw in their own critique.

I discussed their original attack in February last year (see Anti-fluoridationist’s flawed attacks on Calgary study). But this new attack is based on a published critique of the original study. I think that is good progress – the anti-fluoride campaigners have made a detailed critique and published it in the journal which published the original paper. The original authors have then responded. This is how things should be done.

Timeline

For those of you with the interest and time who want to go into the details, the original study was published in:

McLaren L, Patterson S, Thawer S, Faris P, McNeil D, Potestio M, Shwart L. (2016) Measuring the short-term impact of fluoridation cessation on dental caries in Grade 2 children using tooth surface indices. Community Dent Oral Epidemiol 2016.

The anti-fluoride critique was recently published in:

Neurath, C., Beck, J. S., Limeback, H., Sprules, W. G., Connett, M., Osmunson, B., & Davis, D. R. (2017). Limitations of fluoridation effectiveness studies: Lessons from Alberta, Canada. Community Dentistry and Oral Epidemiology, (October 2016), 1–7.

The response from the original authors was then published in:

McLaren, L., Patterson, S., Thawer, S., Faris, P., McNeil, D., & Potestio, M. (2017). Fluoridation cessation: More science from Alberta. Community Dentistry and Oral Epidemiology, (October), 1–3.

Other data which have been used in the critique and which I will use here can be found in:

McLaren, L., McNeil, D. A., Potestio, M., Patterson, S., Thawer, S., Faris, P., … Shwart, L. (2016). Equity in children’s dental caries before and after cessation of community water fluoridation: differential impact by dental insurance status and geographic material deprivation. International Journal for Equity in Health, 15(1), 24.

And:

McLaren, L., Patterson, S., Thawer, S., Faris, P., McNeil, D., Potestioa, M. L., & Shwart. L. (2017). Exploring the short-term impact of community water fluoridation cessation on children’s dental caries: a natural experiment in Alberta, Canada. Public Health, 146, 56–64.

Most of the authors of the critique are listed as members of the Fluoride Action Network (FAN) team and I can understand that FAN would feel proud that their critique was published. However, I feel their press release was rather underhand to imply the original study is:

“seriously flawed science  . . . Citizens should be concerned that their tax dollars have funded this biased work.”

And that the work was funded by state and public bodies:

“whose policy is to promote fluoridation.”

But let’s look at the critique itself – because it has some pretty big flaws itself.

What did the original study find?

My article, Anti-fluoridationist’s flawed attacks on Calgary study describes the details of this study. But briefly, it showed that child tooth decay increased in the Canadian city of Calgary after cessation of fluoridation. It used a comparison fluoridated city (the nearby and similar sized city of Edmonton) – and just as well because tooth decay also increased in that city during that time. However, there was still an increase in tooth decay in Calgary after cessation of fluoridation even after correction for the increase due to other factors apparent in Edmonton.

What did the critique claim?

A number of the criticisms are debatable and relatively minor.

How suitable was Edmonton as a comparison city? Neurath et al., (2017) claim it wasn’t suitable (but did not suggest a better alternative). McLaren et al., (2017) claim there is “no better comparison community for Calgary than Edmonton.”

Confounding – Neurath claims consideration of confounding factors was inadequate. McLaren et al., (2017) refer to extra data in two other papers and describes their consideration of several likely confounding factors like public health programmes and use of sealants. Whether the correct factors or sufficient factors were considered is always a bone of contention between authors and critics and, in the end, available data and funding decides.

Study design – Neurath et al., (2017) argue for randomised controlled trials. McLaren et al. (2017) point out that in studies of social programmes one must go with what exists. They say:

“While we agree with the value of stronger designs, one must be thoughtful about evaluation of public health measures, which by definition are complex and context-dependent. We used the best available
data and design for our circumstances”

Data ignored?

But Neurath et al (2017)’s major criticism is that some important data was ignored. And they claim that when that data is included the conclusions are not valid.

Of course, the FAN authors are stretching things quite a bit. The original study was based on data for tooth surfaces – the decay, extracted and filled tooth surfaces (defs). This was used as it is more sensitive than the tooth data itself – the decayed missing and filled teeth (deft).

Data for defs were only available for the 2004/05 and 2014/15 surveys. Unfortunately, there were no defs data for the pre-cessation period closer to the time of cessation (2011). That is the sort of problem researchers face when dealing with existing surveys and existing social programmes.

But the bright sparks at FAN latched on to the fact there was a survey with deft data in Calgary closer to the cessation time – 2009/2010. The fact that there was no equivalent survey for Edmonton didn’t hold them back – they proceeded to imply the 2009/2010 data had been purposely held back, despite McLaren making clear she could not use that data for Calgary in the absence of similar data for Edmonton. That would have negated the requirement for a comparison city and the existing data surely shows that requirement was very wise.

So Neurath et al., (2017) chose to ignore the obvious requirement for a comparison city and proceeded to argue their case on the Calgary data alone. They argued the study was “fatally flawed” and that “key data [was] omitted.” The argument implied the study was somehow fraudulent and that the authors had hidden the 2009/2010 survey data – despite the fact this data is used in another of their papers!

Neurath et al., (2017) pretend that a comparison city is not really necessary – relying only on the tooth data (deft) for Calgary they argue that as 50% of the increased in tooth decay had occurred between the 2004/05 and 2009/10 surveys then the increases seen after cessation of fluoridation was due to the same trend (see their Figure 1 below). They argued this proved that cessation of fluoridation had no effect. Ignoring completely the Edmonton data.

So, an obvious flaw in their critique – but wait, there is more! They actually go so far as to falsify data.

Falsifying a “correction factor”

Not satisfied with the plots in Figure 1B they found a way to make the data look even worse for McLaren et al. (2015). They came up with a “correction” factor to convert the deft data for 2009/2010 survey into defs data. Here is their Figure 2 using the “converted” deft data

Looks bad, doesn’t it?

However, the trick is in the way the conversion factor is calculated. They “used the ratio of defs to deft in the 2013/2014 survey to make the conversion.” The table below for subset (dmft>0) data they used shows this produces a conversion factor of 2.41 – big enough to dramatically push the 2009/10 data point right up so that it is sitting on the Edmonton “trend line” in their Figure 2 above.

But they could have equally used the ratio of defs/deft in the 2004/2005 survey to make the conversion. That produces a much lower conversion factor of 1.63 – which is not at all consistent with their claim “when we applied this conversion [2.41] to the 2004/2005 Calgary survey, where both deft and defs are known, the calculated defs was very close to the known defs.”

In fact, it may have been more appropriate to take the average conversion factors from the two available surveys. In the figure below I have done this (green data point) and compared this with the use of the conversion factors from the 2004/05 survey (purple data point) and that from the 2004/15 survey used by Neurath et al (yellow data point).

I guess this shows the danger of making these sort of adjustments – especially when there is a bias to confirm. And also that readers should beware of vague assertions of the sort:

“when we applied this conversion [2.41] to the 2004/2005 Calgary survey, where both deft and defs are known, the calculated defs was very close to the known defs.”

Conclusion

The McLaren et al., (2017) study has its limitations, limitations admitted and described by the authors. But, it is the FAN critique of Neurath et al., (2017) rather than the original study, that is fatally flawed. Flawed because of confirmation bias and a porky.

1: They ignored the necessary use of a comparison city and assumed the increase in tooth decay in Calgary was linear over the time between the two surveys McLaren at al used.

2: The use of any correction factor would be questionable but Neurath et al., (20127) clearly used a biased value to suit their argument. Further, they purposely misrepresented their correction factor by implying a similar value would have been obtained from the 2004/2005 survey data. Completely wrong.

Similar articles

Anti-fluoridationist’s flawed attacks on Calgary study

antiscience-logic-web

Image credit: Why I will no longer approve anti-fluoride dishonesty – New Anthropocene

The anti-fluoride propagandists are at it again – attacking a recently published scientific paper because the reported data does not fit with their own agenda.

The are attacking this paper:

McLaren L, Patterson S, Thawer S, Faris P, McNeil D, Potestio M, Shwart L. (2016) Measuring the short-term impact of fluoridation cessation on dental caries in Grade 2 children using tooth surface indices. Community Dent Oral Epidemiol 2016.

The Fluoride Action Network (FAN) claims the study is “fatally flawed” and that “key data [was] omitted.” The local FAN organisation – Fluoride Free New Zealand (FFNZ) has gone further claiming the study is “basically a fraud.”

Serious (even slanderous) accusations – let’s see if those charges hold up.

What did the study find?

The paper reports changes in dental caries levels of grade 2 children in Alberta, Canada between 2005/2005 and 2013/2014. Two cities were chosen – Calgary (which stopped fluoridation in May 2011 – after having been fluoridated since 1991) and Edmonton where fluoridation started in 1967 and is still in place. This enabled an assessment of the effect of fluoridation cessation on the dental health of children in Calgary.

Dental caries actually increased in both cities but the data shows a larger increase in dental caries where fluoridation had been stopped (Calgary) than where it continued (Edmonton).

mean-defs

defs = decayed, extracted (due to caries) filled primary tooth surfaces.

The change in Calgary (3.8) is greater than for Edmonton (2.1) and the difference is statistically significant.

The above graphic is for all tooth surfaces – similar results were obtained with smooth surfaces only – Calgary change = 2.9 and Edmonton change = 1.6.

Important features of study

There are several important features of the paper and the data reported:

1: Whereas previous cessation studies had used tooth-level data this study used tooth surface-level data. The authors had previously reported tooth level data which showed similar results.

The WHO prefers and recommends tooth surface-level data to evaluate an intervention such as starting or stopping fluoridation because of its greater sensitivity. The study authors used  tooth surface-level data and focused on smooth tooth surfaces which are most likely influenced by fluoride in this age group.

Most earlier fluoridation cessation studies had used less sensitive tooth-level data so this study was a definite improvement

2: A comparison (or “control”) city was used – Edmonton. Many previous cessation studies suffered from not using a comparison community. Interpretation is more difficult without some sort of “control” as changes due to other factors may be important. Factors like different assessment personnel, changes in diet (eg sugar consumption or bottled water), changes in dental health care, etc. It is difficult to take account of all factors but the inclusion of a similar comparison community helps reduce their effects on the results.

Again, this study was a definite improvement over previous cessation studies which had not used a comparison community.

3: Pre-cessation data were collected several years before cessation. The authors list this as a limitation, saying:

“it would have been preferable to have data collected closer to the year of cessation.”

However, this being the real world, the study used the only tooth surface-level data available for Calgary and Edmonton. There was absolutely no question of data being omitted (or “fraud”) that the anti-fluoride campaigners allege.

So how has FAN supported their charges?

Well, basically by retreating from (or ignoring) the advantages of this study – use of tooth surface-level data and use of a comparison community.

1: They use the limitation that the pre-cessation tooth surface-level data was collected several years before cessation to resort to more recent 2009/2010 data. But, importantly, that data is not tooth surface-level and it is for only one city – Calgary! 

So, they claim, the researchers “omitted key data” – the 2009/2010 data – pretending the researchers had  “hidden” it. But that data was not surface-level data. And, far from “hiding,” it the same research group specifically used this tooth-level data in an accompanying  paper:

McLaren L, McNeil D, Potestio M, Patterson S, Thawer S, Faris P, Congshi S,  Shwart L. (2016). Equity in children’ s dental caries before and after cessation of community water fluoridation: differential impact by dental insurance status and geographic material deprivation. International Journal for Equity in Health 15: 24

2: They ignore completely the importance of using a comparison community. Sure, they might claim that no survey was taken in Edmonton in 2009/2010. But they have thrown the baby out with the bath water in their attempt to discredit the research. They have fallen back on the limitation of so many fluoridation cessation studies which rely on cross-sectional measurements at different times without using a control. Worse still, they do this despite the tooth surface-level data for Edmonton showing an increase in dental caries even though there had been no cessation of fluoridation! They are willingly falling into the trap of ignoring all the other factors which could influence the results.

3: The FAN press release attempts to discredit the researchers saying:

“The lead author of the study, Lindsay McLaren, is not an independent scientist on the fluoridation controversy.”

This “damning by association” of researchers with their institutions, the fact that they receive a wage from their employers and that they write articles about their scientific results (which do not support the claims of the anti-fluoridation lobby) is really disingenuous. It’s a way of discrediting any scientist who produces results an anti-science group does not like. (Anti-science groups often use this argument – for example, anti-fluoride activists haver told me I am not “independent” because I once worked as a scientific researcher and,  therefore, cannot be trusted!).

Ironically, they are hanging themselves with their own rope on this one. Their press release makes mileage by quoting an assessment from an “independent” scientist Dr Trevor Sheldon. I checked with Sheldon and he confirmed that he:

“was invited to comment on this by Fluoride Action Network.  I did not have much time to do this but agreed to do so and received no incentive or reward.”

Now, doesn’t that admission  compromise Sheldon as an “independent” commentator? He immediately responds to a request from an activist political organisation, a group funded by the “natural”/alternative health industry – without any qualms! That confirms in my mind the links and political sympathies Sheldon has with FAN that I have long suspected.

It is perfectly normal (and healthy) for credible scientists to disagree with published papers and there are publication mechanisms to do this effectively. For example “letters to the editor” or comments in the specific journal – or even alternative journals. The fact Sheldon chose an almost immediate critique in the newsletter of a political activist group confirms to me that he is not “independent” on this subject and that he, in effect, belongs to the stable of “tame” anti-fluoride scientists FAN makes regular use of in their press releases. People like Bill Hirzy, Kathleen Theissen, Bill Osmunson, Hardy Limeback, etc.

Some other critical details

FAN relied on Sheldon’s evaluation for their main criticism but there are other details in that evaluation which I can comment on.

1: Sheldon would have preferred “a randomised controlled trial where areas are randomly allocated to start/cease water fluoridation.”

Yes, that would have been nice wouldn’t it.But the fact this has never been done – and Sheldon himself has never initiated such a trial – rather indicates how impossible such a trial is. The Cochrane review of community water fluoridation, which anti-fluoride campaigners love to misrepresent, commented on the silliness of such complaints:

But the fact this has never been done – and Sheldon himself (and similar anti-fluoride critics) has never initiated such a trial – rather indicates how impossible such a trial is. The Cochrane review of community water fluoridation, which anti-fluoride campaigners love to misrepresent, commented on the silliness of such complaints:

“particularly for research questions where evidence from randomised controlled trials is never going to be available due to the unfeasibility of conducting such trials. Community water fluoridation is one such area.”

2: He attempts to  extrapolate from tooth-level data to “fill in the gaps” in the tooth surface-level data. This highly speculative analysis allows him to conclude that most of the increase in dental caries shown by children in Calgary occurred before the cessation of fluoridation.

He accuses the researchers of making a “heroic assumption” but it seems to me the assumptions involved in his extrapolation are worse than “heroic.” Yet  the analysis enables him to make conclusions which seem to be derived more from confirmation bias than from intelligent analysis of the data. Especially as he ignored any data from Edmonton.

3:  He comments on problems resulting from population movements, confounding factors, assessment methods and staff differences, sample size differences, etc. But these are common problems when attempting to make sense of existing data. Importantly, these problems and worse, plague the cessation studies that anti-fluoride campaigners continually quote to support their claims that community water fluoridation is ineffective.

Summary

The Calgary/Edmonton study does not have the limitations of most fluoridation cessation studies – in particular, a comparison community was used and the more sensitive and appropriate tooth surface-level data was used rather than tooth-level data. The period between the first assessment and cessation is an admitted limitation but certainly does not indicate the omission of data or hiding data as anti-fluoridationists are claiming. There was just no intermediate data available.

The resort to speculative extrapolation of tooth-level data for only Calgary is disingenuous as it ignores the low sensitivity of that data and the influence of other factors compensated for by the use of Edmonton as a comparison community.

Anti-fluoridation campaigners often make ready use of older cessation studies which showed no effect – despite the obvious limitation. In particular, they often use studies where no comparison community was included and ignore completely the other factors involved such as changes in dental health provisions or social health policies such as replacing fluoridation with mouth rinse campaigns.

It is, therefore, ironic for them to come out fighting in an attempt to discredit a study which has fewer limitations than the ones they rely on. Worse, to accuse researchers of purposely omitting data – or even of scientific fraud.

The real reason for these attacks on honest researchers is that the data just does not fit the anti-fluoridation agenda and therefore, in their eyes, must be discredited.

Oh, and beware of such activists using the word “independent’ to describe a researcher. It inevitably indicates someone with allegiances to their campaign. Someone who can be relied on to give the appropriate quote to be used in their press releases.

Similar articles