Flaw and porkie in anti-fluoride report claiming a flaw in Canadian study

Anti-fluoride group, Fluoride Action Network, ironically stamps their own critique of the Calgary fluoridation cessation study as “debunked.”

Anti-fluoride campaigners have launched another attack on a Canadian fluoridation cessation study. They claim it is flawed – but there is a huge flaw in their own critique.

I discussed their original attack in February last year (see Anti-fluoridationist’s flawed attacks on Calgary study). But this new attack is based on a published critique of the original study. I think that is good progress – the anti-fluoride campaigners have made a detailed critique and published it in the journal which published the original paper. The original authors have then responded. This is how things should be done.

Timeline

For those of you with the interest and time who want to go into the details, the original study was published in:

McLaren L, Patterson S, Thawer S, Faris P, McNeil D, Potestio M, Shwart L. (2016) Measuring the short-term impact of fluoridation cessation on dental caries in Grade 2 children using tooth surface indices. Community Dent Oral Epidemiol 2016.

The anti-fluoride critique was recently published in:

Neurath, C., Beck, J. S., Limeback, H., Sprules, W. G., Connett, M., Osmunson, B., & Davis, D. R. (2017). Limitations of fluoridation effectiveness studies: Lessons from Alberta, Canada. Community Dentistry and Oral Epidemiology, (October 2016), 1–7.

The response from the original authors was then published in:

McLaren, L., Patterson, S., Thawer, S., Faris, P., McNeil, D., & Potestio, M. (2017). Fluoridation cessation: More science from Alberta. Community Dentistry and Oral Epidemiology, (October), 1–3.

Other data which have been used in the critique and which I will use here can be found in:

McLaren, L., McNeil, D. A., Potestio, M., Patterson, S., Thawer, S., Faris, P., … Shwart, L. (2016). Equity in children’s dental caries before and after cessation of community water fluoridation: differential impact by dental insurance status and geographic material deprivation. International Journal for Equity in Health, 15(1), 24.

And:

McLaren, L., Patterson, S., Thawer, S., Faris, P., McNeil, D., Potestioa, M. L., & Shwart. L. (2017). Exploring the short-term impact of community water fluoridation cessation on children’s dental caries: a natural experiment in Alberta, Canada. Public Health, 146, 56–64.

Most of the authors of the critique are listed as members of the Fluoride Action Network (FAN) team and I can understand that FAN would feel proud that their critique was published. However, I feel their press release was rather underhand to imply the original study is:

“seriously flawed science  . . . Citizens should be concerned that their tax dollars have funded this biased work.”

And that the work was funded by state and public bodies:

“whose policy is to promote fluoridation.”

But let’s look at the critique itself – because it has some pretty big flaws itself.

What did the original study find?

My article, Anti-fluoridationist’s flawed attacks on Calgary study describes the details of this study. But briefly, it showed that child tooth decay increased in the Canadian city of Calgary after cessation of fluoridation. It used a comparison fluoridated city (the nearby and similar sized city of Edmonton) – and just as well because tooth decay also increased in that city during that time. However, there was still an increase in tooth decay in Calgary after cessation of fluoridation even after correction for the increase due to other factors apparent in Edmonton.

What did the critique claim?

A number of the criticisms are debatable and relatively minor.

How suitable was Edmonton as a comparison city? Neurath et al., (2017) claim it wasn’t suitable (but did not suggest a better alternative). McLaren et al., (2017) claim there is “no better comparison community for Calgary than Edmonton.”

Confounding – Neurath claims consideration of confounding factors was inadequate. McLaren et al., (2017) refer to extra data in two other papers and describes their consideration of several likely confounding factors like public health programmes and use of sealants. Whether the correct factors or sufficient factors were considered is always a bone of contention between authors and critics and, in the end, available data and funding decides.

Study design – Neurath et al., (2017) argue for randomised controlled trials. McLaren et al. (2017) point out that in studies of social programmes one must go with what exists. They say:

“While we agree with the value of stronger designs, one must be thoughtful about evaluation of public health measures, which by definition are complex and context-dependent. We used the best available
data and design for our circumstances”

Data ignored?

But Neurath et al (2017)’s major criticism is that some important data was ignored. And they claim that when that data is included the conclusions are not valid.

Of course, the FAN authors are stretching things quite a bit. The original study was based on data for tooth surfaces – the decay, extracted and filled tooth surfaces (defs). This was used as it is more sensitive than the tooth data itself – the decayed missing and filled teeth (deft).

Data for defs were only available for the 2004/05 and 2014/15 surveys. Unfortunately, there were no defs data for the pre-cessation period closer to the time of cessation (2011). That is the sort of problem researchers face when dealing with existing surveys and existing social programmes.

But the bright sparks at FAN latched on to the fact there was a survey with deft data in Calgary closer to the cessation time – 2009/2010. The fact that there was no equivalent survey for Edmonton didn’t hold them back – they proceeded to imply the 2009/2010 data had been purposely held back, despite McLaren making clear she could not use that data for Calgary in the absence of similar data for Edmonton. That would have negated the requirement for a comparison city and the existing data surely shows that requirement was very wise.

So Neurath et al., (2017) chose to ignore the obvious requirement for a comparison city and proceeded to argue their case on the Calgary data alone. They argued the study was “fatally flawed” and that “key data [was] omitted.” The argument implied the study was somehow fraudulent and that the authors had hidden the 2009/2010 survey data – despite the fact this data is used in another of their papers!

Neurath et al., (2017) pretend that a comparison city is not really necessary – relying only on the tooth data (deft) for Calgary they argue that as 50% of the increased in tooth decay had occurred between the 2004/05 and 2009/10 surveys then the increases seen after cessation of fluoridation was due to the same trend (see their Figure 1 below). They argued this proved that cessation of fluoridation had no effect. Ignoring completely the Edmonton data.

So, an obvious flaw in their critique – but wait, there is more! They actually go so far as to falsify data.

Falsifying a “correction factor”

Not satisfied with the plots in Figure 1B they found a way to make the data look even worse for McLaren et al. (2015). They came up with a “correction” factor to convert the deft data for 2009/2010 survey into defs data. Here is their Figure 2 using the “converted” deft data

Looks bad, doesn’t it?

However, the trick is in the way the conversion factor is calculated. They “used the ratio of defs to deft in the 2013/2014 survey to make the conversion.” The table below for subset (dmft>0) data they used shows this produces a conversion factor of 2.41 – big enough to dramatically push the 2009/10 data point right up so that it is sitting on the Edmonton “trend line” in their Figure 2 above.

But they could have equally used the ratio of defs/deft in the 2004/2005 survey to make the conversion. That produces a much lower conversion factor of 1.63 – which is not at all consistent with their claim “when we applied this conversion [2.41] to the 2004/2005 Calgary survey, where both deft and defs are known, the calculated defs was very close to the known defs.”

In fact, it may have been more appropriate to take the average conversion factors from the two available surveys. In the figure below I have done this (green data point) and compared this with the use of the conversion factors from the 2004/05 survey (purple data point) and that from the 2004/15 survey used by Neurath et al (yellow data point).

I guess this shows the danger of making these sort of adjustments – especially when there is a bias to confirm. And also that readers should beware of vague assertions of the sort:

“when we applied this conversion [2.41] to the 2004/2005 Calgary survey, where both deft and defs are known, the calculated defs was very close to the known defs.”

Conclusion

The McLaren et al., (2017) study has its limitations, limitations admitted and described by the authors. But, it is the FAN critique of Neurath et al., (2017) rather than the original study, that is fatally flawed. Flawed because of confirmation bias and a porky.

1: They ignored the necessary use of a comparison city and assumed the increase in tooth decay in Calgary was linear over the time between the two surveys McLaren at al used.

2: The use of any correction factor would be questionable but Neurath et al., (20127) clearly used a biased value to suit their argument. Further, they purposely misrepresented their correction factor by implying a similar value would have been obtained from the 2004/2005 survey data. Completely wrong.

Similar articles