Cherry picking fluoridation data

Anti fluoridationists certianly go in  for picking cherries when the produce “evidence” to discredit fluoridation. Two anti-fluoridatioon actvists, Bruce Spittle and Russel McLean, did this in there opinion piece in the Otago Daily Times recently (see No consent given for fluoridation). In particular, they carefuly selected data from the Ministry of Health’s database on the oral health of children.

How to “prove” fluoridation ineffective

They claimed:

The efficacy of water fluoridation is modest.The Dental School staff referred to the 2009 New Zealand Oral Health Survey which noted that in 2008, for all of New Zealand, the 5-year-olds in fluoridated areas had a percentage caries-free rate of 58.7% compared to the rate of 55.0% for those in non-fluoridated areas. However, the 2011 figures show little difference, with the rate for fluoridated areas being 59.91% and that for non-fluoridated areas being 59.18%.

In this case, while they chose the total data set, they selected just 2 years (2008 and 2011), selected only one age group, considered just “percentage caries-free” and ignored the data for “decayed, missing and filled teeth,” and also ignored the data for Maori (important because these show the influence of social and economic deprivation).

I looked at the whole data set in my article Fluoridation – it does reduce tooth decay. So I will just repeat a few of my data plots from that article toi show the effect of cherry picking. The plots of the data below give an idea of variability and trends. They also show the influence of social and economic deprivation is long-term. (Click on the graphs to enlarge for details).

% CARIES FREE

MEAN DECAYED, MISSING AND FILLED TEETH

There could well be some story in the apparent reduction of the effect of fluoridation shown by the % caries free of 5 year olds but that has to be put into the context of the whole data set. It is dishonest to just select the small samples Spittle and McLean did – but of course you can see why they did select those years and restricted their comments to just 5 year olds and “% caries free.”.

How to prove fluoridation damages oral health

The Fluoride Free New Zealand Facebook page provides another example of cherry picking (see Waikato Dental Health Stats). In this case their “findings” were so ludicrous that you might have thought they would blush at presenting them. There are some mistakes in their data, but it tends to show that children had better teeth in the non-fluoridated areas than the fluoridated areas!

Well, they achieved this by cherry picking data for one year (2011) and one region (the Waikato). If we look at some of the data over the availabke time period (2002 – 2011) for Waikato, we can see why they cherry picked this region and year.

But, comparing the Waikato data with the total data in the previous figures we can see a greater variability from year to year. This variability makes any honest comparison very difficult – but it does give opportunities for creative cherry picking. (Yes, I have just “cherry-picked” the caries free data for 5 years old in this graph – but I am making a point).

Be careful of variation and cherry picking

The opportunities for cherry picking in a field like this are everywhere because of the variability. This is not like the data one gets in a carefully controlled laboratory experiment. We are dealing with a biological system – which introduces biological variability. But on top of that, it is also a social system which introduces an extra set of variability.

As an example, I was recently discussing with my granddaughter her new school in Hamilton. She told me that all her friends actually lived out-of-town. But the dental data will have recorded them as being from a fluoridated area because, at the time, Hamilton was fluoridated. Then there are problems of getting consistent evaluation from a large number of dental nurses. Differences in dietary intake, drinking of bottled water, etc., – the list goes on.

Some of this variation “evens out” when the data set is large (and of course has more influence when only part of the data set is chosen. Yes, it would be nice to control for all these social effects but in the real world one rarely gets the opportunity.

However, my point is that the variability introduced into this sort of data by biological and social effects provides ample opportunity for political activists to cherry pick data to support their own story – confirmation bias if innocent and dishonest misrepresentation if not.

So, it is easy to make claims one way or the other in the fluoridation controversy – and to find data to support these claims. But serious assessment of the claims requires critical evaluation of the data – something many people have no experience with.

This seems to have been the case with the Hamilton City Council who concluded from presentations heavily biased toward anti-fluoridationists that fluoridation of water supplies is not effective. But why should we expect city councillors to have the critical evaluation skills required to assess such data? They should never be put in the position of being asked to make scientific judgements in this way.

See also:

Similar articles on fluoridation
Making sense of fluoride Facebook page
New Zealanders for fluoridation Facebook page