Cherry-picking data is an old technique used by those who wish to raise doubt about a scientific consensus. On the one hand, it isolates the cherry-picked data from their context and the rest of the data. On the other hand is present a “sciency” authority to the argument by pretending to be evidence-based.
I have written about cherry-picking in several articles discussing the fluoridation issue – so nothing new here. But the current surge of activity by anti-fluoridation activists attempting to raise doubt with the upcoming parliamentary discussion of new legislation on fluoridation is producing a fresh wave of cherry-picked arguments.
The image here is just one example in a letter to the Christchurch Press a few days ago. I thought it worthwhile to actually check out the quoted figures to see if the arguments awere justified.
Firstly, the figures were taken from the Ministry of Health’s 2014 data for the dental health of New Zealand children. Unfortunately, while the actual numbers used are correct, the data has since been withdrawn because of errors in the spreadsheet. So I will use the data for earlier years, 2005 – 20013, in my analysis.
The overall picture
First off – the overall picture shown by the Ministry of Health data is that community water fluoridation does reduce tooth decay. Of course, that is why the anti-fluoride campaigner rarely discusses the overall picture – instead, they cherry-pick data to confirm their bias. The figure below is for 5-year-olds averaged over the years 2005-2013
I have separated the data by ethnicity because of the big differences Māori and Pacifica on the one hand and the other ethnic groups on the other. In particular, the dental health of Māori and Pacifica children is poorer. This is an important factor which needs to be taken into account when comparing data from different regions. I discussed this further in my article Anti-fluoridation cherry-pickers at it again.
Data for Canterbury
It is likely that at least some of the 2014 spreadsheet mistakes were in the Canterbury data – but still the claim that there is no real difference between data for fluoridated and non-fluoridated areas could well be true – at least for some years. The figure below displays the data for 5-year-old children. Choose your year and you will get the answer you want to confirm your bias. Children from fluoridated areas seem to have poorer teeth in 2008 and 2010 and better teeth in 2012 and 2013.
The plots in the above figure indicate how unreliable such comparisons are for Canterbury because the fluoridated data is all over the place. This is because of the very low number of children in the fluoridated area: 22 – 70 over the years, 42 on average. There were on average 4720 children in the non-fluoridated areas. Children from the fluoridated area usually comprised less than 1% of the total.
The data for Canterbury does not deny the effectiveness of fluoridation, as the letter writer claims. They just show that no conclusion can be drawn from this cherry-picked data. At least I cherry-picked the data from 2005-2013 which enabled me to see how unreliable they were. The letter writer just cherry-picked one year! What will they do if the corrected spreadsheet for 2014 no longer supports their bias – switch to 2010 instead?
Comparing Canterbury and Waikato data
Here we have a different problem. The letter writer has simply cherry-picked these figures because they confirm her bias. She has not taken into account the important influence that ethnic composition has. Any intelligent analysis of this comparison must consider this aspect.
This is the ethnic composition of the 5-year-olds MoH data (averaged over 2005-2013).
So, whereas only 13.4% of Canterbury 5-year-olds are Māori or Pacifica, 33.6% of Waikato 5-year-olds are Māori or Pacifica. This is an important difference – especially as the dental health of Māori and Pacifica is poorer than others as demonstrated in the first figure .
Any analysis that does not take this difference into account will be misleading.
As well as ethnic distribution between regions there is also the influence of ethnic distribution between the fluoridated and non-fluoridated areas. This was a factor I discussed in Anti-fluoridation cherry-pickers at it again. The graphic below for 5-year-old children shows Māori and Pacifica are more concentrated in the fluoridated Waikato areas than in the non-fluoridated ones.
We can ignore the bar for fluoridated Canterbury because of the very small numbers.
What this means is that the mean value for fluoridated Waikato areas is decreased by the higher presence of Māori and Pacifica than in the non-fluoridated Waikato areas. This higher proportion Māori and Pacific in the Waikato region also affects the comparison of the two regions made by the letter writer.
Rather than comparing oranges with apples, let’s compare Canterbury and Waikato for the same ethnic group – Others (not including Māori and Pacifica). As the figure below shows, removal of the effect of Māori and Pacifica from the Canterbury data increase the caries-free percentage – but it is still slightly less than the equivalent data for the fluoridated Waikato areas.
So much for children from non-fluoridated Canterbury areas having better teeth than children from fluoridated Waikato areas.
Auckland and Counties/Manakau
Some anti-fluoride campaigners are pulling the same trick – asserting the dental health of non-fluoridated Canterbury children is better than for the fluoridated Auckland and Counties/Manakau children.
Here is a comparison of the ethnic composition of the three regions for the 2013 5-year-old MoH data.
|% Māori + Pacifica|
See the problem? It is just completely naive – or worse, dishonest – to compare data between regions like this without taking ethnic composition into account.
But that is not going to stop the determined activist who will just cherry-pick whatever fits their bias. I think the naive presentation of data in this way is no more justified by the declaration “Do the math” than misrepresentation of the science is justified by the declaration “Do the research!”
Note: I am well aware that the MoH data have other problems. A truly scientific analysis would also take into account factors like the degree of misallocation of children due to different fluoridation status of home and school, dental treatments such as fluoride varnishes differently used in different regions, missing data, different proportion of attendance according to region and ethnicity, etc. I am not the person to make such a thorough analysis. My sole purpose here is to show how such raw data can be misused for confirmation bias and “sciency” support of mistaken political agendas.