Cherry-picking data is an old technique used by those who wish to raise doubt about a scientific consensus. On the one hand, it isolates the cherry-picked data from their context and the rest of the data. On the other hand is present a “sciency” authority to the argument by pretending to be evidence-based.
I have written about cherry-picking in several articles discussing the fluoridation issue – so nothing new here. But the current surge of activity by anti-fluoridation activists attempting to raise doubt with the upcoming parliamentary discussion of new legislation on fluoridation is producing a fresh wave of cherry-picked arguments.
The image here is just one example in a letter to the Christchurch Press a few days ago. I thought it worthwhile to actually check out the quoted figures to see if the arguments awere justified.
Firstly, the figures were taken from the Ministry of Health’s 2014 data for the dental health of New Zealand children. Unfortunately, while the actual numbers used are correct, the data has since been withdrawn because of errors in the spreadsheet. So I will use the data for earlier years, 2005 – 20013, in my analysis.
The overall picture
First off – the overall picture shown by the Ministry of Health data is that community water fluoridation does reduce tooth decay. Of course, that is why the anti-fluoride campaigner rarely discusses the overall picture – instead, they cherry-pick data to confirm their bias. The figure below is for 5-year-olds averaged over the years 2005-2013
I have separated the data by ethnicity because of the big differences Māori and Pacifica on the one hand and the other ethnic groups on the other. In particular, the dental health of Māori and Pacifica children is poorer. This is an important factor which needs to be taken into account when comparing data from different regions. I discussed this further in my article Anti-fluoridation cherry-pickers at it again.
Data for Canterbury
It is likely that at least some of the 2014 spreadsheet mistakes were in the Canterbury data – but still the claim that there is no real difference between data for fluoridated and non-fluoridated areas could well be true – at least for some years. The figure below displays the data for 5-year-old children. Choose your year and you will get the answer you want to confirm your bias. Children from fluoridated areas seem to have poorer teeth in 2008 and 2010 and better teeth in 2012 and 2013.
MoH 5-year-old child dental health data for 2005-2013. dmft = decayed, missing and filled teeth.
The plots in the above figure indicate how unreliable such comparisons are for Canterbury because the fluoridated data is all over the place. This is because of the very low number of children in the fluoridated area: 22 – 70 over the years, 42 on average. There were on average 4720 children in the non-fluoridated areas. Children from the fluoridated area usually comprised less than 1% of the total.
The data for Canterbury does not deny the effectiveness of fluoridation, as the letter writer claims. They just show that no conclusion can be drawn from this cherry-picked data. At least I cherry-picked the data from 2005-2013 which enabled me to see how unreliable they were. The letter writer just cherry-picked one year! What will they do if the corrected spreadsheet for 2014 no longer supports their bias – switch to 2010 instead?
Comparing Canterbury and Waikato data
Here we have a different problem. The letter writer has simply cherry-picked these figures because they confirm her bias. She has not taken into account the important influence that ethnic composition has. Any intelligent analysis of this comparison must consider this aspect.
This is the ethnic composition of the 5-year-olds MoH data (averaged over 2005-2013).
| Canterbury | Waikato | |
| Māori (%) | 10.2 | 31.5 |
| Pacifica (%) | 3.2 | 2.1 |
| Other (%) | 86.6 | 66.4 |
So, whereas only 13.4% of Canterbury 5-year-olds are Māori or Pacifica, 33.6% of Waikato 5-year-olds are Māori or Pacifica. This is an important difference – especially as the dental health of Māori and Pacifica is poorer than others as demonstrated in the first figure .
Any analysis that does not take this difference into account will be misleading.
As well as ethnic distribution between regions there is also the influence of ethnic distribution between the fluoridated and non-fluoridated areas. This was a factor I discussed in Anti-fluoridation cherry-pickers at it again. The graphic below for 5-year-old children shows Māori and Pacifica are more concentrated in the fluoridated Waikato areas than in the non-fluoridated ones.
Distribution of Māori and Pacifica between fluoridated and non-fluoridated areas.
We can ignore the bar for fluoridated Canterbury because of the very small numbers.
What this means is that the mean value for fluoridated Waikato areas is decreased by the higher presence of Māori and Pacifica than in the non-fluoridated Waikato areas. This higher proportion Māori and Pacific in the Waikato region also affects the comparison of the two regions made by the letter writer.
Rather than comparing oranges with apples, let’s compare Canterbury and Waikato for the same ethnic group – Others (not including Māori and Pacifica). As the figure below shows, removal of the effect of Māori and Pacifica from the Canterbury data increase the caries-free percentage – but it is still slightly less than the equivalent data for the fluoridated Waikato areas.
So much for children from non-fluoridated Canterbury areas having better teeth than children from fluoridated Waikato areas.
Auckland and Counties/Manakau
Some anti-fluoride campaigners are pulling the same trick – asserting the dental health of non-fluoridated Canterbury children is better than for the fluoridated Auckland and Counties/Manakau children.
Here is a comparison of the ethnic composition of the three regions for the 2013 5-year-old MoH data.
| % Māori + Pacifica | |
| Canterbury |
13 |
| Auckland |
32 |
| Counties/Manakau |
52 |
See the problem? It is just completely naive – or worse, dishonest – to compare data between regions like this without taking ethnic composition into account.
But that is not going to stop the determined activist who will just cherry-pick whatever fits their bias. I think the naive presentation of data in this way is no more justified by the declaration “Do the math” than misrepresentation of the science is justified by the declaration “Do the research!”
Note: I am well aware that the MoH data have other problems. A truly scientific analysis would also take into account factors like the degree of misallocation of children due to different fluoridation status of home and school, dental treatments such as fluoride varnishes differently used in different regions, missing data, different proportion of attendance according to region and ethnicity, etc. I am not the person to make such a thorough analysis. My sole purpose here is to show how such raw data can be misused for confirmation bias and “sciency” support of mistaken political agendas.
Open Parachute 
When scientists are forbidden from speaking out public are going to lose faith in the “science” message: http://tvnz.co.nz/q-and-a-news/science-muzzled-video-6468513
LikeLike
Brian – could you explain the relevance of your link to my article?
I am well aware of how business interests can apply pressure to prevent communication of scientific evidence which conflicts with their interest. It certainly happened to me during my career when certain businesses threatened legal action if our group published their research.
But in this situation, I have the advantage that no-one can control what I write here. That is an advantage of retirement.
Yes, I have been prevented from communicating on websites, blogs and facebook pages financed or ideologically aligned with the big business of the “natural”/alternative health indiustry.
Perhaps you should complain to the organisations involved – FFNZ and FA, for a start? After all, they claim they want a debate – but they do everything in their power to prevent that debate. Even when I offer them space here they refuse.
LikeLike
Ken would it be possible to have a science commission, (as referred to in the TVNZ article) like an ombudsperson to check up on truthfulness of the output of the current party in government? You have linked to MoH output and I am not saying it is wrong just that the TVNZ article starts to engender doubt in government output in this area rather fertile for it.
Such a commission could be asked for example to examine Broadbent et al’s reply to criticism in AJPH letters: their sketchy statement as to where their control subjects were drinking,
Good on you for pointing out faults.
LikeLike
What the hell has the “MoH output” got to do with the issue of inhibition of scientists making public statements?
The MoH spreadsheets are not a scientific study – they are simply a record of dental treatment of children in NZ. Sure, they can provide material for a scientific study – but that requires a much greater intellectual input than we have seen from the FFNZ people who are msirepresenting the data.
You misunderstand the essence of Shaun’s suggestions and book. He is talking about the way that scientists can be inhibited from speaking up. Sure, that inhibitrion can come from governments – it can also come from business interests (who can also pressure governments and insitutions).
But I put to you the case of inhibition by the big business of the “natural”/alternative health industry and their astro turf organisations. (Shaun mentioned the role of such organisations in his interview)
Why do you not respond to my point? Why do you refuse to speak against these people who are attempting to prevent information on the fluoridation issue being discussed?
What aere you afraid of? 🙂
LikeLike
Don’t trust me here, Ken, but there may be something in what I say.
In the 1970s I gave my doctor a version of this wall chart and he put it on the wall of his consulting room. I should really have got myself one too, But I went on to big electronic charts like complex fire alarm panels, doing run ups and fault tracing. In both charts things could be faulty, or perhaps missing. For example if extinguisher gas releases as a result of something else than a fire, you still need the alarm to go off to get people out, which not all designers may have thought of.
The chart orginally by D E Nicholson (who died a few years ago) shows a lot of our biochemistry of our life and also gives inborn errors where such as an enzyme may be missing or faulty.
Click to access InbornErrors.pdf
It circles a “folic acid pool” which I think has various folates but also points to a missing enzyme (1.5.1.3) so I say simply, people are not able to reduce folic acid to the folates and they must get the more complex folates. And having folic acid-supplemented bread or other grain products may make them ill.
I suggest those ill people could be one of the category of people Shawn refers to astroturfing against folic acid. I am not trying to negate the concept just pointing out his probable error with the folic acid example.
You may have been implying the astroturfers where the natural vitamin industry. But sometimes they may be actually trying to fault trace a situation.
I’ll come back to other of your points later.
LikeLike