Tag Archives: Cherry Picking

Madonna teaches us a lesson in critical thinking

Maybe just a small lesson – but an important one. We should always look at context and not cherry-pick that which confirms our bias. In particuilar, we shouldn’t simply repeat social media claims without doing a bit of fact-checking for ourselves.

It’s an important lesson at the moment because there is a lot of this going on in the social media hysteria surrounding the US presidential elections.

I was called out recently because I shared a Facebook meme questioning the moral authority of people like Madonna (who regularly makes lewd public performances) making moral judgments on the lewd language used by President Trump in a private conversation many years ago. The commenter suggested her lewd performances were irrelevant and that I should instead criticise her for making stupid and distasteful comments about blowing up the White House.

So, I decided to check out what she did say – and the video above is the section of her speech at one of the Women’s Marches where she referred to bombing the White House.

Putting Madonna’s famous comment in context

Now, I realise that the secret service is required to check out people who make comments like this – but it would be a perversion of justice for them to take any action against her. The context of the bombing comment makes clear she used the expression of personal thoughts as a rhetorical device to bring home her main message – which supported the direct opposite of such a terrorist act. In fact, she specifically said such an act would be pointless as it would change nothing (she is clearly wrong there as there would surely be huge changes  – but you get my point. She was not advocating anything like bombing).

Madonna has reacted to the news the secret service will investigate her with this comment:

“I am not a violent person, I do not promote violence and it’s important people hear and understand my speech in it’s entirety rather than one phrase taken wildly out of context.

My speech began with ‘I want to start a revolution of love,” Madonna wrote. “I spoke in metaphor and I shared two ways of looking at things — one was to be hopeful, and one was to feel anger and outrage, which I have personally felt. However, I know that acting out of anger doesn’t solve anything. And the only way to change things for the better is to do it with love.”

I think that sums up what she did say in that section of her speech and her critics should take that lesson on board.

But, not only her critics – also her supporters and allies. It is telling that one of those commenting on her explanation wrote:

“You want people to listen to your speech in its entirety and not a phrase taken wildly out of context….hmmmmm isn’t that what you and your followers have been doing to President Trump all along.”

To my mind, the extreme partisanship of social media commenters, and the #fakenews promoted by mainstream media, has often relied on such cherry picking and removal of context. It started early on in the US election campaign and is still proceeding. The recent rather undignified spat over the numbers attending the inauguration in the Mall, and the numbers observing it internationally on TV is just one example. It reflects how childish – on both sides – this spat between President Trump and the mainstream media has become.

Nor does it impress me that some otherwise rational social media commenters have taken the unfortunate “alternative facts” statement completely out of context to use as a political whip. Although, I suppose it does not surprise me. Even the most self-declared rational of us can be very irrational at times. It is part of being human.

Partisanship and a biased media is counterproductive

I wish we would all calm down and attempt to be more rational and critical in approaching news media reports of the current election hysteria. On the one hand, things do run a lot better when we avoid confirmation bias, partisanship and cherry-picking. (And we wouldn’t have to run the gauntlet of being ‘unfriended” on facebook for questioning things).

On the other hand, I think this opinion piece today from Frank Bruni in the New York Time makes a point that opponents of President Trump have seemingly been oblivious to (see The Wrong Way to Take On Trump). This lack of critical thinking, the cherry-picking, partisanship , confirmation bias and #fakenews from a biased media, actually helped Trump win the election – and is currently probably helping to cement support from those who voted for him. He writes:

“There’s so much substantive ground on which to confront Trump. There are acres upon acres. Why swerve into the gutter? Why help him dismiss his detractors as people in thrall to the theater of their outrage and no better than he is?”

And why risk that disaffected Americans, tuning in only occasionally, hear one big mash of insults and insulters, and tune out, when there’s a contest — over what this country stands for, over where it will go — that couldn’t be more serious.”

I am sure that last point happened with people who may have supported or considered supporting Trump during the campaign. Once you have heard or seen a few reports full of what you know is #fakenews, personal attacks, partisan commentary, insults and swearing you do tend to turn off. You do tend not to trust future news reports, especially those ridiculing your possible electoral choice. And in the more committed cases, this experience of news you know to be fake means that from then on you will whole-heartedly accept the #fakenews from your own side. “The other side lied so what your side says must be true!”

In my own case, I know the experience of the partisanship and media bias during the US election campaign has made me very untrustful of the mainstream media – in fact, all the media. I now refuse to accept claims made in the news as factual until I can check them out for myself. I have sort of done this with Madonna’s speech here.

That means more work and a resignation that there are many things I will not form an opinion on (because I don’t have time to fact-check). But I feel better about myself – even if some of my associates may be upset that I don’t go along with their current thinking.

Note: The video above is also out of context – being just the part of Madonna’s speech relevant to her bombing comment. If you are really interested you can find her full speech here. I watched it and found myself becoming less and less sympathetic to her so won’t post the video here as it would sort of distract from my point.

Similar articles

Anti-fluoridation cherry-pickers at it again

Anti-fluoride campaigners seem to be a sixes and sevens on whether community water fluoridation (CWF) is effective or not. Some will accept CWF is effective in improving oral health but moan about the ethics or reports of harm. Others will simply claim CWF is not effective.Stans-lie-annot

Stan Litras is a Wellington dentist and anti-fluoride campaigner in the later camp. He continually denies that CWF is effective and claims he has science to back up his claim. However, what he actually means is that of he cherry-picks the science, holds his mouth the right way and prevents you from looking at the context and data – he can find a quote to support his position.

I have debunked his claims before in my articles – such as, Cherry-picking and misinformation in Stan Litras’s anti-fluoride article and Cherry-picking and misinformation in Stan Litras’s anti-fluoride article. But he is at it again. This time he is misrepresenting a recent New Zealand research paper in a recent letter to the Christchurch Press.

There are several misrepresentations in his letter but here I will just take issue with the highlighted text. Stan claims:

“The DHB’s Dr Martin Lee published a research paper recently which showed little or no benefit from water fluoridation, a finding consistent with the modern weight of evidence.”

It is always best to check out such claims and I have hunted down the paper Stan probably refers to. It is:

Schluter, P. J., & Lee, M. (2016). Water fluoridation and ethnic inequities in dental caries profiles of New Zealand children aged 5 and 12–13 years: analysis of national cross-sectional registry databases for the decade 2004–2013. BMC Oral Health, 16(1), 21.

This is linked to the full-text version so readers can check out the paper itself.

Far from showing “little or no benefit” from water fluoridation the abstract actually says:

“Significant and sustained differences were observed between Māori and non-Māori children, and between CWF and non-CWF exposed groups.”

Stan ignored that sentence but latches on to (or cherry-picks) the next sentence:

“However, a convergence of dental profiles between non-Māori children in CWF and non-CWF regions was observed.”

But he ignores completely the authors’ discussion of this apparent convergence. Obviously, the discussion doesn’t support Stan’s anti-fluoride bias! But that discussion is interesting and worth considering.

Is there a convergence?

I have noted this convergence before in my articles Cherry picking fluoridation data and Fluoride debate: Response to Paul’s 5th article where I used similar data to that used in Schluter & Lee (2016). That data is available on the Ministry of Health’s website.

The graphs below show the raw data for all (“total”) children and for Māori:

% CARIES FREE

MEAN DECAYED, MISSING AND FILLED TEETH

So, yes, there is a convergence in the sense that the differences in the oral health of children in fluoridated and unfluoridated areas appears to be reducing with time. I have speculated that the apparent convergence could have something to do with the introduction of “hub and spoke” dental clinics after 2004. One problem with this raw data is that children are allocated according to the fluoridation status of thew school – rather than their residence. This will lead to incorrect allocation in some cases.

However, this paper suggests another reason for the convergence which I hadn’t considered.

Changes in and composition of the non-Māori group

The authors say:

“Another notable feature was the apparent convergence of prevalence estimates amongst non-Māori children in CWF and non-CWF areas. It is likely that a substantial driver of this convergence was due to significant changes within the dynamic and heterogeneous non-Māori groups both within and between DHB regions. In effect, the ecological fallacy – a logical flaw whereby analyses of group data are used to draw conclusions about an individual – may be operating within the non-Māori group.”

This is interesting and is supported by the data.

First, let’s note that while I compared data for Māori children with the total data in my figures above these authors have actually compared data for Māori and non-Māori. This shows a clearer convergence for non-Māori children than for all children – see this figure for 5-year-old children from the paper (dmft = decayed, missing and filled teeth per child):

12903_2016_180_Fig1_HTML

Fig. 1 No obvious decay experience (caries-free) percentages and mean dmft for 5-year old children over years 2004 to 2013, partitioned by Māori and non-Māori ethnicities and fluoridated (F) and non-fluoridated (NF) areas

The picture is similar for year 8 children.

So you can see why anti-fluoride campaigners would love to cherry pick the non-Māori data. I predict that Stan and other anti-fluoridation campaigners will be reproducing parts of this figure in their propaganda for future use.  That graph is just too good for them not to cherry-pick.

But we need to remember that the non-Māori group is not ethnically uniform. In particular, Pacifica make a large contribution to this group. That contribution is unevenly distributed between the fluoridated and fluoridated groups. And it has changed over time.

  • In 2013 about 86% of Pacific live in fluoridated areas – over the period covered by these MoH records this proportion has varied between 80 and 90%.
  • In 2013 Pacifica make up about 12% of the non-Māori group (between 2007 and 2o13 this has varied between 9.0 and 12.7%)
  • In 2013 Pacific make up about 19.3% of the non-Māori fluoridated group (between 2007 and 2013 this has varied between 14.9 and 20.7% of the non-Māori fluoridated group).

MoH data confirms problem of Pacific inclusion

So the oral health of Pacifica can have a relatively large influence on the data for the non-Māori group – particularly for the fluoridated non-Māori group where they are included. This becomes important when we realise that the oral health of Pacifica is markedly poorer than for the rest of the non-Māori group. I have illustrated this using the average of data for fluoridated 5-year-olds in the period 2007 – 2013.

other-pacifica

Data for 5-year-old children. dmft = decayed, missing and filled teeth. The “other” group is non-Māori and non-Pacifica

So the poorer oral health of Pacifica will drag down the % caries free and drag up the mean dmft data for the fluoridated non-Māori group. However, this will have little influence on the unfluoridated non-Māori group because of the very small Pacific contribution.

We can confirm this with the raw data from the Ministry of Health website. That data is given separately for Māori, Pacifica and “other” (non-Māori/non-Pacific). The figure below shows this data for the 5-year-old children.

Other

Comparison of data for “other” (non-Māori/non-Pacifica) children in fluoridated (F) and unfluoridated (UF) areas.

OK – there is still some evidence of convergence from about 2007 on between fluoridated and unfluoridated children. But the graphs do indicate that CWF is still having  a beneficial effect.

But I think Stan and his mates will prefer to cherry-pick the data for the non-Māori group and keep very quiet about the distorting effect that inclusion of Pacific in this group has had on the apparent convergence.

Note: I have used the raw Ministry of health data in this discussion. Schluter & Lee (2016) used standardised estimates to account for the difference in the numbers of unexamined children according to ethnicity.

Similar articles

Cochrane fluoridation review. II: “Biased” and poor quality research?

Avoid-despair-about-biases

Here again, the language used in the Cochrane review (Water fluoridation for the prevention of dental caries) is very misleading. Especially when cherry-picked and taken out of context. The word “bias” used in the review does not have the meaning an uninformed reader might think.

It does not mean motivated experimental design or selection of data to “prove” a predetermined outcome. Rather it has specific meaning related to common (and usually innocent) problems encountered clinical drug trials. These problems also occur in real-life epidemiological studies and trials of the sort used for evaluating social health measures like community water fluoridation (CWF) but the lack of control in such studies means they are harder to combat.. The Cochrane ideal of randomised double-blinded trials is just not realistic in these situations. As the American Academy of Pediatrics comments in their article on the Cochrane review:

“it would be a logistical nightmare to try creating a public water system that pumps fluoridated water to the first house on the block, delivers non-fluoridated water to the following two houses and then provides fluoridated water to the 4th and final house on that block.”

The review describes the types of “biases”  considered:

“Assessment of risk of bias in included studies
. . . . . The domains assessed for each included study included: sampling, confounding, blinding of outcome assessment, completeness of outcome data, risk of selective outcome reporting and risk of other potential sources of bias. . . . . .  We had identified the following factors as important confounders for the primary and secondary outcomes: sugar consumption/dietary habits, SES, ethnicity and the use of other fluoride sources.”

Unfortunately, all these sorts of “biases” are inevitable to some extent in the real world. Researchers do not always have the budget to include consideration of all confounders, or the degree of control required. There are inevitable gaps in data when families move or withdraw children from schools. Yet it is real-world studies, not idealised laboratory experiments, that give the data and other evidence reviewers and decision-makers must consider. Humans can’t be treated like experimental rats.

What “biases” did the review find

As far as “caries outcome” is concerned the review reports a  “high risk of bias overall,” but this “bias may occur in either direction.” This indicates there is not a motivated selection of data or experimental design to produce a predetermined result as that would show up as a systematic bias.

The major cause of “bias” arose from lack of control of the confounding issues of “sugar consumption/dietary habits, SES, ethnicity and the use of other fluoride
sources.” These are of course important factors which could influence results. In the real world it is difficult to control of variations in dietary intake, although socioeconomic status (SES) and ethnicity can be included in statistical analyses of data. My impression is that this is usually done in more recent studies (which the Cochrane review team had excluded from their review – see Cochrane fluoridation review. I: Most research ignored).

Most studies were at low risk of “bias” from sampling methods but on “detection bias” the reviewers report:

The majority of the studies did not blind outcome assessors. This is perhaps unsurprising when considering the efforts that may be required to blind assessors for this type of study.”

The qualification here surely indicates the inadequacy of the Cochrane criteria for using word’s like “bias” which are more fitted to clinical drug trials than evaluation of social health policies.

Most studies did not suffer from “incomplete outcome data” where some data is not measured, but some showed the “bias” of “selective reporting” where data sets reported were incomplete. Interestingly the reviewers report one study where:

“the baseline fluoridation status of the children was determined by the location of the school they attended, which may not have taken into account any children attending schools in fluoridated areas who resided outside those areas.”

This must be a common problem researchers face when they do such real-world epidemiological studies.

Inappropriate criteria used to judge quality of research

Given the nature of evaluating a social health policy like community water fluoridation (CWF) I think the criteria used by the review team to judge the quality fo available research was quite wrong. Their criteria were more fitted to judging clinical drug trials and not social health policies. They acknowledge this in their discussion section “Quality of evidence:”

“However, there has been much debate around the appropriateness of GRADE when applied to public health interventions, particularly for research questions where evidence from randomised controlled trials is never going to be available due to the unfeasibility of conducting such trials. Community water fluoridation is one such area.”

And:

“we accept that the terminology of ’low quality’ for evidence may appear too judgmental. We acknowledge that studies on water fluoridation, as for many public health interventions, are complex to undertake and that researchers are often constrained in their study design by practical considerations. For many public health interventions, the GRADE framework will always result in a rating of low or very low quality. Decision makers need to recognise that for some areas of research, the quality of the evidence will never be ’high’ and that, as for any intervention, the recommendation for its use depends not just upon the quality of the evidence but also on factors such as acceptability and cost-effectiveness (Burford 2012).” My emphasis.

These are important qualifications which, however, did not make it into the review’s Abstract or Plain Language Summary – and certainly not into media reporting. I think the review team was irresponsible to omit such qualifications from their summaries – and many people might suggest they were irresponsible to use such inappropriate criteria for their judgements in the first place.

The scientific literature is not perfect

I keep stressing that readers should always approach scientific reports and papers critically and intelligently. The problems identified by the Cochrane reviewers are inevitable and should always be taken into account by sensible readers. Simple reliance on the abstract of a paper or report often gives a misleading interpretation of the findings. Unfortunately, even unmotivated reporters tend not to read reports in full. Motivated activists will purposely resort to cherry-picking and distortion.

Decision-makers don’t necessarily need perfect scientific papers as they have to consider far more than the abstract conclusion of a scientific paper. There are the democratically expressed views of the electorate and the real situation where social health policies are put into effect. While the Cochrane reviewers expressed “limited confidence” in the size of the effect of CWF on tooth decay. However, policy-makers are more interested in the fact that there is a positive effect on oral health than the possible “theoretical” size of that effect.After all, policy-makers have to also consider the possible role of confounding effects like alternative sources of fluoride, the quality of dental health in the area, socioeconomic status of the population and school health programmes when making decisions about local CWF programmes.

Conclusions

1: The Cochrane reviewers’ use of terms like “bias” and judgment of studies as being of poor quality is inappropriate for evaluation of a social health policy. According to them:

“The main areas of concern were confounding and lack of blind outcome assessment. The evidence was additionally downgraded for indirectness due to the fact that about 71% of the caries studies that evaluated the initiation of water fluoridation were conducted prior to 1975.”

Yet they qualify this by acknowledging such judgement of “bias” and poor quality is inappropriate for a social health policy. And it was their own criteria for rejecting studies that produced a paucity of more recent studies (see Cochrane fluoridation review. I: Most research ignored).

2: These qualifications were not mentioned in the review’s Abstract or Plain Language Summary. I believe this was irresponsible of the authors. especially  given the controversial nature of the subject and the well-understood fact that media reporters rarely read beyond abstracts and summaries.

Such inappropriate and unqualified language provides a godsend to anti-fluoridation propagandists who are already cherry-picking and misrepresenting the review’s main findings.

3: We can remove the inappropriate and judgmental language and still accept that many of the problems identified in the review are inevitable for studies of social health measures. The review actually acknowledges that.

However, the sensible reader of scientific literature is surely aware of these problems. Any research paper must be assessed intelligently and critically – especially regarding the treatment of confounding factors. This is a point I have continually stressed in my posts on this subject.

In my experience, it has been the confirmation bias of anti-fluoride activists which leads them to ignoring such advice. One need only consider their use of studies related to IQ and fluoride in areas of endemic fluorosis, or their recent promotion of poor quality papers claiming a relationship between CWF and Attention-Deficit Hyperactivity Disorder (see ADHD linked to elevation not fluoridation) or hypothyroidism (see Paper claiming water fluoridation linked to hypothyroidism slammed by experts). 

I urge readers to follow this same advice with the Cochrane review. Don’t accept media reports or a limited reading of its Abstract or Plain Language Summary.

Read the whole review – intelligently and sceptically.
See also:

Cochrane fluoridation review. I: Most research ignored
Cochrane fluoridation review. III: Misleading section on dental fluorosis

Similar articles

Cochrane fluoridation review. I: Most research ignored

With the publication of the new Cochrane Fluoridation Review (Water fluoridation for the prevention of dental caries) we have, once again,  both fluoridation supporters and opponents claiming it as evidence for their contradictory positions. They surely both cannot be right.

The reality is that any such review is going to have its own complexities and limitations which allow committed believers on either side to confirm their biases. Unfortunately, this confirmation bias is promoted by inadequate reporting relying on “sound bites” from the executive summary.  Real understanding of the review and its results requires more thoughtful analysis, a reading of the full review and not just media reports and a bit of thinking about its limitations.

The post is the first of three articles looking a bit more deeply at the Cochrane fluoridation review. Here I discuss the strict criteria used by the review team for selecting the studies they considered,  the limitations this has caused for their findings and the misinterpretation of the review results this has produced.

97% of fluoridation research ignored

This seems amazing – why ignore so much of the research? We can understand the need to filter poor research or poorly reported claims. But 97%?

Yet, that is what the review reports – and summarises in their Figure 1:

Cochrane-1

The high exclusion rate was caused by the review teams decision to only consider studies which conformed to strict criteria:

“For caries data, we included only prospective studies with a concurrent control, comparing at least two populations, one receiving fluoridated water and the other non-fluoridated water, with at least two points in time evaluated. Groups had to be comparable in terms of fluoridated water at baseline. For studies assessing the initiation of water fluoridation the groups had to be from nonfluoridated areas at baseline, with one group subsequently having fluoride added to the water. For studies assessing the cessation of water fluoridation, groups had to be from fluoridated areas at baseline, with one group subsequently having fluoride removed from the water.
For the purposes of this review, water with a fluoride concentration of 0.4 parts per million (ppm) or less (arbitrary cut-off defined a priori) was classified as non-fluoridated.”

This criteria requiring measurements at several time periods and the inclusion of data from before the commencement of fluoridation was probably the main reason for excluding studies. In the review’s Table “Characteristics of excluded studies” the most often mentioned reason was “Evaluated caries in a single time point cross-sectional study.”

OK, you can sort of see the logic behind these strict criteria:

“The cross-sectional studies, whilst able to provide information on whether water fluoridation is associated with a reduction in disparities, are not able to address the question of whether water fluoridation results in a reduction in disparities in caries levels.”

But this inevitably resulted on consideration of only a small part of the available research:

“155 studies (162 publications) met the inclusion criteria for the review. However, only 107 studies (15 caries studies; 92 studies reporting data on either all fluorosis severities or fluorosis of aesthetic concern) presented sufficient data for inclusion in the quantitative syntheses.”

Inability to comment does not mean no effect

Exclusion of so many important studies meant the review was unable to come to any conclusions about important aspects like the effect of community water fluoridation (CWF) on socioeconomic difference in tooth decay,  the effect of stopping CWF programmes on later tooth decay and the effectiveness of CWF in reducing adult tooth decay.   Yet, in the review’s discussion they did make note of research which did draw some conclusions in these areas – research they refused to consider. (And it is rather ironic that one of the review’s authors, Helen V. Worthington, has co-authored several papers which conclude that CWF does reduce socioeconomic differences in dental health).

Of course, anti-fluoride propagandists have chosen to misrepresent the review – reporting its inability to draw conclusion on these questions as evidence that CWF does not influence socioeconomic differences, is not effective for adults and tooth decay does not increase when CWF is stopped!  (See Misrepresentation of the new Cochrane fluoridation review). Clear misrepresentation – but helped by the combination of exclusion of most research and the  vague language used in the review summary.

And you do sort of wonder at ignoring so much evidence when considering issues related to community health. Was Cochrane throwing away the baby with the bathwater?

Confirmation fluoridation is effective

The strict exclusion criteria enabled the review team to winnow studies down to a small number which could be analysed quantitatively. They were able to confirm from analysis of 9 studies that CWF:

“resulted in a 35% reduction in decayed, missing or filled baby teeth, and 26% reduction in decayed, missing and filled permanent teeth.”

But the strict exclusion criteria, specifically rejection of cross-sectional studies, is still a fly in the ointment. Recent studies of situations where fluoridation has been in operation for a long time did not fall within the strict selection criteria because pre-fluoridation data would not realistically be available in most cases. The review consequently did not consider properly the recent evidence – 71% of the research considered occurred before 1975!

The review, therefore, raised the issue of how applicable their findings are to the current situation in developed countries because of improved dental care and use of fluoridated toothpaste. A reasonable proviso which could have been discussed properly using the research they had excluded. But again a proviso which enables misrepresentation by anti-fluoride propagandists who imply that their findings are irrelevant to our current situation.

The review authors acknowledge that exclusion of such data presents a problem for their conclusions:

“In the past 20 years, the majority of research evaluating the effectiveness of water fluoridation for the prevention of dental caries has been undertaken using cross-sectional studies with concurrent control, with improved statistical handling of confounding factors (Rugg-Gunn 2012). We acknowledge that there may be concerns regarding the exclusion of these studies from the current review. A previous review of these cross-sectional studies has shown a smaller measured effect in studies post-1990 than was seen in earlier studies, although the effect remains significant. It is suggested that this reduction in size of effect may be due to the diffusion effect (Rugg-Gunn 2012); this is likely to only occur in areas where a high proportion of the population already receive fluoridated water.”

Of course, the review team was correct to raise the question of the possible reduced efficacy of CWF in modern developed societies. But doesn’t that suggest they should not have used such restrictive criteria in selecting studies to consider? And isn’t it irresponsible to leave the impression that CWF is no longer effective when they excluded the studies which could have provided better answers?

Conclusion

The Cochrane fluoridation review suffers from the fact that only 3% of available studies were considered. The restrictive selection criteria enable quantitative estimates showing  CWF is effective for children but excluded the possibility of answering questions related to the effectiveness for adults, the ability of CWF to reduce socioeconomic differences in oral health,  the effect of stopping fluoridation on later tooth decay and whether improved availability of dental treatments and use of fluoridated toothpaste has reduced the efficacy of CWF in modern developed societies.

The language of the review report itself encourages misinterpretation – and this is even worse in their blog post about the review – Little contemporary evidence to evaluate effectiveness of fluoride in the water.” Here they repeatedly refer to lack of evidence but only explain this is due to their exclusion of such evidence in a few places. What is the uninformed reader, who does not bother to read the full document, make of points in the summary such as:

  • “There is insufficient evidence to determine the effect of water fluoridation on disparities in caries levels across socio-economic status
  • There is insufficient evidence to determine the effect of water fluoridation on caries levels in adults
  • There is insufficient evidence to determine the effect of removing water fluoridation programmes from areas where they already exist”

Finally, anti-fluoride propagandists are motivated enough to misrepresent the findings in any fluoridation review or other documents. The very restricted selection criteria used by the Cochrane review and the language of its summary and news reporting of the review is a bit of a godsend to such propagandists.

Expect to see a lot of cherry-picked quotes from the Cochrane review. Twisted to turn the lack of evidence of effects (due to the exclusion of studies) into evidence for no effect.


My next article on the Cochrane review deals with its discussion of “bias” and poor quality in the studies it considered. See Cochrane fluoridation review. II: “Biased” and poor quality research.

See also:
Misrepresentation of the new Cochrane fluoridation review
Cochrane fluoridation review. II: “Biased” and poor quality research
Cochrane fluoridation review. III: Misleading section on dental fluorosis

Similar articles

Cherry-picking and misinformation in Stan Litras’s anti-fluoride article

This is the second article in a series critiquing contributions to the Fluoride Free NZ report Scientific and Critical Analysis of the 2014 New Zealand Fluoridation Report.”

My first article Peer review of an anti-fluoride “peer review”  discussed Kathleen Theissen’s contribution. (It also discussed a draft contribution by Chris Neurath which does not appear in the final version).

I will shortly post a 3rd article discussing H. S. Micklem’s contribution.

See The farce of a “sciency” anti-fluoride report for an analysis of the close relationships between the authors and peer reviewers of the Fluoride Free report and anti-fluoride activist groups.


There is a lot in Stan Litras’s article to criticise – there is a lot which is misleading or outright wrong. I hope Stan will seriously consider my criticisms and respond to them, especially where he thinks I am wrong.

My criticisms should also be considered by Bruce Spittle and Hardy Limeback who Fluoride Free NZ listed as “peer reviewers” of Stan’s article. They must bear some responsibility for allowing the article to go ahead without the necessary corrections.

Litras makes many of his criticisms of community water fluoridation (CWF) in passing – without argument or evidence. But he declares:

“My comments will focus on the gross over statement of the purported benefits of fluoridation in our society, New Zealand, 2014.”

So, I will start with the claims he makes on this.

“Overseas studies” – The WHO data

Central to this are Stan’s assertions:

“The “elephant in the room” is that while decay rates fell in areas where fluoridation was implemented, it also fell in areas that weren’t, often at a faster rate. (8)”

And

“Globally, fluoridation is seen to make no difference to reduced decay rates, there being no difference between the few countries which use artificial fluoridation, and those that don’t. (8,7)”

His only evidence for this is a figure prepared by Chris Neurath from the Fluoride Action Network – using data from the World Health Organisation (WHO). Here it is in a slightly simpler version to the one used by Stan.

I am amazed that anti-fluoride propagandists keep using this graphic as “proof” that fluoride is ineffective. But they do – which can only mean they haven’t thought it through.

While the plots do show improvements in oral health for countries independent of fluoridation they say nothing about the effect of fluoride. Simple comparison of countries obscures all sorts of effects such as differences in culture, history, social and political policies, etc. Such plots are also influenced by changes and differences in dental treatment and measurement techniques.

Robyn Whyman in his report Does delayed tooth eruption negate the effect of water fluoridation? exposes the little trick Stan is trying to pull with the WHO data:

“Studies that appropriately compare the effectiveness of water fluoridation do not compare poorly controlled inter-country population samples. They generally compare age, sex, and where possible ethnicity matched groups from similar areas. Inter-country comparisons of health status, including oral health status, are notoriously difficult to interpret for cause and effect, because there are so many environmental, social and contextual differences that need to be considered.”

There are some within country data within the WHO data set Neurath used which can give a better idea of the beneficial effects of fluoridation. This plot shows the results for the WHO data for Ireland. A clear sign that fluoridation plays a beneficial role.

Neurath covered up evidence for the benefits of CWF by simply using the mean of fluoridated and unfluoridated areas for countries like Ireland and New Zealand. Also, the straight lines in Chris Neurath’s plots are a real give away to the poor quality of the data used. Two data points for each country!

New Zealand – Cherry-picking the MoH data

I have criticised Stan’s misrepresentation the Ministry of Health (MoH) data before. At the time he was using and misrepresenting some of my own graphics on his business website. He has since removed the offending article but now he returns with a vengeance – with tables and figures of his own.

This has given him free hand to cherry-pick and misrepresent to his heart’s content.

He claims:

“Ministry of Health figures recorded every year in 5 year olds and year 8s (12-13 year olds) consistently show minimal or no differences between fluoridated and nonfluoridated areas of NZ.”

stan_1

Cherry-picked data from Stan Litras

And he backs this up with a graph.

That looks about right. The data for 2011 shows 59.9% of 5 year olds in fluoridated areas were caries-free while 59.2% were carries free in non-fluoridated areas. No real difference.

But come on! A single data point, one year, one of the age groups for the fluoridated and unfluoridated areas! That is blatantly cherry-picking – as I mentioned in my article Cherry picking fluoridation data. In that I presented all the data for 5 year olds and year 8s, and for the total population and Maori, and for % caries free and decayed, missing and filled teeth (DMFT).

I have reproduced this data here in a simpler form using several figures.

caries-freeConsidering the % caries free data there are several points:

1: These do not “consistently show minimal or no differences between fluoridated and nonfluoridated areas” as Stan claims.

2. They do show a decline in differences between fluoridated and non-fluoridated areas in recent years.

3: This trend is less obvious for Maori but still present.

4: Stan has blatantly cherry-picked the  data points for 5 year-olds in 2011 to give him the least possible difference (see red circle in figure).

dmft

The data for decayed, missing and filled teeth (DMFT) shows similar trends.

Presumably both measures (% caries free and DMFT) are useful indicators of oral health but they probably convey complementary and not exactly the same information.

I discussed features of the graphs and their trends in in my articles Cherry picking fluoridation data and Fluoride debate: Response to Paul’s 5th article where I also discussed limitations in the data.

We need to appreciate this is just normal school clinic data, without technique standardisation for those making the measurements or proper recording of place of residence. The latter effect probably shows up more strongly after 2004 when a “hub and spoke” dental clinics system was introduced further confusing proper records of likely fluoride intake. One school dental clinic could serve a number of areas – both fluoridated and non-fluoridated. This mixing is a likely explanation for the apparent decline of the effectiveness of fluoridation after 2006.

So, yes, the MoH data is not straightforward. But this means it should be considered sensibly, taking into account its limitations and the social factors involved.  Instead, Stan has leapt in – found the data points which best fit his own biases and then tried to claim those data  are representative when they aren’t.

Stan presented another self-prepared graphic using data for the 4 different regions for 5-year-olds (see his page 27). He appears not to have used the correct data – at least for the Northern and Southern regions.  My own graphic for this shows differences to his. (Of course, the mistake may be mine – if Stan can show I am wrong I will happily delete this part from my critique).

region-correct

Again, that data should also not just be considered at face value – or selected to confirm a bias. It has limitations. For example in this case there were only 55 children in the fluoridated Southern region compared with 7568 in the non-fluoridated area. A footnote on the data sheet says:

“2. Excludes Southern DHB because data were not reported for 1 Jan-20 Feb 2012, and fluoridation status was not captured for most children throughout 2012, due to transition to a new data system. “

Proper consideration of such data must take these sort of limitations into account. But of course all Stan Litras did was select data to support his assertions and ignore the rest. Any limitations in the data did not concern him.

Lifetime benefit

Stan has a thing about the “lifetime benefits,” or lack of benefits, of CWF. Most studies of CWF have used data for children – data for adults is less common but there is still research literature on this available.

But all Stan did on this was to cherry-pick a graphic (Figure 53) from the NZ Oral Health Survey showing no significant change in DMFT for 65-74 year olds between the years 1976, 1988 and 2009. He then claims:

“Data from the NZOHS 2010 do not support statements of a lifetime benefit, indicating that the action of fluoride is simply to delay the decay. (13)”

But he has had to work hard to avoid other data like that in Figure 49 below which do show a significant improvement in the number of retained teeth of that age group. The Oral Health Survey report itself says:

“In dentate adults aged 65–74 years, the mean number fell from 17.1 to 12.1 missing teeth per person on average from 1976 to 2009.”

mising-teeth

Again, instead of cherry-picking, searching for an image to fit his story, Stan should have considered the data and figures critically and intelligently. Perhaps the DMFT data does not show what he claims because more teeth have been retained in recent years. The decline in missing teeth could have been balanced by increases in fillings due to increase in remaining teeth. The lack of a significant difference in DMFT actually suggests the opposite to what he claims.

Litras also misrepresent the York review on the question of benefits from CWF for adults. He says:

“The York Review found there was no weight of evidence to support benefit in adults or in low SES groups, or increase of decay in cessation studies. (7)”

Just not true. The York report says:

“One study (Pot, 1974) found the proportion of adults with false teeth to be statistically significantly greater in the control (low-fluoride) area compared with the fluoridated area.”

Sheiham and James (2014) stressed that a proper assessment of oral health problems should include data for adults as well as children. Recent research is starting to take up this issue. For example O′Sullivan and O′Connell (2014) recently showed that water fluoridation provides a net health gain for older Irish adults.

Systemic vs topical

Stan promotes the common mythology of the anti-fluoridation propagandist that any mechanism for a beneficial effect of fluoride in restricting tooth decay is purely “topical.” He claims:

“It has been widely accepted since the 1990s that any effect on tooth decay from swallowing fluoride is insignificant or non-existent. To quote: CDC 1999: “the effect of Fluoride is topical “ (5); J Featherstone 1999: “the systemic effect is, unfortunately, insignificant” (6).”

Let’s consider what the sources Stan cites actually do say. I will quote from the 2001 edition of Stan’s citation 5 which he (partly) cites on page 36:

“Fluoride works to control early dental caries in several ways. Fluoride concentrated in plaque and saliva inhibits the demineralization of sound enamel and enhances the remineralization (i.e., recovery) of demineralized enamel (12,13 ). As cariogenic bacteria metabolize carbohydrates and produce acid, fluoride is released from dental plaque in response to lowered pH at the tooth-plaque interface (14 ). The released fluoride and the fluoride present in saliva are then taken up, along with calcium and phosphate, by demineralized enamel to establish an improved enamel crystal structure. This improved
structure is more acid resistant and contains more fluoride and less carbonate (12,15–19 ) (Figure 1). Fluoride is more readily taken up by demineralized enamel than by sound enamel (20 ). Cycles of demineralization and remineralization continue throughout the lifetime of the tooth.”

topical-mechanism

And

“Saliva is a major carrier of topical fluoride. The concentration of fluoride in ductal saliva, as it is secreted from salivary glands, is low — approximately 0.016 parts per million (ppm) in areas where drinking water is fluoridated and 0.006 ppm in nonfluoridated areas (27 ). This concentration of fluoride is not likely to affect cariogenic activity. However, drinking fluoridated water, brushing with fluoride toothpaste, or using other fluoride dental products can raise the concentration of fluoride in saliva present in the mouth 100-to 1,000-fold. The concentration returns to previous levels within 1–2 hours but, during this time, saliva serves as an important source of fluoride for concentration in plaque and for tooth remineralization (28 ).”

(Note: Stan simply quotes the first part of this statement (in red) in his article (page 36) and completely omits the second part (in black) – presumably because he wants to deny a role for fluoridated water in influencing the saliva fluoride concentrations. This cherry-picking of the CDC statement is typical for anti-fluoride propagandists – see Fluoridation – topical confusion).

There is an attempt to confuse a “topical” or “surface” mechanism with a “topical” application (eg toothpaste or dental treatments). However, fluoride is transferred to saliva from food and drink during ingestion so that ingested fluoride also contributes to the “topical” or “surface” mechanism.

However Stan wants to deny a “topical” role for ingested fluoride and claims (page 36):

“The required elevation of baseline levels only occurs after using fluoridated toothpaste or mouth rinse, a concentration of 1,000 ppm or more instead of 1 ppm from water.(24)”

His citation 24 is to Bruun (1984) and he misrepresents that paper which actually said:

“It was concluded that direct contact of the oral cavity with F in the drinking water is the most likely source of the elevated whole saliva fluoride and that the increased availability of fluoride in the oral fluids has an important relationship to the reduced caries progression observed in fluoridated areas.”

Systemic role.

Featherstone does say:

“Fluoride works primarily via topical mechanisms which include (1) inhibition of demineralization at the crystal surfaces inside the tooth, (2) enhancement of remineralization at the crystal surfaces (the resulting remineralized layer is very resistant to acid attack), and (3) inhibition of bacterial enzymes. Fluoride in drinking water and in fluoride-containing products reduces tooth decay via these mechanisms. Low but slightly elevated levels of fluoride in saliva and plaque provided from these sources help prevent and reverse caries by inhibiting demineralization and enhancing remineralization. The level of fluoride incorporated into dental mineral by systemic ingestion is insufficient to play a significant role in caries prevention. The effect of systemically ingested fluoride on caries is minimal.”

There is some debate over the role of systemic fluoride exuded by salivary glands. Many feel the concentration is too low – but because its effect is also determined by the presence of calcium, phosphate, organic species and pH it is best not to be dogmatic about this. It is, anyway, difficult to separate salivary fluoride derived from transfer from food and beverage in the oral cavity from that exuded by the salivary glands from systemic sources.

Stan is determined to deny a role for systemic fluoride during tooth development asserting:

“the erroneous theory that fluoride incorporated into children’s developing tooth enamel would make teeth more resistant to decay.”

While often neglected because of the concentration on surface mechanisms with existing teeth the theory that fluoride is incorporated into the developing teeth of children and confers a degree of protection is far from erroneous.

Newbrun (2004), for example, stressed in a review of the systemic role of fluoride and fluoridation on oral health:

“The role of systemic fluoride in caries prevention is neither “minimal” nor “of borderline significance.” On the contrary, it is a major factor in preventing pit and fissure caries, the most common site of tooth decay. Maximal caries-preventive effects of water fluoridation are achieved by exposure to optimal fluoride levels both pre- and posteruptively.”

Cho et al (2014) presented data showing that children exposed to CWF during teeth development retained an advantage over those never exposed to it even after fluoridation ceased.

Let’s stop confusing the issue. Systemic fluoride may not play a role with existing teeth but it does during tooth development – even if the relative contributions of systemic fluoride and “topical” or surface fluoride to lasting oral health are difficult to determine.

Tooth eruption delays

Stan resorts to special pleading when he claims with reference to NZ MoH data:

“Small apparent differences could be accounted for by other factors such as delayed eruption of teeth in fluoridated communities, therefore less time in the mouth exposed to plaque acids, ethnic distribution and urban/rural differences.”

He relies on the “York review” (McDonagh et al., 2000) to back up his “delayed tooth eruption” excuse:

“Importantly, the York Review noted that the variation of tooth eruption times between fluoridated and unfluoridated areas was not taken into account. (7)”

But that review actually said on this subject:

“It has been suggested that fluoridation may delay the eruption of teeth and thus caries incidence could be delayed as teeth would be exposed to decay for a shorter period of time. Only one study compared the number of erupted teeth per child. The difference was very small and in opposite directions in the two age groups examined, however no measure of the statistical significance of these differences was provided. Only one of the studies attempted to control for confounding factors using multivariate analysis (Maupomé 2000).”

Robyn Whyman has gone into this claim in more detail in his report Does delayed tooth eruption negate the effect of water fluoridation?Here he critiques Paul Connett’s reliance on this excuse and concludes from his review of the literature:

“The studies and reports cited by Professor Connett to try and validate an argument for delayed tooth eruption either do not make the claims he suggests, or do not have direct relevance to trying to assess the issue. The claimed association is at odds with the published literature which indicates minimal variation in eruption time of permanent teeth by exposure to fluoride. A rational explanation exists for the minimal variations that have been reported based on the relationship between fluoride exposure, caries experience in the primary teeth and emergence timing for the permanent teeth.”

The “delayed tooth eruption” excuse is nothing more than special pleading and straw clutching.

Socio-economic factors

Stan again misrepresented the York review regarding socio-economic effects on oral health and the effectiveness of CWF when he claimed “there was no weight of evidence to support benefit in adults or in low SES groups.” The York review actually said:

“Studies should also consider changes in social class structure over time. Only one included study addressed the positive effects of fluoridation in the adult population. Assessment of the long-term benefits of water fluoridation is needed.”

And

“Within the UK there is a strong social gradient associated with the prevalence of dental caries. This is found both in adults and in children. Those who are more deprived have significantly greater levels of disease. There is also geographical variation with the northwest of England, Scotland and Northern Ireland most severely affected. (Pitts, 1998; Kelly, 2000)”

There have been a range of studies internationally showing that fluoridation can aid in reducing differences in oral health due to socio-economic effects. See for example Cho, et al., (2014).

What happens when fluoridation is stopped

Stan briefly refers to this issue, citing (as anti-fluoridation activists always do) Künzel and·Fischer (2000). I will simply refer him, and interested readers to my article What happens when fluoridation is stopped? This boils down to the need to read the scientific literature properly as usually the anti-fluoridation activists ignore the details referring to fluoride treatments and procedures which replaced CWF.

There are a number of other points mentioned briefly by Stan Litras which could be discussed but this article is already too long so I will leave that to the comments section.

Conclusions

Stan Litras has simply indulged in blatant cherry-picking of data, and misrepresentation of the literature, in his critique of the recent review Health Effects of Water Fluoridation: a Review of the Scientific Evidence produced by the Royal Society of NZ together with the Office of the Prime Minister’s Chief Science Advisor. Perhaps we shouldn’t expect better from a political activist in the anti-fluoride movement but he, and Fluoride Free NZ, attempt to present this, and other articles in the collection, as objective and scientifically credible. It is neither – such cherry-picking and misrepresentation violates any scientific ethics and needs to be exposed for what it is. The Fluoride Free NZ claimed “peer reviewers,” Bruce Spittle and Hardy Limeback, must share responsibility because, by their endorsement, they signal their approval of such behaviour.

Note

I offered Stan Litras a right of reply to this post, or even an ongoing exchange with him along the lines of my debate with Paul Connett. He replied:

“I look forward to your comments on my review, as a lay person, but I cannot engage in a serious dialogue with someone who is not a peer with the same level of knowledge as myself in the dental field. “

Hopefully this means he will at least comment here, take issue with me where he thinks I am wrong and correct me where I am mistaken. I also hope than Bruce Spittle and Hardy Limeback will also take advantage of their right to comment here.

References

Bruun, C., & Thylstrup, A. (1984). Fluoride in Whole Saliva and Dental Caries Experience in Areas with High or Low Concentrations of Fluoride in the Drinking Water. Caries Research, 18(5), 450–456.

Centers for Disease Control and Prevention. (2001). Recommendations for Using Fluoride to Prevent and Control Dental Caries in the United States (Vol. 50, p. 50).

Cho, H.-J., Jin, B.-H., Park, D.-Y., Jung, S.-H., Lee, H.-S., Paik, D.-I., & Bae, K.-H. (2014). Systemic effect of water fluoridation on dental caries prevalence. Community Dentistry and Oral Epidemiology.

Cho, H.-J., Lee, H.-S., Paik, D.-I., & Bae, K.-H. (2014). Association of dental caries with socioeconomic status in relation to different water fluoridation levels. Community Dentistry and Oral Epidemiology.

Fluoride Free New Zealand. (2014). Scientific and Critical Analysis of the 2014 New Zealand Fluoridation Report.

Künzel, W.;·Fischer, T. (2000). Caries Prevalence after Cessation of Water Fluoridation in La Salud, Cuba. Caries Res, 34, 20–25. Retrieved from http://www.karger.com/Article/Fulltext/16565

McDonagh, M., Whiting, P., Bradley, M., Cooper, J., Sutton, A., & Chestnutt, I. (2000). A Systematic Review of Public Water Fluoridation.

Ministry of Health. (2010). Our Oral Health Key findings of the 2009 New Zealand Oral Health Survey. Wellington, Ministry of Health.

Ministry of Health (2014) Age 5 and Year 8 oral health data from the Community Oral Health Service. http://www.health.govt.nz/nz-health-statistics/health-statistics-and-data-sets/oral-health-data-and-stats/age-5-and-year-8-oral-health-data-community-oral-health-service.

National Fluoridation Information Service (2011): Does Delayed Tooth
Eruption Negate The Effect of Water Fluoridation? National Fluoridation Information Service Advisory June 2011, Wellington, New Zealand.

O′Sullivan, V., & O′Connell, B. C. (2014). Water fluoridation, dentition status and bone health of older people in Ireland. Community Dentistry and Oral Epidemiology.

Sheiham, A., & James, W. P. T. (2014). A reappraisal of the quantitative relationship between sugar intake and dental caries: the need for new criteria for developing goals for sugar intake. BMC Public Health, 14(1), 863.

Similar articles

 

Approaching scientific literature sensibly

thinking-conf-bias

We all suffer more or less from confirmation bias – it is just human.  So it’s natural for people to be selective, and to indulge in some cherry-picking and biased interpretation, when quoting scientific literature to support an idea they promote.

pseudoscience-cherry-picking

In the scientific community peer review and continual submission of ideas to scrutiny by colleagues helps keep this under control. But it can really get out of hand when used political activists use the literature to support their claims.

I have got used to anti-fluoride commenters on social media simply citing a paper or even providing a bare link, without comment, as if this somehow makes their claims irrefutable. Perhaps, in truth, they have not even read the paper they cite, or understood it, so do not feel confident discussing it.

But this tactic is particularly lazy – and stupid. To simply give a Google Scholar search as proof. Lately I have been presented with links to such searches to argue that fluoridation is toxic. Just a search for “fluoride toxicity.”

This is what that search produces – 234,000 hits:

Fluoride toxicity – 234,000 results

fluoride-toxicity

Sounds good to the uninitiated, I guess. It does seem to produce a large number. But does that mean anything?

What about searching for water toxicity. This produces over 2 million hits. Are we to assume from this that water is toxic, seemingly 10 times more toxic than fluoride?

Water toxicity – 2,190,000 results

water-toxicity

Yes, I know some social media do not offer much space for commenting but that should not be an excuse for such silly citations.

Similar articles

 

“What, me worry?” – distorting climate change data

Mad

Peter Gleick has a simple useful article in Forbes illustrating various ways that climate change deniers/contrariness cherry pick data to support claims that global warming is not occurring (see “Global Warming Has Stopped”? How to Fool People Using “Cherry-Picked” Climate Data). He is referring to an increasingly common argument that deniers/contrarians are promoting in OpEd columns and opinion pieces in newspapers. We have seen it here too, and the comments at New Zealand’s premium climate change denier blog Climate Conversation Group are full of such claims.

Recently I dealt with that argument as expressed by an old colleague of mine in a letter to the NZ Listener (see Open letter across the barricade). In the “open letter’ I reconstructed my own example of how global temperature data can be cherry picked to promote the “global warming has stopped” myth. But I had in the back of my mid  an animated graph which showed this more clearly.  Now I have found it – and reproduce it below. It’s from Skeptical Science (a mine of information) and is usually referred to as The Escalator.

I think this demonstrates the dishonesty of cherry-picking beautifully.
Mind you, it doesn’t stop motivated deniers and bloggers picking up these dishonest articles and promoting them to similarly motivated reader. Have a look at Local Whale Oil’s No Need to Panic About Global Warming.”
A classic example of the activity that goes on in the climate change denier echo chamber.