Tag Archives: statistics

Anti-fluoride authors indulge in data manipulation and statistical porkies

Darrell Huff & Irving Geis wrote a classic book – How to Lie With Statistics. They outline various ways data can be presented to give the wrong story. However, there is an even more naive use of statistics to misrepresent data – just declare that a relationship is statistically significant, don’t show any data or statistical analysis.

Unfortunately, many people are fooled by the use of those magical words – “statistically significant.”

I suppose the lay person could be excused – although it would pay even them to be a bit more sceptical about such claims. But it seems that even some “scientific” journals, or perhaps inadequate peer reviewers, can be fooled by those magical words. Here is an example in the paper by Hirzy et al., (2015) in the journal Fluoride. (Yes, I know, this journal is well known for its anti-fluoride stance and poor scientific quality but I would have thought the editor, Bruce Spittle,  would have picked this one up – even if they do not have an adequate peer review system. Perhaps the fact Spittle is one of the authors of the paper is a factor).

I critiqued the paper in my article Debunking a “classic” fluoride-IQ paper by leading anti-fluoride propagandists and have submitted a more formal critique to the journal (see – Critique of a risk analysis aimed at establishing a safe dose of fluoride for children.) But here I just want to deal with those magical words used in the paper – “statistically significant.”

Hirzy et al (2016) rely completely on data reported by Xiang et al., (2003) and claim they “found a statistically significant negative relationship between . . . .  drinking water fluoride levels and IQ.” Trouble is – you can search through the data presented by Xiang et al., (2003) and there is absolutely nothing to indicate a “statistically significant” relationship. Sure, that paper actually claims “This study found a significant inverse concentration-response relationship between the fluoride level in drinking water and the IQ of children.”  But there is no table or graphic presenting the individual data points and no statistical analysis for drinking water F and IQ. Rather surprising because Xiang et al., (2003) did present the individual data points for urinary fluoride and did present some results for statistical analysis of other relationships.

The trick behind the misleading use of Xiang’s data

However, what Xiang et al (2003) did do was separate their drinking water fluoride and IQ data into different ranges. This is a table of their result.

While group F was data for one village (Xinhuai) and the data in the other groups were for a separate village (Wamiao), there was no explanation of the criteria used for the groups – and the numbers in each group very tremendously. Over half the children (290 of the total 512) were in Group F and the size of the other groups seem to arbitrarily vary between 8 and 111.

This manipulation produces data which can be used to imply a statistically significant relationship. Do the statistical analysis for water F and IQ in the above table and sure you get a lovely straight line, a correlation of 0.96 and very significant statistically (p=0.003). But because of the manipulation, this says exactly nothing about the original data.

I will illustrate this by taking some data which Xiang et al (2003) did provide – for urinary fluoride and IQ. The data are illustrated in the figure below from the paper.

A statistical analysis of that data did show it was statistically significant – Xiang et al. (2003) cite a “Pearson correlation coefficient –0.174 , p = 0.003.” Now, that explains about 3% of the variance in IQ and I would have liked to see a similar analysis for water F as other workers have usually found weaker relationships for water F than for urinary F.

But let’s try using the manipulation of Xiang et al (2o03) and Hirzy et al (2016) to make the relationship between urinary F and IQ look a lot better than it is. I used a software tool to extract data from the figure – it didn’t extract all the points (264 out of a total 290) because of overlaps but statistical analysis of my extracted data gave a Pearson correlation coefficient of 0.16, p=0.010. Very similar to that reported by Xiang et al., (2003).

The tricky manipulations

I have absolutely no idea why Xiang et al., (2003) used different group sizes – so, to be fair, I have divided my extracted data into 6 groups of 44 pairs each (after sorting them into order based on urinary F) to produce the following table.

Urinary F IQ
A 1.79 105.57
B 2.30 89.45
C 2.30 77.72
D 2.69 68.58
E 2.48 56.25
F 2.69 40.10

This produces a lovely graph:

But, just a minute, I can get a better graph if I sort  the data according to IQ instead of urinary F:

But why stop there. If I choose different group sizes – remember Xiang et al., (2003) had groups ranging from 8 to 250 in size – I am sure I can get an even better presentation of the data.

TBut these graphs look far better than the one presented in Xiang et al (2003) for urinary F. We have taken data where the urinary F data explains only about 3% of the variance in IQ and produced graphics implying it “explains” up to about 75% of the variance. And we could “explain” more with a bit of extrra manipulation.

Conclusion

Data manipulation like this doesn’t change the fact that while the relationship between urinary F and IQ is statistically significant it only explains about 3% of the variance in IQ. This means that other factors, or confounders, should be considered – and when they are it is likely the significant relationship of IQ to urinary F would disappear.

Although Xiang et al., (2003) did not provide any statistical analysis to support their claim there was a significant relationship between water F and IQ I am sure the relationship is similar to that for urinary F – maybe even worse. Manipulating the data by using a range of groups of different sizes has certainly made the data look a lot better – but it is completely misleading.

I think it shocking that the authors of the Hirzy et al., (2016) paper have used manipulated data in this way – first to claim that fluoride in drinking water has a major negative effect on IQ and secondly to use such massaged data to work out a “safe dose.”

Worse, the journal Fluoride, and its peer reviewers, should never have accepted this paper without querying the claim of a significant relationship between drinking water F and IQ.

Similar articles

 

Advertisements

The chances of Royal Weddings arising randomly…

Royal Wedding - no chance - "not in a billion universes."

Now, I am not a Royalist. The whole subject bores me and I profess ignorance about the intricacies and origins of the institution. But I couldn’t help notice there was a Royal Wedding recently.  And having just read the blog post The chances of life arising randomly…., by New Zealand’s leading village creationist Ian Wishart, I thought I would apply his reasoning to help me understand the event.

You see. Something really strange happened. Hundreds of people spontaneously appeared at Westminster Abbey. Not only that. Half of them were adorned with strange contraptions on their heads and the other half dressed like penguins. But the coincidences extended even further. At the same time hundreds of thousands of people congregated in surrounding streets in London. But that’s not all. The coincidences were even more amazing. It’s estimated 2 billion (2,000,000) people watched the events on TV.

Continue reading

Treating statistics sensibly

People love to quote statistical studies to support their claims. And often its just a matter of confirmation bias. The statistical studies may not provide the support required – or may suffer from all sorts of flaws.

We see a lot of this in discussions on health, diet and life style. But I have also noticed statistics being liberally thrown around when religion and religious attitudes are discussed. If there is any area ripe for confirmation bias this is certainly it.

Consider this little graphic below which appear at a dating site OKCupid (see The REAL ‘Stuff White People Like’). Just imagine what negative conclusions one could draw about religion from that. To be fair, most references I have seen to it (all atheist – strangely, no religious sites are quoting it) do advise taking it with a grain of salt. (If you are interested have at look at the source. It provides other statistics from the study which will help make sense of this graph).

On the other hand, I have had statistical studies quoted at me which claim to “prove’ the religious people are happier, more honest, more moral, etc. Typical those quoting the studies have never bothered to check out the details and always ignore studies which might have provided different conclusions. In other words the normal confirmation bias.

Continue reading

New Zealand’s climate change deniers’ distortions exposed.

I am pleased to see that the attempt to promote a New Zealand version of “climategate” has more or less foundered. Sure the ACT party and some more extreme opponents of the findings of climate scientists are still campaigning (see for example Auckland Public Meeting: Climategate, NIWA and the ETS). And well know local climate change denier Ian Wishart managed to get international reporting of his slanderous press release (BREAKING: NZ’s NIWA accused of CRU-style temperature faking) in several more conservative and extreme international blogs and papers (for example BREAKING: NZ’s NIWA Accused of CRU-Style Temperature Faking, Climategate Scandal Spreads to New Zealand as MSM Continues Ostrich Act, Oops! Now New Zealand NIWA Accused Of Faking Data, New Zealand’s NIWA Gets Busted “Tricking” Their Climate Data and New Zealand Climate Data Shows Clear Evidence Of Fraud). But the New Zealand media has, in general, been more balanced in its reporting. The information from climate scientists at NIWA has been getting through.

Continue reading

Ranking NZ blogs with sitemeter data

There’s a school of thought that the only reliable way to rank blogs is by using actual sitemeter data for visits. Unfortunately, most blogs don’t make this information public. Perhaps if more did bloggers could compare their statistics with those for other sites or have a listed ranking. This would help their interpretation.

I have managed to identify 37 New Zealand blogs with public sitemeter stats, or who self report this information – mostly from Tumeke’s blog ranking reports (see nz blogosphere rankings : March 2009). So how do these sites rank amongst themselves? Have a look at the table below for the April ’09 data. And for those interested in comparing sitemeter data with the traditional blog ranking indices, and with the rankings of Tumeke, Halfdone and Open Parachute – look at my analysis below the table.

Update: Homepaddock has passed on (see Naked Stats) that her stats are also public so we now have 38 blogs on the list.

Continue reading

Ranking methods for NZ blogs

I had a go at ranking New Zealand blogs at the end of February (see Rating NZ blogs). A month later I think I will try again. But first – let’s check out the ranking method I am using.

This is similar to that used in the Atheist Blog Ranking which ranks each available statistics and then aggregates these ranks to produce an over all rank. Hopefully the procedure evens out the quirks inevitable in ranking using individual statistical measures.

Continue reading