Beware of scientific paper abstracts – read the full text to avoid being fooled

Inaccuracies are very common in scientific paper abstracts. Image credit: Science in the Abstract: Don’t Judge a Study by its Cover

I have come to accept anti-fluoride campaigners will never stop cherry-picking research and findings to support their claims but it annoys me when scientists themselves cherry-pick their findings to support a preconceived belief. I guess it’s only human, but it does reinforce my belief that one should never rely on abstracts to understand a paper – one should read the full text. Abstracts are the very place authors will cherry-pick their findings to promote a preferred theory.

Hilda Bastian wrote “Ivan Oransky from Retraction Watch says it’s “journalistic malpractice” to report on a study after only reading an abstract or press release” (see Science in the Abstract: Don’t Judge a Study by its Cover): 

“To be fair, even though it’s a worry, every inaccuracy isn’t critical. But the spin is serious. It’s not just about language that exaggerates or massages results. It’s also about choosing the most exciting results, even when that’s misleading.

Spin in press releases and media coverage often reflects the spin in abstracts. And until more research is open access, that will be all that most people can get to read. In 17 studies of inconsistencies of all kinds in abstracts of biomedical medical research papers, the rate of major problems ranged from 5% to 45% (median 19%)!”

Here is a rather blatant example in recent findings on possible effects of fluoridation for formula-fed babies. The research findings were reported in two places. Firstly a conference paper:

Till, C., Green, R., Flora, R., Hornung, R., Martinez-Mier, E., Blazer, BFarmus, L., … Lanphear, B. (2019). Fluoride Exposure from Infant Formula and Child IQ in a Canadian Birth Cohort. Environmental Epidemiology, 3.

And secondly, a journal paper which provides a full text and the findings in full:

Till, C., Green, R., Flora, D., Hornung, R., Martinez-mier, E. A., Blazer, M., … Lanphear, B. (2020). Fluoride exposure from infant formula and child IQ in a Canadian birth cohort. Environment International, 134(September 2019), 105315.

Let’s look at the actual findings and then consider how they were presented in the abstracts of these two reports.

The findings these reports rely on

This is the data taken from Till et al (2020) illustrating the differences in IQ measures for children (3 – 4 years old) living in fluoridated and nonfluoridated areas. Full-scale IQ (FSIQ) is the usual IQ measures and Verbal IQ (VIQ) and Performance IQ (PIQ) are indices resulting from subtests.

The only statistically significant differences are for VIQ (greater for children breastfed as babies living in fluoridated areas) and PIQ (lower for children formula fed as babies living in fluoridated areas)

The authors don’t comment on those results in either report but they also considered possible relationships between measures of fluoride exposure (water-F concentration and an estimate of F intake from formula). Here are the findings for these.

As we can see the relationship with FSIQ and VIQ are not statistically significant, but the ones with PIQ are. It’s a bit strange – and I do not have the understanding of IQ measurements to interpret the fact that while there were no relationships with the normal IQ measure, there were with the PIQ subtest. However, here is a statement from the reference used by Till et al (2020) in the Methods section (Wechsler, D., 2002. Wechsler Preschool and Primary Scale of Intelligence – Third Edition: Canadian. Pearson Clinical Assessment, Toronto, ON, Canada):

“Subtest and profile interpretation methods are not consistent with Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999) and should not be used in clinical decision-making until psychometric support for them is provided. In the words of Weiner (1989), the ethical psychologist will “(a) know what their tests can do and (b) act accordingly” (p. 829).”

The conference abstract

Here is the relevant section from the conference abstract for Till et al (2019). Only FSIQ is discussed and there is no mention of VIQ or PIQ:

“An increase of 0.5 mg/L fluoride concentration (the difference between fluoridated and non-fluoridated region) corresponded to a drop of 4.4 FSIQ points (95% CI: -8.34, -0.46, p=.03) in the formula-fed group. In contrast, this relationship was not significant in the breastfed group (B=-1.34, 95% CI: -5.04, 2.38, p=.48). Controlling for prenatal fluoride exposure weakened the association between water fluoride concentration and FSIQ in the formula-fed group (B=-3.88, 95% CI = -8.12, 0.37, p=.07).
Conclusions: The results, which indicate that fluoride concentration in drinking water was associated with lower FSIQ in children who were formula-fed, underscore the need to reduce the use of fluoridated water to reconstitute formula during infancy.”

So concentration on FSIQ – and special pleading to make the relationship important (we “need to reduce the use of fluoridated water”) despite the fact it is not statistically significant.

OK, I acknowledge p<0.05 is arbitrary and we should not have hangups on that but one should also consider the scatter of the data and how that affects the strength (or in this case weakness) of the relationship. Here is the actual data as presented in Till et al (2020) – that, and the large confidence interval ( -8.12 to 0.37), warn us that any relationship, significant or not, is extremely weak.

[I discussed this in my article Anti-fluoride propagandists appear not to read the articles they promote and mentioned that although I was able to digitally extract about 90% of the data from this figure my statistical analysis was quite different to that of Till et al (2020).This is probably an inevitable result of the large scatter in data points and the very weak relationship – any change in the position of a few data points could have a big effect on the regression result.]

The paper abstract

Here is the relevant section from the full-text paper abstract for Till et al (2020). Now, there is no mention of FSIQ – only PIQ is mentioned:

“An increase of 0.5 mg/L in water fluoride concentration (approximately equaling the difference between fluoridated and non-fluoridated regions) corresponded to a 9.3- and 6.2-point decrement in Performance IQ among formula-fed (95% CI: −13.77, −4.76) and breast-fed children (95% CI: −10.45, −1.94). The association between water fluoride concentration and Performance IQ remained significant after controlling for fetal fluoride exposure among formula-fed (B=−7.93, 95% CI: −12.84, −3.01) and breastfed children (B=−6.30, 95% CI: −10.92, −1.68). A 0.5 mg increase in fluoride intake from infant formula corresponded to an 8.8-point decrement in Performance IQ (95% CI: −14.18, −3.34) and this association remained significant after controlling for fetal fluoride exposure (B=−7.62, 95% CI: −13.64, −1.60).

Conclusions: Exposure to increasing levels of fluoride in tap water was associated with diminished non-verbal intellectual abilities; the effect was more pronounced among formula-fed children.”

Changing the goalposts – but activists don’t care

Before the full-text journal paper was available activists promoted this work – completely ignoring the lack of statistical significance in the relationship – or at least relying completely on the significant relationship when unadjusted for other important factors.

I have said that activists usually don’t bother reading actual papers, sometimes not even abstracts. But we still have activists promoting this research and citing the relationships for PIQ – but still calling it IQ!

For example:


ChristineM@Christi45657364

All the more reason not to ADD F to tap water. Today the 63rd human study linking F with reduced IQ was published. Fluoride exposure from infant formula and child IQ in a Canadian birth cohort: https://www.sciencedirect.com/science/article/pii/S0160412019326145 

Fluoride exposure from infant formula and child IQ in a Canadian birth cohort

Infant consumption of formula reconstituted with fluoridated water can lead to excessive fluoride intake. We examined the association between fluoride…


And


ChristineM@Christi45657364

Example of recent Canadian studies: NIH-funded study: Fluoride exposure from infant formula and child IQ in a Canadian birth cohort https://ncbi.nlm.nih.gov/pubmed/31743803  found children fed infant formula mixed with fluoridated water (vs. unfluoridated) had significantly lower IQ’s.


So, this woman is heavily promoting a paper with misinformation as the story is not about IQ or FSIQ at all. The only significant relationships are with a subtest which we are warned against using anyway. (Mind you, this woman seems to alternate her prolific anti-fluoride tweets with prolific anti-vaccination tasks so we can see she is hardly worried about the truth of her claims).

Reader beware

I have always urged people not to rely on social media, or even mainstrem media, reports about new research but to actually read the scientific reports themselves. But this example makes clear that it is not enough to just read abstracts – a very easy lazy habit to fall back on as there is more work to obtaining the full text of papers.

Similar articles

 

4 responses to “Beware of scientific paper abstracts – read the full text to avoid being fooled

  1. Here’s another abstract which Limeback is now spinning after the Gloucester article, and will undoubtedly keep hammering elsewhere. He claims: “The most heavily funded American prospective fluoride study was unable to show fluoridation works and you say ‘so what’? I don’t get this from reading the entire study, which seems to be just a subset of the full Iowa study, but that’s his spin.

    Longitudinal associations between dental caries increment and risk factors in late childhood and adolescence
    Alexandra M. Curtis MS John VanBuren PhD …
    JPHD
    First published: 12 May 2018

    https://onlinelibrary.wiley.com/doi/full/10.1111/jphd.12275

    Steven D. Slott, DDS

    Like

  2. This section on page 7 confirmed my first impression that the study should not be expected to show a relationship for fluoride because of the low standard deviation in F values – and other factors like high income:

    “Of note, fluoride exposure during late childhood and adolescence were not significantly associated with ADJCI. Although fluoride exposure generally plays an important caries preventive role (32), this cohort has relatively high fluoride exposures, low fluoride exposure variability, and compared to NHANES data, this cohort has low disease burden (4). It appears that adequate fluoride exposures resulted in low caries burden, and the low variability in fluoride exposures did not explain differences in ADJCI after adjustment for age, sex, SES, and brushing frequency.”

    Like

  3. That was the section that didn’t mesh with Limeback’s spin. My simplified view was that with the already low oral health burden of this relatively small cohort, there really was nothing left to improve beyond the lower ADJCI realized from the fluoride in the toothpaste.

    Like

  4. It’s typical of Hardy Limeback and other anti-fluoride/anti-vaccination propagandists – to resort to citation frenzy with no understanding, or even concern, about what the evidence is in what they cite.

    Like

Leave a Reply: please be polite to other commenters & no ad hominems.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s