When scientists get political: Lead fluoride-IQ researcher launches emotional attack on her scientific critics

age credit: Science and Politics – Skeptically.org

It seems impossible to keep politics out of science. It’s a pity because politics can end up forcing science to produce the results desired by politicians. When this happens the ideal aim of science – the pursuit of objective knowledge – can get lost.

We rightly disapprove of external political and commercial influences on science. But there is another insidious form of politics derived from ego, personal ambition and the promotion of research by institutes and individual researchers. This is very often driven by the competition for research funding – the loudest researcher gets the grant. These days this is a real problem – the classical introverted scientist, no matter how bright, is often at a complete disadvantage when it comes to the fight over research funding.

In my own career, I have seen excellent researchers driven to redundancy simply because they did not have the political skills to fight for research funds. And at the same time, I have seen mediocre scientists, often producing poor quality or even misleading science, get those funds – simply because of their ambition and political skills.

These thoughts came flooding back to me as I read the new opinion piece by Christine Till, the leader of the research group that published several of the recent fluoride-IQ/ADHD and similar papers that I have critiqued here. Papers that have been heavily promoted by the authors and Till’s institute as well as the anti-fluoride/anti-vaccination crowd and, at the same time, extensively critiqued by the scientific community.

The citation  for her article is:

Till, C., & Green, R. (2020). Controversy: The evolving science of fluoride: when new evidence doesn’t conform with existing beliefs. Pediatric Research.

Unfortunately, the article is not a reply to critics or a good faith scientific engagement with the scientific issues. It is simply an attack on those who have made honest and respectful critiques. An attack which attributes unjust motives to her critics and comes close to personal.

Attempt to close down science by personal attacks

I discussed some of these issues before in my articles Scientific integrity requires critical investigation – not blind acceptance and Fluoridation science and political advocacy – who is fooling who?

In this case, I was concerned about the way  Dr William Ghali, one of the promoters of  Christine Till’s work, attacked and attempted to belittle scientific colleagues who were indulging in the normal peer-review process of critiquing published papers which they considered had faults. Nothing new about that scientific critique – it goes on all the time. It is expected by authors and, in the end, it helps to improve the science. I Personally think such critique should be welcomed by researchers.

If the opponents of scientific exchange like Dr Ghali are successful in their attempt to prevent such scientific debate then we are all losers. How can we trust scientific findings that are protected from scrutiny?

This is what is wrong with the opinion piece by Till and Green cited above. Instead of entering into a good-faith scientific exchange with their critics they attribute motives and biases to them. Even accusing them of attempting to prevent the progress of science. They accuse critics of a “tendency to ignore new evidence,” of  “overt cognitive bias” and of promoting  a “polarized fluoride debate.”

But, in fact, these critiques have come because the “new evidence” is not being ignored but is being evaluated. It is being critically considered. The article more or less admits this when it says “critics attacked the methodology of the study [Green et al (2019] and discounted the significance of the results.”

True, the so-called “fluoride debate” is polarised. After all, it is being promoted by anti-fluoride/anti-vaccination activists who are attempting to prevent or remove, a health policy known to benefit children. Till & Green may be unhappy that they have not been able to win over the scientific community with their paper but it is hardly honest to reject the critiques of the paper by calling them “attacks” or by claiming they “ignore” the evidence.

An admission the paper had difficulties

The article admits the  Green et al (2019) paper had difficulties right from the beginning. It took three attempts before a journal would accept it for consideration. Even then it ended up having “several additional rounds of review by the JAMA editors until we eventually reached a compromise.” This gives some substance to my speculation of problems in the review process which lead to the unprecedented publication of an editor’s note – a political action  I have never seen before (see If at first you don’t succeed . . . statistical manipulation might help).

They acknowledge that even their colleagues in environmental epidemiology “were initially sceptical.” And so they should have been – all new research should be reviewed sceptically and critically.

Refusing to engage scientifically

But the annoying thing is that these authors attempt to write off the scepticism and critical review of the wider scientific community as being due to “experts” (yes in quotes), “who held strong beliefs . .” This despite the fact that in the published critiques it is not “strong beliefs” which were presented, but detailed consideration of the methodology and statistical analyses used in the original paper.

All these critiques were made respectfully – and often with thanks to the Green et al (2019) for their new work. Yet Till and Green accuse these reviewers of making “vitriolic comments and claims with little scientific basis” – a comment which is, in itself, disrespectful to those who took time to make their critiques. They resort to smearing two of the reviews (by the UK-based Science Media Centre and Dr Berezow, a specialist from the American Council on Science and Health) by accusations these bodies are “both heavily funded by the pharmaceutical and food and beverage industries.” This funding smear is commonly used by anti-science activists who attempt to discredit scientific findings or analysis but refuse to consider the science itself.

They say of these two reviews that they claim “the results are driven by outliers” – yet a simple search shows that this comment simply does not appear in the cited reviews.  The critique of Dr Berezow from the American Council on Science and Health does not include either of the words “outlier” or “driven.”

The only reference to “outliers” in the Science Media Centre review was by Dr Oliver Jones, Associate Professor of Analytical Chemistry, RMIT University who wrote:

“The authors state that an increase of 1 milligram per liter (1 mg/L) increase in fluoride was associated with a 4.49 point lower IQ score but fluoride intake appears to have been below 1 mg/L for most people in the study, even for those with fluoridated water, and nearly everyone (bar a few outliers) had a fluoride intake of less than 2 mg/L (which multiple previous studies have shown is safe) . There is also a Lot of variation in the data – which makes drawing firm conclusions/ predictions from it difficult.”

A valid criticism which needed a response – not a smear.

My search for the word “driven” produced these two comments:

Dr Joy Leahy, Statistical Ambassador, Royal Statistical Society, wrote:

“if a woman is living in an area with fluoridated water during pregnancy, then her child is likely to grow up drinking this same fluoridated water. Therefore, it is difficult to say whether any association found is driven by the fluoride consumption in pregnancy, or an assumed fluoride consumption in the infant after birth.”

Prof Rick Cooper, Professor of Cognitive Science, Birkbeck, University of London, said:

“a significant decrease in IQ was found only in boys – girls showed a non-significant increase in IQ. The negative effect was driven by a small number of boys whose mothers had extreme levels of fluoride exposure, but even these children had IQ in the normal range.”

These are valid points that again deserve a scientific response yet Till & Green describe them as “vacuous claims exemplify attempts to manipulate the scientific evidence and manufacture doubt.”

We all support using new knowledge to adjust policies

Till & Green attempt to claim the high moral ground by asserting:

“Science advances by continuously challenging old ideas and adjusting our beliefs as new knowledge emerges, even if this new evidence conflicts with conventional wisdom or is inconvenient.”

Of course, this is true and I think it is dishonest of them to pretend this is not also the position of those who critiqued their paper. These reviewers were interested in looking at the new results, evaluating them and seeing how relevant they are. Seeing if they do indeed require us to adopt new thinking.

After all, look at what Dr Berozow, one of the critics they smeared by implying he was influenced by industry funding  and was falling back on “vitriolic comments” and “vacuous claim”, says in introducing his critique:

“The investigation by Green et al into the effect of maternal consumption of fluoride on the IQ of children is important. It is always wise to constantly evaluate and reevaluate long-standing public health practices in the light of new evidence.”

Till & Green are simply resorting to attributing motive and asserting their critics are not open to new knowledge as a way of avoiding facing up to the valid criticisms made by experts who reviewed and critiqued their work.

Confirmation bias – the pot calls the kettle black

We all suffer from confirmation bias and scientists (including Till & Green) are not immune. It is well understood that scientists are the last people to recognise problems in their own work. That is why peer review and open critique of scientific reports is so essential. But, in line with the whole approach of this opinion piece, Till & Green attempt to present a picture that only their critics suffer from this problem. They say:

“We typically fret about subtle biases, like recall bias and unmeasured confounding, but confirmation bias, the tendency to ignore or debunk data that does not conform to what we believe, is arguably a much larger problem.”

I find their attempt to belittle concerns about “unmeasured confounding” rather ironic. After all, this was the problem with Till’s original fluoride-ADHD work (which she used to win research grants for her later fluoride research) that I highlighted in Perrott (2018) Fluoridation and attention deficit hyperactivity disorder – a critique of Malin and Till (2015).

I am aware that Till has read my paper in its pre-publication and published forms but studiously ignores it. For example, the ADHD paper of Riddell et al (2109), which she co-authored, simply does not include Perrott (2018) in its discussion and continues to present Malin & Till (2105) as authoritative despite the obvious flaw that it ignored important confounders, and when these are considered their claim of a relationship between fluoridation and ADHD prevalence proved to be false.

So much for her attribution of confirmation bias to others – when she is obviously guilty of it in this case by ignoring “data that does not conform, to what [she] believes.”

The Till & Green opinion piece is unwise

Till & Green seem to have simply reacted emotionally to the reviews and critiques the Green et al (2019) paper received. They, of course, had the right – even the obligation – to respond scientifically to the reviews. But I believe their response in this article is unwise, maybe even professionally damaging,  and they should not have committed these emotional outbursts to print. After a cooling down period, it is possible they will withdraw the article – and that would be best for them in the end.

Similar articles

Anti-fluoridation propaganda now relies on only four studies. 6: Incestuous relationship of these studies

A Fluoride Action Network (FAN) propaganda video where Paul Connett urges listeners to consider only four studies when considering the possible harmful effects of fluoridation.

Paul Connett, director of the Fluoride Action Network (FAN), now claims “You only have to read four studies…” to come to the conclusion that community water fluoridation (CWF) is bad for your health. He wants you to ignore all the other research – which is just bad science.

But this is even worse than it looks because these four studies are hardly independent. They basically represent the work of one or two groups and weaknesses in the studies indicate the groups are basically “torturing” data to produce relationships which confirm their likely biases against CWF. The same researchers appear as authors on most of the published papers from these studies.

But that is not all. There is evidence of an “old boy/girl network” operating within these research groups where journal peer reviewers are selected from the same groups.

It’s called “taking in each other’s laundry.”

For earlier articles in this series see:

• Part 1: Anti-fluoridation propaganda now relies on only four studies. 1: Bashash et al (2018).
• Part 2: Anti-fluoridation propaganda now relies on only four studies. 2: Green et al (2019).
• Part 3: Anti-fluoridation propaganda now relies on only four studies. 3: Riddell et al (2019)
• Part 4: Anti-fluoridation propaganda now relies on only four studies. 4: Till et al (2020).
• Part 5: Anti-fluoridation propaganda now relies on only four studies. 5: Don’t censor yourself.

In this article, I discuss the incestuous relationship of the studies promoted by Connett and show these researchers have links to anti-fluoride activism.

Links between the four studies

One indication of the lack of independence of these studies is the fact that the papers have common authors. The figure below reveals these links between the studies via their authors with the names in red being authors on more than one of the papers.

The above diagram indicates the four studies came from no more than 2 groups.

Martinez-Meir is common to both groups – probably because her laboratory was responsible for the analysis of maternal urinary fluoride.

Christine Till has been responsible for several of these studies as she obtained funding on the back of the flawed Malin & Till (2015) study (see Leader of flawed fluoridation study gets money for another go).

Till and Lanphear appear to have responsibility for formulating and designing these studies.

So it is wrong to see these as completely independent studies. They will all be influenced by the biases of the groups involved and the links shown in the figure above suggest coordination in publishing their research findings.

This becomes more apparent when we look at the journals involved in publishing some of this work and the peer reviewers used.

Links with peer reviewers

Unfortunately, very few journals make available the names of peer reviewers or the contents of their reviews. A pity, as I would like to understand better the controversy that seemed to erupt during the journal review of the Green et al (2019) paper, for example. (This controversy resulted in an unprecedented brief statement from the Editor and an opinion piece by David Bellinger promoting the paper – see If at first you don’t succeed . . . statistical manipulation might help).

However, some of the researchers have published their fluoride work in the journal Environmental Health which in recent years has published peer reviewer names and the contents of their reviews. So let’s look at papers published by the authors Connett is promoting where this peer review information is available. They were all published in Environmental Health and are:

• Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14(1), 17.
• Liu, Y., Téllez-Rojo, M., Hu, H., Sánchez, B. N., Martinez-Mier, E. A., Basu, N., … Peterson, K. E. (2019). Fluoride exposure and pubertal development in children living in Mexico City. Environmental Health: 18(1), 1–8.
• Grandjean, P. (2019). Developmental fluoride neurotoxicity: an updated review. Environmental Health, 1–17.
• Malin, A. J., Bose, S., Busgang, S. A., Gennings, C., Thorpy, M., Wright, R. O., … Arora, M. (2019). Fluoride exposure and sleep patterns among older adolescents in the United States: a cross-sectional study of NHANES 2015 – 2016. Environmental Health, 1–9.
• Uyghurturk, D. A., Goin, D. E., Martinez-mier, E. A., Woodruff, T. J., & Denbesten, P. K. (2020). Maternal and fetal exposures to fluoride during mid-gestation among pregnant women in northern California. Environmental Health, 2, 1–9.

This diagram illustrates some of the papers and the reviewers and links the authors and reviewers.

Note: The extra papers considered in the figure are:

• Choi, A. L., Sun, G., Zhang, Y., & Grandjean, P. (2012). Developmental Fluoride Neurotoxicity: A Systematic Review and Meta-Analysis. Environmental Health Perspectives, 1362(10), 1362–1368.
• Bashash, M., Thomas, D., Hu, H., Martinez-mier, E. A., Sanchez, B. N., Basu, N., … Hernández-avila, M. (2018a). Prenatal Fluoride Exposure and Cognitive Outcomes in Children at 4 and 6 – 12 Years of Age in Mexico. Environmental Health Perspectives, 125(9).
• Bashash, M., Marchand, M., Hu, H., Till, C., Martinez-Mier, E. A., Sanchez, B. N., … Téllez-Rojo, M. M. (2018b). Prenatal fluoride exposure and attention deficit hyperactivity disorder (ADHD) symptoms in children at 6–12 years of age in Mexico City. Environment International, 121(August), 658–666.
• Grandjean, P. (2019). Developmental fluoride neurotoxicity: an updated review. Environmental Health, 1–17.

Yes, it’s a real network and perhaps it’s not necessary to follow the details of each link. We can see, though, that authors on these papers often appear as journal peer reviewers of other papers from these research groups.

I suspect this situation may be more common in science publishing than we realise – especially as journals now often ask authors to suggest possible reviewers and to specifically name researchers they do not want to review their work.

I think that is bad for the quality of published research. It’s easy to see that in a network like this peer review is done within the groupthink (or bias) that exists in such a network. I raised this problem when commenting on the peer review of an earlier paper from the network – Malin & Till (2015) – in my articles Poor peer-review – a case study and Poor peer review – and its consequences. In this case, the reviewers were fixated on chemical toxicity as the reason for health problems so did not consider all the other possible factors that might be responsible for the prevalence of ADHD diagnoses. They, therefore, missed completely possible regional effects which were at the time shown as important (see Perrott 2018).

This author/peer-reviewer network is particularly bad in situations like this where a controversial or even flawed paper gets approval simply because of the common biases of authors and peer reviewers.

Links with anti-fluoride campaigners

Notice in the figure above that two of the reviewers for Grandjean’s paper are members of FAN – senior members at that. While their contribution to improving the paper was probably minimal (Spittle’s comment – “The review capably considers recently available information and is highly pertinent to the public health” was worthless) the fact these reviewers were selected by the journal (and possibly by the author who is also the Chief Editor of the journal) indicates some influence.

These links indicate some sort of “under the table” influence and linking of researcher with FAN which probably explains why FAN often seems to have early information about upcoming publications which enable them to launch timely propaganda pieces.

Journals used for publication

Environmental Health is open access and a pay-to-publish journal. Pay-to-publish is becoming more common but many researchers steer away from these journals because they tend to have a reputation that payment encourages publication of bad research. On the positive side (as I said above) the open access policy, in this case, helps us see when the peer reviewers are and understand the problems I have discussed.

A relevant aspect of the author-peer-reviewer network, in this case, is the involvement of journal editors in the network as shown by this diagram.

NOTE: See notes for previous figure.

Phillipe Grandjean is also the author of Grandjean et al (2019) which appeared in the previous figure. I have written about his specific biases regarding fluoride in the past (see Special pleading by Philippe Grandjean on fluoride) and it is notable that as Editor in Chief of Environmental Health he refused to even consider for publication my paper critiquing Malin & Till (2015) (see Fluoridation not associated with ADHD – a myth put to rest).

David Bellinger is linked to the Bashash et al (2018a) study, not as an author, but as the note in the paper says:

“David Bellinger collaborated on the design and execution of this study’s cognitive testing.”

So it’s not surprising to see him authoring a promotion of the Green et al (2019) paper – although the inclusion of that promotion and the special note from the journal’s editor in that issue of the journal is very unusual.

Conclusions

The lack of independence in these four studies really reinforces the danger of limiting one’s reading. It’s not just a matter of restricting reading to four papers – its a matter of restricting information sources to one (or perhaps two) research groups influenced by the same groupthink and biases.

Group thinking and bias within research groups are not new. Nor is it a surprise that journals can be influenced by such group thought and bias and that this influences their acceptance of papers for publication. But if you are aware of the problem then you realise the need not to restrict your reading in the way that Connett is suggesting.

If anything, Connett’s statement is an admission that the overall findings of scientific studies on this issue do not support his case. He admits that all the studies anti-fluoride activists had been relying on in the past suffer from relating only to high fluoride concentrations. His plea people now restrict their reading to only four studies which really limits information sources to one or perhaps two research groups with a bias against CWF, is aimed at censoring the wider information availble.

Even more reason for readers to beware. One should never restrict information sources in the way Connett is suggesting.

See also:

• Poor peer review – and its consequences
• Poor peer-review – a case study
• Fluoridation not associated with ADHD – a myth put to rest
• Some fluoride-IQ researchers seem to be taking in each other’s laundry

Similar articles

 

 

 

 

 

Anti-fluoridationists put faith in new “strong” studies to provide evidence missing in draft NTP review

The US National Academies of Science (NAS) found the National Toxicity Programme’s (NTP) conclusion in their recent draft report that fluoride is neurotoxic was not supported by the reviewed evidence (see Another embarrassment for anti-fluoride campaigners as neurotoxic claim found not to be justified). This was a huge defeat for anti-fluoride activists who had been roundly promoting the conclusion of the draft report – despite the warning on every page that it “should not be construed to represent any NTP determination or policy” and that the draft had been “distributed solely for the purpose of pre-dissemination peer review.”

So the anti-fluoridationists declared belief that the NTP draft was going to mean the end of community water fluoridation (CWF) was dashed. But that has not stopped Paul Connett, head of the main US anti-fluoride group Fluoride Action Network (FAN) from putting his own spin on the NAS peer review report. In his attempt to present this huge defeat as somehow positive he issued a press release where he claims:

“If the recommendations are adopted, they will make the final report ‘iron-clad’ against criticisms. The NAS suggestions should strengthen the draft report’s conclusion that fluoride is a presumed neurotoxin in children, which is based on 149 human studies. This finding brings into question the long-standing assurances from public health officials that water fluoridation is safe.”

The NAS peer review is very critical – and not just about the fact that the draft conclusion was not supported by the reviewed evidence. There are criticisms of biased study selection, insufficient consideration of the effect of confounders and problems with statistical analyses used. But besides playing with words to misrepresent the peer review, Paul Connett has declared a belief that the NTP will now consider new studies, published since the draft report was released, and these will provide the evidence missing in the original draft. He is hoping the revised review will include new “strong” (in his opinion) studies which will finally swing things his way:

“Multiple strong scientific studies, at exposures relevant to fluoridation, have been published after the NTP’s review. They link fluoridation in Canada to greatly lowered IQ in formula-fed infants (Till 2020) and 300% higher rates of ADHD (Ridell 2019); fluoridation in USA with sleep disturbances in adolescents (Malin 2020); and fluoride with lower IQ by thyroid disruption (Wang 2020).”

Notably, Connett actively ignores, yet again, a new study which does not fit his bias because it reports a positive relationship between maternal prenatal urinary F and child IQ – the complete opposite of what he wishes (see The anti-fluoride brigade won’t be erecting billboards about this study). But Connett has been specific so we can objectively consider just how strong or weak these new studies are and evaluate for ourselves the likelihood that they can be used to provide support for the unwarranted conclusion expressed in the draft NTP report.

I have discussed several of these studies in previous articles but briefly consider them together here.

Till et al (2020) formula-fed child study

I discussed this study in my article Anti-fluoride propagandists appear not to read the articles they promote. It’s citation is:

Till, C., Green, R., Flora, D., Hornung, R., Martinez-mier, E. A., Blazer, M., … Lanphear, B. (2020). Fluoride exposure from infant formula and child IQ in a Canadian birth cohort. Environment International, 134(September 2019), 105315.

This has been blindly promoted by anti-fluoride propagandists as finding a decrease in IQ for children formula-fed as babies and a negative relationship of IQ for these children with drinking water F. However, the relationships are not statistically significant. It’s possible those anti-fluoride activists claiming a significant relationship had been influenced by Till’s previous conference paper where she indulged in special pleading for a non-significant relationship.

The findings reported by Till et al (2020) are summarised below:

While there are no significant relationships for IQ (FSIQ – Full-Scale IQ), the authors did report a significant negative relationship of performance IQ (PIQ) with water fluoride for children both breastfed and formula-fed as babies (and also for estimated F-intake for formula). Strangely, the anti-fluoride promoters of this paper generally don’t refer to this – perhaps because very few of them actually read the papers they promote.

Why there is a difference between IQ and PIQ is a bit of a mystery but there is a recommendation that PIQ, a subset of IQ tests, should not be used for clinical decision making because it is not consistent with current standards (see Beware of scientific paper abstracts – read the full text to avoid being fooled).

Riddell et al (2018) ADHD study

My article ADHD and fluoride – wishful thinking supported by statistical manipulation? discussed this study. Its citation is:

Riddell, J. K., Malin, A., Flora, D., McCague, H., & Till, C. (2019). Association of water fluoride and urinary fluoride concentrations with Attention Deficit Hyperactivity Disorder in Canadian Youth. Submitted to Environment International, 133(May), 105190.

The authors expressed disappointment that they could not find any relationship of ADHD prevalence to urinary F which they considered the best measure of F exposure. However, they were able to squeeze out some significant relationships with water F when the data was divided according to youth age. Significant for older youth but not younger. The findings of Riuddell et al (2019) are summarised in this figure.

The relationships are tenuous with large confidence intervals indicating their weakness. Manipulation of the data by age to find significant relationships reminds me of the saying that if data is tortured enough it will produce the answer you want

So, again, hardly a “strong” study and I cannot see Connett’s wish that the NTP will be able to justify its unwarranted draft conclusion with this.

Malin et al (2020) sleep “disturbances” study

I reviewed this study in my article Sleep disorders and fluoride: dredging data to confirm a bias. The paper citation is:

Malin, A. J., Bose, S., Busgang, S. A., Gennings, C., Thorpy, M., Wright, R. O., … Arora, M. (2019). Fluoride exposure and sleep patterns among older adolescents in the United States: a cross-sectional study of NHANES 2015 – 2016. Environmental Health, 1–9.

First of all, this paper has been poorly peer-reviewed. Effectively just an “in-house” review by coworkers all involved in publishing the same work (see Some fluoride-IQ researchers seem to be taking in each other’s laundry). This figure illustrates the relationships between the authors and their coworkers working with the same and similar data:

Relationships between Malin and her peer reviewers as indicated by joint publications. Links to the papers listed from the top are: Malin & Till (2015),  Thomas et al (2014), Bashash et al (2017), Marlin et al (2018), Malin et al (2018), Bashash et al (2018), Thomas et al (2018), and Riddell et al (2018)

I think this sort of self-selected peer-review during journal publication is almost scientifically corrupt. In my article, I concluded that the journal was chosen for publication and the absence of genuine peer review, together with the weak findings, indicated  “this is another poor quality paper on fluoride and health effects which make unwarranted claims – and which will be used by anti-fluoride activists in their campaign against community water fluoridation.”

The study used a range of parameters which facilitated dredging of the data to find apparently statistically significant relationships (see  Statistical manipulation to get publishable results). None of the relationships with blood plasma F were statistically significant (despite a bit of special pleading by the authors). They did report significant relationships with water F for sleep apnea, snoring, bedtime and waketime. An apparent reduction of snoring with an increase in water F baffled the authors. The confidence interval for sleep apnea was very large and small differences in bedtime and wake time are easily understood as connected with social differences between rural and city residents. There were no relationships with sleep duration.

So much for Connett’s description of this as a “strong scientific study” capable of reinstalling NTP confidence in their unsupported draft conclusion of a neurotoxic effect of fluoride.

Wang et al (2020) IQ study

This study is reported in:

Wang, M., Liu, L., Li, H., Li, Y., Liu, H., Hou, C., … Wang, A. (2020). Thyroid function, intelligence, and low-moderate fluoride exposure among Chinese school-age children. Environment International, 134(September 2019), 105229. https://doi.org/10.1016/j.envint.2019.105229

Although Connett claims the study was “at exposures relevant to fluoridation” this is not so. The study was aimed at measuring “low-moderate fluoride exposure affects” in the Chinese context and involved comparison of children from areas of “endemic and non-endemic fluorosis areas in Tianjin, China.” Water fluoride concentrations ranged from 0.20 to 3.90 mg/L and urinary F from 0.01 to 5.54 mg/L. Not at all comparable to the concentration where community water fluoridation (CWF) is present.

However, the study did separate results into different quartiles according to water F and urinary F concentrations and the data for the lowest quartiles are worth considering because they are for concentrations similar to that present where CWF is used.

I have extracted that data for quartile 2 of water F (0.7 to 1.00 mg/L) – a bit high compared with CWF but the data for the more relevant quartile 1 was not presented. Also the data for the second and third quartile of urinary F (UF) with the ranges 0.15 to 0.41 mg/L and 0.451 top 2.28 mg/L. Again these values are high.

Linear regression in these ranges showed no significant relationship of child IQ with either water F or Urinary F (UF) (see figure below). So much for Connett’s claim that this is a “strong scientific studies, at exposures relevant to fluoridation” that will lead to the NTP strengthening the NTP’s unsupported draft conclusion of a neurotoxic effect.

The elephant in the room – Santa-marina et al (2019)

Of course, Connett studiously ignores this study because it reported a positive relationship between maternal prenatal urinary F and child IQ (see The anti-fluoride brigade won’t be erecting billboards about this study).

The citation for this study is:

Santa-Marina, L., Jimenez-Zabala, A., Molinuevo, A., Lopez-Espinosa, M., Villanueva, C., Riano, I., … Ibarluzea, J. (2019). Fluorinated water consumption in pregnancy and neuropsychological development of children at 14 months and 4 years of age. Environmental Epidemiology, 3.

I have summarised the findings reported by Santa Marina et al (2019) below. These are certainly not going to provide any hope to Connett that the unsupported NTP conclusion will somehow be retrieved.

Conclusions

Connett is simply attempting, in his press release, to put a brave face on the embarrassment of the NAS peer review report. He and his supporters had been actively promoting the unsupported conclusion of the draft report – even suggesting it would shortly lead to the end of CWF. So a public embarrassment and he responds by holding out hope that his position will be retrieved by new studies.

His reliance on new studies is effectively an acknowledgement that the studies considered by the NTP were not adequate for the draft conclusion presented. However, he is wrong to describe these studies as “strong.” They are actually very weak and are hardly likely to change the final NTP assessment. He has also actively ignored, and hidden, the fact that the new studies are contradictory (Santa-marina et al 2019 produced the opposite result to what Connett wants) and conflicting (the different studies rely on different measures of cognitive ability and fluoride intake as well as manipulation of the data to find significant relationships).

The revised NTP report will not be produced soon as the peer reviewers identified a range of problems which will require major changes. There may well be even newer studies reported in the interim and these may not help Connett’s case any more the ones he cited. On top of that, the NTP may well discover older studies that their biased selection process missed in their original literature search (I still hold out hope that they will this time in include Perrott (2018) which, I believe, raises important issues about the effect of confounders ignored by other studies).

Similar articles

Industry-funded translation can introduce bias in selection of studies for scientific review

Image credit: Assessing and addressing bias in systematic reviews

The Fluoride Action Network (FAN), in the last decade, paid for translation of a lot of Chinese-language scientific papers linking high fluoride dietary intake to IQ deficits in children. They, of course, selected papers to fit their own ideologically-motivated bias. This is perfectly understandable for an activist group. But has this caused a bias in available English-language sources on this topic? And does this mean recent scientific reviews of this subject unintentionally suffer from selection bias?

I hadn’t considered this possibility before, but it is an issue raised in the recent US National Academies of Sciences (NAS) peer review of the US National Toxicity Program’s (NTP) review of possible neurotoxic effects of fluoride (see Another embarrassment for anti-fluoride campaigners as neurotoxic claim found not to be justified).

Use of FAN sources introduces biased study selection

The NAS peer reviewers are harshly critical of the NTP draft review. A central concern was the way the NTP evaluated the literature on the subject. The NAS peer reviewers say on page 3 of their report:

“The committee had substantive concerns regarding NTP’s evaluation of the human evidence as noted below. The strategy used for the literature search indicated that NTP used FAN as a source to identify relevant literature. The process by which FAN identified and selected studies is unclear, and that uncertainty raises the question of whether the process could have led to a biased selection of studies. Such a concern raises the need for a formal evaluation of any potential bias that might have been introduced into the literature-search process.”

OK, I am not impressed that the NTP used FAN as a source. FAN is hardly a reliable source and its “study tracker” certainly does not pick up anywhere near the full literature available (see Cherry-picking and ring-fencing the scientific literature). But, at first thought, I imagined that the FAN source simply produced a subset of anything that is picked up using a more reliable source like PubMed to do literature searches.

Injection of study bias into English-language scientific literature

But the NAS peer reviewers raise an important problem with reliance on FAN as a source and its effect on the available English-language scientific literature. On page 24 of their report they say:

“. . the process by which FAN identified and selected studies is not clear. FAN identified a number of studies published in Chinese language journals—some of which are not in PubMed or other commonly used databases—and translated them into English. That process might have led to a biased selection of studies and raises the question of whether it is possible that there are a number of other articles in the Chinese literature that FAN did not translate and about which NTP is unaware. NTP should evaluate the potential for any bias that it might have introduced into the literature search process. Possible ways of doing so could include conducting its own searches of the Chinese or other non–English-language literature and conducting subgroup analyses of study quality and results based on the resource used to identify the study (for example, PubMed vs non-PubMed articles). As an initial step in such evaluations, NTP should consider providing empirical information on the pathway by which each of the references was identified. That information would also improve understanding of the sources that NTP used for evidence integration and the conclusions drawn in the monograph.”

In a nutshell, FAN arranged and paid for translation of quite a large number of Chinese papers on this issue (fluoride intake and child IQ deficits). Naturally, they have selected papers supporting their political cause (the abolition of community water fluoridation) and ignored papers which they could not use to that end. It is therefore likely they have introduced into English-language scientific literature a biased selection of Chinese papers because FAN effectively “republished” the translated papers in the journal “Fluoride” – a well-known repository of anti-fluoride material.

Maybe I was wrong to assume anything from FAN would simply be a subset of what is available through more respectable searching sources. But, according to the peer reviewers, some of the translated papers may be picked up when FAN is used as a source of studies but not when PubMed or similar respected sources are used. A warning, though – many of the FAN-promoted translated studies have only been partly translated, maybe only the abstract is available. This is not sufficient for a proper scientific review (see Beware of scientific paper abstracts – read the full text to avoid being fooled).

I am not saying this bias introduction into the English-language scientific literature was intentional, but it is a likely end-result of their actions. Importantly, it is also a likely end-result of funding from big money sources (the “natural”/alternative health industry which funds FAN and similar anti-fluoride and anti-vaccination groups – see Big business funding of anti-science propaganda on health).

So, is this a way that big industry can inject their bias into the available scientific literature? A way to ensure that reviewers will, maybe unintentionally, convey this industry bais into their own summary of scientific findings?

Reviewers should make a critical assessment of studies

The FAN-promoted Chinese studies really do not contribute to any rational discussion of issues with CWF because they were all made in areas of endemic fluorosis. Ironically they often compare child IQ in villages where fluoride intake is high, with that in villages where the fluoride intake is low. It is the low -fluoride villages which are relevant to areas of CWF because their drinking water F concentrations are comparable.

In reality, these Chinese studies could be used to support the idea that CWF is harmless. Even if that is an inherent assumption for low fluoride intake in these studies.

So, perhaps the bias introduced to the literature by translation of the FAN-promoted studies really is of no consequence to the evaluation of CWF. However, consideration of reviews like the recent one by Grandjean (2019) indicates there is a tendency to simply extrapolate from high concentration studies to make unwarranted conclusions about CWF. In this case, the tendency is understandable as Grandjean is well known for his opposition to CWF and is often used by FAN to make press statements raising doubts about this health policy (see Special pleading by Philippe Grandjean on fluoride, Some fluoride-IQ researchers seem to be taking in each other’s laundry, and Fluoridation not associated with ADHD – a myth put to rest).

This was also a problem with the draft NTP review which produced the (unwarranted) conclusion “that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans.” The draft did actually mention that the conclusion “is based primarily on higher levels of fluoride exposure (i.e., >1.5 ppm in drinking water” and “effects on cognitive neurodevelopment are inconsistent, and therefore unclear”  for “studies with exposures in ranges typically found in the water distribution systems in the United States (i.e., approximately 0.03 to 1.5 ppm according to NHANES data).” But, of course, it is the unwarranted conclusion that gets promoted.

Conclusions

Reviewers need to be aware of this and other ways activist groups and big business can inject bias into the scientific literature.

This problem underlines the responsibility reviewers have of recognising all possible ways that biased selection of studies they consider can occur. It also means they should make every effort to include negative studies (not supporting the effect they may personally prefer) as well as positive studies. They also need to include all the findings (positive and negative) included in the individual studies they review.

In cases like the FAN-promoted Chinese studies, there is an obligation to at least note the possibility of bias introduced by activists and industry-funded translations. Even better, to ensure that the reviewer undertakes to independently search for all studies on the subject and arrange for translations where necessary.

Above all, reviewers should critically consider the quality of the studies they include in their reviews and not simply rely on their own confirmation bias.

Similar articles

Another embarrassment for anti-fluoride campaigners as neurotoxic claim found not to be justified

Anti-fluoride campaigners have made the classic mistake of promoting claims based on a draft report. Now peer reviewers have found the claim unjustified and the report will be rewritten. But will the anti-fluoride brigade stop making the claim?

Anti-fluoride campaigners have just lost another of their propaganda claims with the release of a US National Academies of Science (NAS) peer review of the recent National Toxicity Program’s (NTP) draft monograph discussing fluoride exposure and neurotoxicity.

Ever since the release of this draft last October anti-fluoride campaigners have been making hay out of this statement on page 2 of the draft (and repeated several times in its text):

“Conclusions: NTP concludes that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans.”

In general, these campaigners have presented this conclusion as a finding of the NTP, even though the draft includes this statement on every page:

“This DRAFT Monograph is distributed solely for the purpose of pre-dissemination peer review under the applicable information quality guidelines. It has not been formally disseminated by NTP. It does not represent and should not be construed to represent any NTP determination or policy.”

In a clear case of counting eggs before they hatch, the anti-fluoride campaigners now face the embarrassment of losing this claim because the NAS peer reviewers have found the conclusion is not supported by the evidence presented in the draft report. The NAS press release announcing the results of their peer review clearly says:

It “does not find that NTP has adequately supported its conclusion that “fluoride is presumed to be a cognitive neurodevelopmental hazard to humans.”

For readers wanting more detail these are the relevant documents:

• DRAFT NTP MONOGRAPH ON THE SYSTEMATIC REVIEW OF FLUORIDE EXPOSURE AND NEURODEVELOPMENTAL AND COGNITIVE HEALTH EFFECTS
• National Academies of Sciences press release 
• Review of the Draft NTP Monograph

Anti-fluoride activists response to the NAS peer review

The peer review has been out for only a few days and the anti-fluoride propagandists on social media have already denounced it (without reading it in most case) as only one committee’s opinion. Or they have indulged in the straw-clutching of quoting the NAS peer review’s sentence after the one that found the conclusion fo the NTP report was not supported. This says.

“The committee emphasized its finding does not mean that the NTP’s conclusion is incorrect; rather, further analysis or reanalysis is needed to support conclusions in the monograph.”

Perfectly normal for a scientific assessment and, in this case, a bit of a face-saver for the NTP (in many of its details the NAS peer review is quite scathing). And if the NTP has done a reasonable job of accessing the literature we all know that no “further analysis or reanalysis” will magically support the draft claim.

But, as an indication of the embarrassment of these people the “authoritative” comment on the peer review from Paul Connett, leader of the anti-fluoride US activist group Fluoride Action Network (FAN) almost reads like an endorsement of the peer review. Despite FAN having originally stressed the faulty NTP draft conclusion that “fluoride is presumed to be a cognitive neurodevelopmental hazard to humans” Connett does not even mention the NAS finding in his press release (see “Federal report finding fluoride lowers IQ of children reviewed by National Academy of Sciences.“) Strangely he quotes: “The NAS emphasized its finding ‘… does not mean that the NTP’s conclusion is incorrect.'” without even saying what that conclusion was.

The rest of Connett’s comments amount to spin – putting a brave face on his disappointment at this major loss. His claim that “The NAS suggestions should strengthen the draft report’s conclusion that fluoride is a presumed neurotoxin in children” is fanciful given that the material reviewed by the NTP simply does not support this claim.

He is also attempting diversion with his claim “The NAS review has been misinterpreted by fluoridation defenders. The NAS did not independently review the scientific evidence but instead limited itself to comments on whether the NTP clearly and thoroughly explained their methods” The fact is very few comments have been made by “fluoridation defenders” yet – the peer review has been public for only a few days – barely enough time for busy scientists to get around to reading it. And I have not seen a single person claim that the NAS peer reviewers “independently reviewed the evidence.”

What was required of the peer review?

The peer review report makes their action and purpose very clear. They were not tasked with making an independent review of the literature but their job went a lot further than limiting themselves to “comments on whether the NTP clearly and thoroughly explained their methods”

The NTP Project Information document Peer Review of the NTP Monograph on Systematic Review of Fluoride Exposure and Neurodevelopmental and Cognitive Health Effects) provided a list of requirements for the peer reviewers who were asked to “address the following questions:

• Has the systematic review protocol been followed and modifications appropriately documented and justified?
• Does the monograph accurately reflect the scientific literature?
• Are the findings documented in a consistent, transparent, and credible way?
• Are the report’s key messages and graphics clear and appropriate?  Specifically, do they reflect supporting evidence and communicate effectively?
• Are the data and analyses handled in a competent manner?  Are statistical methods applied appropriately?
• What other significant improvements, if any, might be made in the document?
• Does the scientific evidence in the NTP Monograph support NTP’s hazard category conclusions for fluoride in children and adults?”

The NAS peer reviewers clearly answered the last question with an emphatic No! This is not changed by the desperate straw clutching of propagandists who quote the (inevitable and polite) qualification that a further “further analysis or reanalysis” might provide support. Nor is it changed by Connett simply refusing to acknowledge or comment on that very important finding of the peer review committee.

Other findings of the NAS peer reviewers

The NAS peer review report is quite extensive (it’s 48 pages long) and covers several other important issues besides its rejection of the main conclusion. Issues related to how good the original protocol was and how closely it was followed, bias resulting from incomplete consideration of all the findings (negative as well as positive) and selection in the literature considered, issues related to statistical analysis and presentation, and problems with accounting for the range of different fluoride exposure measures and outcome parameters.

I might return to some of these issues later as I think they are relevant to other reviews in this area – especially the recent review of Grandjean (2019) – Developmental fluoride neurotoxicity: an updated review. This has similar faults – and some extra faults of its own.

But it is clear that in it’s current form the draft NTP report is not credible. Its conclusion is flawed and there are a number of other problems. It will have to be extensively rewritten to take the peer reviewers comments into account. I look forward to the new, and hopefully much better, version.

Similar articles

Some fluoride-IQ researchers seem to be taking in each other’s laundry

Image credit: Publish Peer Reviews. Illustration by David Parkins

Scientific peer-review is often touted as a guarantee of the quality of published research. But how good is peer-review? Does it guarantee poor science is weeded out? Or is it sometimes simply a bureaucratic manipulation aimed at endorsing a paper – despite its poor quality.

Any scientist with experience in publication and peer review knows that real-world peer-review is often poor. In fact, the whole process of peer review is being questioned these days because it doesn’t seem to work, is not taken seriously by many viewers who see it as an unpaid imposition on their time, and is open to manipulation by authors, journals and scientific cliques keen on promoting their own research and preventing publication of others.

I mentioned some of the problems from my own experience of scientific publication in Peer review – an emotional roller coaster and Peer review – the “tyranny” of the third reviewer and discussed the issue further in Sceptical humility and peer review in science. But I gave more specific examples of problems related to the peer review of papers that are often promoted by anti-fluoride campaigners in my articles Peer review, shonky journals and misrepresenting fluoride science, Poor peer review – and its consequences, and  Poor peer-review – a case study

The last two articles discussed in detail the peer review of a paper reporting a relationship between fluoridation and ADHD prevalence in the US – Malin & Till (2015): Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. This was later shown to be flawed as the relationship with fluoridation disappeared when other risk-modifying factors were included – see Perrott (2018): Fluoridation and attention deficit hyperactivity disorder a critique of Malin and Till (2015). But the poor, and biased peer-review of this paper may be a major reason for its publication despite its flaws.

The journal which published Malin & Till (2015), Environmental Health, makes the names of peer reviewers public and provides their reports. This is unusual – less than 3% of scientific journals do this despite the fact that about 60% of researchers favour open reports (see Publish Peer Reviews). Open access to the peer-review reports, in this case, provided a unique insight which enabled me to elucidate the relationships between the reviewers and authors, and the particular biases of the reviewers  (see Poor peer review – and its consequences, and  Poor peer-review – a case study).

Now the same journal has published a new paper – the first author is Ashley Malin of the Malin & Till (2015 paper) and one of the two peer-reviewers was Christine Till – the other author of the Martin & Till paper).

An incestuous peer-review by the journal?

This is the paper citation:

Malin, A. J., Bose, S., Busgang, S. A., Gennings, C., Thorpy, M., Wright, R. O., … Arora, M. (2019). Fluoride exposure and sleep patterns among older adolescents in the United States : a cross-sectional study of NHANES 2015 – 2016. Environmental Health, 1–9.

The reviewer’s comments and Ashley Marlin’s responses were made available by the journal (see the Open Peer Review Reports).

The second reviewer was Mara Tellez-Rojo. She is a co-author of several papers published from the prenatal maternal urinary F – Child IQ study in Mexico City, and along with Christine Till author of the Mexico city prenatal maternal urinary F – Child ADHD paper. The image below uses some links I found through published papers to illustrate the incestuous relationship between the main author of this paper and the two peer reviewers

Relationships between Malin and her peer reviewers as indicated by joint publications. Links to the papers listed from the top are: Malin & Till (2015),  Thomas et al (2014), Bashash et al (2017), Marlin et al (2018), Malin et al (2018), Bashash et al (2018), Thomas et al (2018), and Riddell et al (2018)

These people are taking in each other’s laundry. In effect, this journal peer-review of this paper was only an “in-house” review – one you might expect from work colleagues. A good journal peer-review should involve outsiders – and should avoid including coworkers and cooperators in research programmes.

The journal

Environmental Health is an “Open Access” journal. Authors pay to be published (in this case about US$2500) – which is never a sign of good quality. Authors also appear to often choose their reviewers – or at least suggest them and leave the final choice to the editors.

The Chief Editor of this journal is Phillipe Grandjean who is known to have sympathies for the anti-fluoride movement – often being used as an authority for statements about new research the anti-fluoride movement promotes. He is the author of the paper Grandjean and Landrigen (2014) that anti-fluoride people love to use as evidence of fluoride neurotoxicity – but in fact, only references Chinese studies from endemic fluorosis areas.

More importantly, in my experience, he demonstrates bias in the way he manages the journal. He refused to allow the journal to even consider my critique of the Malin & Till (2015) ADHD paper which was published in his journal. I describe this refusal in my article Fluoridation not associated with ADHD – a myth put to rest

My paper was eventually published in another journal (Perrott 2018) – but this has made it very easy for Christine Till’s group to studiously ignore it in their publication discussions – despite being very well aware of it.

The quality of Malin et al (2019)

One might expect that the need to use an open-access journal like Environmental Health and to choose “in-house” peer-reviewers indicates that the quality of the paper might not be the best. I believe that is the case and will make my own critique of the paper in a post here soon.

Conclusions

For all its faults as an “open source” journal which enables publication for a fee and has a suspect peer-review process, Environmental Health does sometimes make the names of peer-reviewers, the content of their reviews and the responses of authors public. This has been very useful in the case of Malin & Till (2015) and Malin et al (2019) as it shows how authors and peer-reviewers are sometimes “in-house” reviewers who take in each other’s laundry.

I have been critical of the quality of the fluoride-IQ and fluoride-ADHD and similar papers coming from Christine Till’s group. The naive use of statistics (for example relying on p-values rather than providing a full statistical analysis), the lack of transparency for some of their methodology, the biased choice of citations meaning they ignore work which does not support their claims, their bias in the discussion of their results, and their promotion of their findings is concerning. Especially considering how they attempt to make their findings relevant to public health policy and their claims and statements are promoted and supported by the anti-fluoride movement.

Others are also concerned. Recently 30 academic and health experts wrote a letter to the funding body which had financed some of the work of Till’s group. The letter outlined ten important concerns and requested public release of the data for one of Till’s papers so that it could be independently analysed (see Experts complain to funding body about quality of fluoride-IQ research).

I  suspect that if the peer-reviewers of other papers from Christine Till’s group, and the content of there reviews were made public we could see a similar situation to that found for Malin & Till (2015) and Malin et al (2019).

Perhaps if more journals followed Environmental Health’s policy of transparency about peer-reviewers and their comments authors would be less likely to choose colleagues for peer review. This practice of taking in each other’s laundry is not good for science.

Similar articles

 

We need more post-publication peer review

We often tout peer review as the reason for accepting the veracity of published scientific studies? But how good is it really? Does it ever match the ideal picture people have of it? And what about peer review before and after publication – are we neglecting these important stages?

Pre-publication peer review

Here I mean the collective process of evaluating ideas and presentations together with scientific colleagues. It’s great when it happens. Ideas flow and the critiques help prevent mistakes from persisting

This happens during discussion of research proposals and of research results. It happens during preparation of presentations.

But, unfortunately, it does not always happen – in fact, I suspect it may be relatively rare. When scientific reforms were introduced into New Zealand almost 30 years ago I noticed some scientific colleagues became less forthcoming about their ideas and research proposals. An air of competition seemed to destroy the previous cooperation.

Maybe things are better now. Hopefully there is less completion between individuals and within groups and institutions – although I imagine the competition between institutions will always be a problem. Quite apart from competing for grants humans simply identify with their own groups and fall victim to the “them vs us” problem.

Publication peer review

There is an impression that publication peer review happens only when the paper is submitted to a journal. But I think some of the best reviewing of a draft paper actually comes from colleagues before submission. That is why I strongly appreciated the institutional requirements I experienced that a draft paper be peer-reviewed within the institution before submission.

Unfortunately, not all institutions require this. I sometimes think many universities which don’t require this are taking “academic freedom” too far.

Perhaps some scientists see this as only landing extra work on them – but surely knocking a paper into better shape before submission is beneficial to both authors (getting a better draft)  and institutes (maintaining a reputation with journals).

Then there is the peer review organised by the journal. Many people think that is the only peer review. Just as well it isn’t because it can be very bad.

I am sure many poor quality papers slip through to be published simply because reviewers do not do a good job or spend insufficient time on that job. Personally, my impression of reviewers and journals drop when I see reviewers comments indicating a lack of attention or responsibility. Even worse, when I have had a paper accepted by an editor saying the reviewers had no comments I seriously questioned the quality of the journal and the advisability of submitting to it in future.

Still, when an author gets conscientious reviewers and comments indicating the paper has been read carefully an author can’t help but be appreciative – even if it means more work knocking the paper into shape.

As a reviewer, I always attempted to do a thorough job – even if it meant producing an over-long and detailed report. I once a received, via an editor, a response from an author I had reviewed expressing appreciation of the detail so I know such attention to detail is worthwhile.

I think most scientific authors will have occasionally faced the problem of brief or perfunctory reviewing of their submitted papers and can, therefore, understand the feelings behind that note.

Post-publication peer review

This is hardly ever considered. Once published the authors move on – their job is done. Readers also tend to be very accepting of published papers – after all peer review means that the paper’s findings must be trustworthy.

But this is obviously not the case. I think the slogan “reader beware” applies just as much to the scientific literature as it does to the news media. The reader should not automatically accept reported findings or conclusions as correct – just because the paper was peer-reviewed. They should do their own due diligence, consider all papers critically and avoid automatic acceptance.

Formal post-publication peer review can occur – but it is not as common as it should be. Some online journals provide space for readers comments. Helpful to the author but not adequate for proper evaluation.

The best post-publication peer review comes from published critiques because they become part of the established literature and available to anyone following up a subject or reviewing a field. Some journals provide space for shorter critiques of this sort – not requiring these authors to present new and original data but simply critique what has been published. Of course, despite the lower requirements such critiques should undergo their own peer review consistent with the policies of the journal.

The ethics of post-publication review

This is sore point for me – having had an editor recently refuse to consider a critique of mine (see Fluoridation not associated with ADHD – a myth put to rest). 

Surely there is a moral obligation for a journal, and its editor, to consider submissions of critiques of paper they have published? This is the obvious place for a critique – and the journal can normally then offer the right of reply to the original authors.  The writer of a critique should not have to search out an alternative journal – especially as the lack of new data or new research in a critique makes its acceptance by an alternative journal problematic. Nor should the original authors be denied an automatic right of reply which can be provided by the original journal.

Authors of a critique can face obstacles like the cost of publication. An original paper may be published in a journal which extracts publication fees from the author. It is the original authors decision whether or not to publish in such journals. But it seems unethical to expect the submitter of a critique to pay such fees. That puts a financial hurdle in the way of proper scientific peer-review. The original authors’ institution may be prepared to cover the cost of publication but institutions are unlikely to financially cover critiques in the same way.

The other obstacle is, of course, the attitude of editors. It is surely just common sense that critiques should undergo the normal peer review but when journals or editors refuse outright to even consider a critique, to not even enable it to undergo peer review, then that is ethically wrong.

Similar articles

Leader of flawed fluoridation study gets money for another go

Professor Christine Till has been given a $300,000 grant to test for harmful effects of fluoride.

Malin and Till (2015) published research indicating a relationship between fluoridation and Attention Deficit Hyperactivity Disorder (ADHD). However, that study was flawed because it omitted important confounders. When these are included the relationship disappears.

I analysed that study in my article ADHD linked to elevation not fluoridation where I showed the relationship of ADHD to elevation was much more important than fluoridation. Huber at al., (2015) published work confirming the relationship of ADHD with elevation. So, obviously, elevation is an important confounder and  Malin and Till (2015) did not consider it in their study.

My own analysis indicated that there were a number of other confounders which are related to ADHD – with correlations similar to (eg., educational attainment, proportion of the sate’s population older than 65  and Per Capita personal income) or better (mean state elevation, home ownership and % living in poverty ) than that for fluoridation. That rings alarm bells – why consider only one factor (fluoridation) if there are other factors which appear equally or more important? Isn’t that confirmation bias? (I concede that Malin and Till did include a socioeconomic measure in their statistical analysis – but this was clearly not enough).

I tested the relative importance of the different facts using multiple regression and – sure enough – found that once a few important confounders were included water fluoridation could not explain any of the variance in ADHD! The statistically significant factors were mean elevation, home ownership, and poverty. The contribution of fluoridation was not statistically significant in this multiple regression.

A model including mean state elevation, home ownership and poverty explains about 45% of the variance in ADHD – much better than fluoridation could (Malin and Till explained 27 -32% for the fluoridation data).

Now, I read that Professor Till has been given research finds to have another go and possible harmful effects of fluoride. (see York professor leads study that could help answer fluoride safety questions). She plans to look at data from a Canadian investigation of pregnant women exposed to  contaminants. She says:

“Our study employs a prospective design that includes biomarkers of exposure to fluoride, detailed assessment of potential confounders, a comparison group, and the use of sensitive cognitive and behavioural measures that have been collected in one of the world’s most comprehensively characterized national pregnancy cohorts (MIREC).”

Now, I am pleased she aspires to a “detailed assessment of potential confounders” but wonder how detailed this will be after the problems with the Malin and Till (2015) study.

I have not yet seen any published response to the Malin and Till paper – maybe the cost of publication (US$2020) that journal is discouraging critics. It certainly discouraged me (I do not have institutional support for publication costs). Nevertheless, I hope professor Till has been acquainted with some of the criticism of that paper so that she can pay more attention to important confounders in the coming work

We can draw a few lessons from this.

Be careful of published statistical relationships

These days it is so easy to hunt down data and do this sort of exploratory statistical searching for significant relationships. But a statistically significant relationship is not evidence of a real cause. For example, there is a strong relationship between the sales of organic produce and prevalence of autism – but I have yet to hear anyone seriously suggest the relationship is at all causal.

But the scientific literature is still full of such studies – and I guess the motivated author can easily find arguments and other data in the literature that they, at least, feel convincing enough to justify publication.

Refereeing of scientific papers is, on the whole, abysmal

All authors have a pretty good idea of which journals, and reviewers, will be friendlier to their work – and which would be antagonistic. It is only natural tosubmitt to the friendlier journal.

Unfortunately, the Malin and Till paper was submitted to a journal with editors known to be friendly to a chemical toxicity model of cognitive deficits. Further, it turns out that the reviewers chosen for the paper were also supportive of such an approach.

While one reviewer did suggest including lead as a possible confounder (again showing a chemical toxicity bias) none of them suggested consideration of other confounders more likely to be connected with ADHD.

I discussed the editorial and reviewer problems of the Malin and Till paper in Poor peer-review – a case study. (The journal, Environmental Health, has a transparent peer-review process which provides access to the names and reports of the reviewers.)

Again – another example of readers beware – even readers of scientific papers in credible journals.

Similar articles

Peer review – the “tyranny” of the third reviewer

Yes, I know, this video clip is old hat. It’s been used so many times it has got boring. And, maybe most readers won’t really relate to the way it is used here.

Still, those who have published scientific papers and suffered the emotional roller-coaster of peer-review will be aware of the problem of the “third reviewer.”

The more cynical will describe the peer review process as a bit of a farce. Usually, a journal will use three reviewers. Many times the first one presents a glowing, but undetailed report, recommending publication. The second reviewer will reject the paper out of hand, recommending that no way should it be published. But their recommendation is similarly undetailed.

The third reviewer will have done the work, hopefully in detail and conscientiously. And they may recommend publication – but only after their detailed critiqued is dealt with. But many authors dread their report because it usually means a lot more work for them – maybe, more experimental work.

I have seen authors get very emotional about specific peer reviews of their papers – and it is usually the “third reviewer” that upsets them. It means more work – and the detailed critique seems to be harder to handle than the undetailed outright rejection of the second reviewer. Perhaps the Hitler video clip is not too far from the truth.

Still, I think the peer-review process hangs on the “third reviewer.” There need to be far more reviewers who take their role seriously in this way. The first two reviewers are just lazy and opinionated – their comments are worth nothing.

For more on the peer-review process and its problems read Is scientific peer review a “sacred cow” ready to be slaughtered? « Science-Based Medicine

Similar articles

Poor peer-review – a case study

“Peer-review” status is often used to endorse scientific papers cherry-picked because they support a bias.

Many scientists are not impressed with the peer-review processes scientific journals use. Like democracy, this peer-review is better than all the available alternatives but it certainly doesn’t guarantee published scientific papers are problem-free.

Sure, peer-reviewed sources are better than others which have no quality control. But it is still a matter of “customer beware.” The intelligent users of scientific literature must do their own filtering – make their own critical judgements of the likely reliability of reported scientific findings.

Despite this people often use the “peer-reviewed” description to endorse published finding (especially if they confirm their own biases) without any critical assessment. This happens a lot in on-line debates of “controversial” issues.

Here I will go through the details of peer-review of a recently published paper which anti-fluoride activists are endorsing and promoting, but others are critcising. The paper is:

Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14.

I have discussed this paper in recent posts (see More poor-quality research promoted by anti-fluoride activists and ADHD linked to elevation not fluoridation). The journal, Environmental Health, has a transparent peer-review process which provides access to the names and reports of the reviewers. This reveals problems with the review process in this case. Below I discuss the responsibility of the authors, reviewers and the journal for the problems with this paper and its reported findings.

Authors’ responsibilities

The authors are clearly committed to a pet theory that fluoride is a neurotoxicant which could contribute to ADHD prevalence. Nothing wrong with that – we all feel committed to our hypotheses. We wouldn’t be human if we didn’t. But the best way to produce evidence for a hypothesis is to test it in a way that could prove it wrong.

In this case the authors found a correlation between ADHD prevalence in US states and the amount of community water fluoridation in each state. Trouble is, one can find just as good a correlation, or even a better correlation, with many other criteria for which state prevalence statistics are available. I listed a few in  ADHD linked to elevation not fluoridation. Some of these factors are also correlated with community water fluoridation suggesting the correlation reported by Malin and Till (2015) may be deceptive.

A proper test of the fluoridation hypothesis would include considering the effect of including such confounders together with fluoridation in their statistical analysis. Malin and Till (2015) did include one other criteria – the median household income for 1992 – but did not include any others. I find this surprising because they acknowledged ADHD results from interaction of genetic and environmental factors. While fluoridation is not usually considered a relevant factor things like smoking and premature births are and there is conflicting evidence about the role of economic factors like poverty and income.

In my article  ADHD linked to elevation not fluoridation I showed ADHD prevalence is better explained by a few of these factors without any input from water fluoridation.

I can’t help feeling the limited consideration of confounding factors results from a desire to protect the fluoridation hypothesis and therefore not test it properly.

Reviewers responsibilities

Again, such a desire is only human. But reviewers should have picked this up during their own considerations.

Interestingly, only one of the two reviewers raised possibility other confounders – specifically lead levels. This is of course valid as lead is a recognised neurotoxicant – but why did none of the reviewers question why other factors like smoking, premature births and social or regional factors were not considered?

I believe that is because both reviewers had research interests directed at chemical toxicity and not ADHD or similar mental characteristics. A matter of someone with a hammer only seeing nails.

The reviewers and their research interests are:
Marc Weisskopf whose reviews are available here and here.

“Some examples of my current work are exploring how exposure to, e.g., lead, manganese, and air pollution affect cognitive function and psychiatric symptoms; how exposure to Agent Orange and other herbicides used in Vietnam relate to the development of PD; and how formaldehyde and lead exposure relate to the development of ALS.”

Anna Choi whose review is here.

“Dr Choi’s research focuses on the effects of environmental exposures on health outcomes. She has been studying the birth cohorts in the Faroe Islands where exposures to environmental chemicals including mercury, PCBs, and PFCs are increased due to traditional marine diets. In addition, she also studies the effects of the contaminants on cardiovascular function and type 2 diabetes among the Faroese septuagenarians. She is also actively involved in the research on the impact of nutrients as possible negative confounders that may have caused an underestimation of methylmercury toxicity. Dr Choi’s other research interests include studying the adverse effects of fluoride exposure in children.”

Why were reviewers with a wider research experience not chosen? This journal allows authors to propose suitable reviewers themselves. Or the reviewers may have been chose by the associate editor handling this paper – Prof David Bellinger. His research focuses on the neurotoxicity of metabolic and chemical insults in children. So again it may just be the blinkered view of someone whose research background stressed the role of neurotoxicants rather than other factors likely to influence ADHD prevalence.

The journal’s responsibility

I noticed that one of the two chief editors (who have final say over acceptance of submitted papers) of this journal is Prof Philippe Grandjean. He himself has been actively promoting the idea that fluoride is a neurotoxicant purely on the evidence of the metareview of Choi et al (2012). Yes he is a coauthor of that review and Choi is one of the reviewers of the Malin and Till paper. The review of Choi et al (2012) related to areas of mainly China where fluoride concentrations are higher than used in community water fluoridation. Areas where endemic fluorosis is common.

I have to wonder if Grandjean’s well-known position on fluoride and community water fluoridation was a consideration in choosing this journal for publication.

Others have commented that the journal Environmental Health is considered low-quality based on its low impact factor. I do not know the area well enough to pass judgement myself. However, I notice that the journal charges authors for publishing their paper (£1290/$2020/€1645 for each article accepted for publication.) This sort of charge, associated with poor quality peer-review makes me suspicious. I have commented on these sort of journal before in my post Peer review, shonky journals and misrepresenting fluoride science.

Conclusions

This is one example of peer-review and paper acceptance which brings into question the idea of using  publication and peer-review as endorsement of a study’s   quality. I am sure this is not an isolated case. Even with the best of intentions journal editors and reviewers are limited by their own areas of expertise. Journal publication and peer-review is a far from perfect process – even if it is preferable to current alternatives.

Unfortunately activists will promote poor quality studies like this by blindly using the study’s peer-review status.

The intelligent reader should beware of such blind endorsements. Knowing the human foibles which exist in the research and publication processes such a reader will consider the contents of the paper and not rely on peer-review status. They will consider the evidence and conclusions critically. And if they don’t have enough background to make their own critical assessment they will consider the views of others with the required expertise and not blindly accepting what political activists tell them.

Footnote

Just came across this article referring to peer-review problems in journals published by BioMed Central – Major publisher retracts 43 scientific papers amid wider fake peer-review scandal.

BioMed Central publishes the Journal Environmental Health discussed in this post. I am not suggesting this paper was part of the peer-review racket discussed in the article. But the news item does highlight the point I am making that intelligent readers need to consider published scientific papers carefully and critically and not blindly rely on “peer-review” endorsement.

Similar articles