Image credit: Publish Peer Reviews. Illustration by David Parkins
Scientific peer-review is often touted as a guarantee of the quality of published research. But how good is peer-review? Does it guarantee poor science is weeded out? Or is it sometimes simply a bureaucratic manipulation aimed at endorsing a paper – despite its poor quality.
Any scientist with experience in publication and peer review knows that real-world peer-review is often poor. In fact, the whole process of peer review is being questioned these days because it doesn’t seem to work, is not taken seriously by many viewers who see it as an unpaid imposition on their time, and is open to manipulation by authors, journals and scientific cliques keen on promoting their own research and preventing publication of others.
I mentioned some of the problems from my own experience of scientific publication in Peer review – an emotional roller coaster and Peer review – the “tyranny” of the third reviewer and discussed the issue further in Sceptical humility and peer review in science. But I gave more specific examples of problems related to the peer review of papers that are often promoted by anti-fluoride campaigners in my articles Peer review, shonky journals and misrepresenting fluoride science, Poor peer review – and its consequences, and Poor peer-review – a case study
The last two articles discussed in detail the peer review of a paper reporting a relationship between fluoridation and ADHD prevalence in the US – Malin & Till (2015): Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. This was later shown to be flawed as the relationship with fluoridation disappeared when other risk-modifying factors were included – see Perrott (2018): Fluoridation and attention deficit hyperactivity disorder a critique of Malin and Till (2015). But the poor, and biased peer-review of this paper may be a major reason for its publication despite its flaws.
The journal which published Malin & Till (2015), Environmental Health, makes the names of peer reviewers public and provides their reports. This is unusual – less than 3% of scientific journals do this despite the fact that about 60% of researchers favour open reports (see Publish Peer Reviews). Open access to the peer-review reports, in this case, provided a unique insight which enabled me to elucidate the relationships between the reviewers and authors, and the particular biases of the reviewers (see Poor peer review – and its consequences, and Poor peer-review – a case study).
Now the same journal has published a new paper – the first author is Ashley Malin of the Malin & Till (2015 paper) and one of the two peer-reviewers was Christine Till – the other author of the Martin & Till paper).
An incestuous peer-review by the journal?
This is the paper citation:
Malin, A. J., Bose, S., Busgang, S. A., Gennings, C., Thorpy, M., Wright, R. O., … Arora, M. (2019). Fluoride exposure and sleep patterns among older adolescents in the United States : a cross-sectional study of NHANES 2015 – 2016. Environmental Health, 1–9.
The reviewer’s comments and Ashley Marlin’s responses were made available by the journal (see the Open Peer Review Reports).
The second reviewer was Mara Tellez-Rojo. She is a co-author of several papers published from the prenatal maternal urinary F – Child IQ study in Mexico City, and along with Christine Till author of the Mexico city prenatal maternal urinary F – Child ADHD paper. The image below uses some links I found through published papers to illustrate the incestuous relationship between the main author of this paper and the two peer reviewers
Relationships between Malin and her peer reviewers as indicated by joint publications. Links to the papers listed from the top are: Malin & Till (2015), Thomas et al (2014), Bashash et al (2017), Marlin et al (2018), Malin et al (2018), Bashash et al (2018), Thomas et al (2018), and Riddell et al (2018)
These people are taking in each other’s laundry. In effect, this journal peer-review of this paper was only an “in-house” review – one you might expect from work colleagues. A good journal peer-review should involve outsiders – and should avoid including coworkers and cooperators in research programmes.
The journal
Environmental Health is an “Open Access” journal. Authors pay to be published (in this case about US$2500) – which is never a sign of good quality. Authors also appear to often choose their reviewers – or at least suggest them and leave the final choice to the editors.
The Chief Editor of this journal is Phillipe Grandjean who is known to have sympathies for the anti-fluoride movement – often being used as an authority for statements about new research the anti-fluoride movement promotes. He is the author of the paper Grandjean and Landrigen (2014) that anti-fluoride people love to use as evidence of fluoride neurotoxicity – but in fact, only references Chinese studies from endemic fluorosis areas.
More importantly, in my experience, he demonstrates bias in the way he manages the journal. He refused to allow the journal to even consider my critique of the Malin & Till (2015) ADHD paper which was published in his journal. I describe this refusal in my article Fluoridation not associated with ADHD – a myth put to rest
My paper was eventually published in another journal (Perrott 2018) – but this has made it very easy for Christine Till’s group to studiously ignore it in their publication discussions – despite being very well aware of it.
The quality of Malin et al (2019)
One might expect that the need to use an open-access journal like Environmental Health and to choose “in-house” peer-reviewers indicates that the quality of the paper might not be the best. I believe that is the case and will make my own critique of the paper in a post here soon.
Conclusions
For all its faults as an “open source” journal which enables publication for a fee and has a suspect peer-review process, Environmental Health does sometimes make the names of peer-reviewers, the content of their reviews and the responses of authors public. This has been very useful in the case of Malin & Till (2015) and Malin et al (2019) as it shows how authors and peer-reviewers are sometimes “in-house” reviewers who take in each other’s laundry.
I have been critical of the quality of the fluoride-IQ and fluoride-ADHD and similar papers coming from Christine Till’s group. The naive use of statistics (for example relying on p-values rather than providing a full statistical analysis), the lack of transparency for some of their methodology, the biased choice of citations meaning they ignore work which does not support their claims, their bias in the discussion of their results, and their promotion of their findings is concerning. Especially considering how they attempt to make their findings relevant to public health policy and their claims and statements are promoted and supported by the anti-fluoride movement.
Others are also concerned. Recently 30 academic and health experts wrote a letter to the funding body which had financed some of the work of Till’s group. The letter outlined ten important concerns and requested public release of the data for one of Till’s papers so that it could be independently analysed (see Experts complain to funding body about quality of fluoride-IQ research).
I suspect that if the peer-reviewers of other papers from Christine Till’s group, and the content of there reviews were made public we could see a similar situation to that found for Malin & Till (2015) and Malin et al (2019).
Perhaps if more journals followed Environmental Health’s policy of transparency about peer-reviewers and their comments authors would be less likely to choose colleagues for peer review. This practice of taking in each other’s laundry is not good for science.
Open Parachute 