“Peer-review” status is often used to endorse scientific papers cherry-picked because they support a bias.
Many scientists are not impressed with the peer-review processes scientific journals use. Like democracy, this peer-review is better than all the available alternatives but it certainly doesn’t guarantee published scientific papers are problem-free.
Sure, peer-reviewed sources are better than others which have no quality control. But it is still a matter of “customer beware.” The intelligent users of scientific literature must do their own filtering – make their own critical judgements of the likely reliability of reported scientific findings.
Despite this people often use the “peer-reviewed” description to endorse published finding (especially if they confirm their own biases) without any critical assessment. This happens a lot in on-line debates of “controversial” issues.
Here I will go through the details of peer-review of a recently published paper which anti-fluoride activists are endorsing and promoting, but others are critcising. The paper is:
Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14.
I have discussed this paper in recent posts (see More poor-quality research promoted by anti-fluoride activists and ADHD linked to elevation not fluoridation). The journal, Environmental Health, has a transparent peer-review process which provides access to the names and reports of the reviewers. This reveals problems with the review process in this case. Below I discuss the responsibility of the authors, reviewers and the journal for the problems with this paper and its reported findings.
Authors’ responsibilities
The authors are clearly committed to a pet theory that fluoride is a neurotoxicant which could contribute to ADHD prevalence. Nothing wrong with that – we all feel committed to our hypotheses. We wouldn’t be human if we didn’t. But the best way to produce evidence for a hypothesis is to test it in a way that could prove it wrong.
In this case the authors found a correlation between ADHD prevalence in US states and the amount of community water fluoridation in each state. Trouble is, one can find just as good a correlation, or even a better correlation, with many other criteria for which state prevalence statistics are available. I listed a few in ADHD linked to elevation not fluoridation. Some of these factors are also correlated with community water fluoridation suggesting the correlation reported by Malin and Till (2015) may be deceptive.
A proper test of the fluoridation hypothesis would include considering the effect of including such confounders together with fluoridation in their statistical analysis. Malin and Till (2015) did include one other criteria – the median household income for 1992 – but did not include any others. I find this surprising because they acknowledged ADHD results from interaction of genetic and environmental factors. While fluoridation is not usually considered a relevant factor things like smoking and premature births are and there is conflicting evidence about the role of economic factors like poverty and income.
In my article ADHD linked to elevation not fluoridation I showed ADHD prevalence is better explained by a few of these factors without any input from water fluoridation.
I can’t help feeling the limited consideration of confounding factors results from a desire to protect the fluoridation hypothesis and therefore not test it properly.
Reviewers responsibilities
Again, such a desire is only human. But reviewers should have picked this up during their own considerations.
Interestingly, only one of the two reviewers raised possibility other confounders – specifically lead levels. This is of course valid as lead is a recognised neurotoxicant – but why did none of the reviewers question why other factors like smoking, premature births and social or regional factors were not considered?
I believe that is because both reviewers had research interests directed at chemical toxicity and not ADHD or similar mental characteristics. A matter of someone with a hammer only seeing nails.
The reviewers and their research interests are:
Marc Weisskopf whose reviews are available here and here.
“Some examples of my current work are exploring how exposure to, e.g., lead, manganese, and air pollution affect cognitive function and psychiatric symptoms; how exposure to Agent Orange and other herbicides used in Vietnam relate to the development of PD; and how formaldehyde and lead exposure relate to the development of ALS.”
Anna Choi whose review is here.
“Dr Choi’s research focuses on the effects of environmental exposures on health outcomes. She has been studying the birth cohorts in the Faroe Islands where exposures to environmental chemicals including mercury, PCBs, and PFCs are increased due to traditional marine diets. In addition, she also studies the effects of the contaminants on cardiovascular function and type 2 diabetes among the Faroese septuagenarians. She is also actively involved in the research on the impact of nutrients as possible negative confounders that may have caused an underestimation of methylmercury toxicity. Dr Choi’s other research interests include studying the adverse effects of fluoride exposure in children.”
Why were reviewers with a wider research experience not chosen? This journal allows authors to propose suitable reviewers themselves. Or the reviewers may have been chose by the associate editor handling this paper – Prof David Bellinger. His research focuses on the neurotoxicity of metabolic and chemical insults in children. So again it may just be the blinkered view of someone whose research background stressed the role of neurotoxicants rather than other factors likely to influence ADHD prevalence.
The journal’s responsibility
I noticed that one of the two chief editors (who have final say over acceptance of submitted papers) of this journal is Prof Philippe Grandjean. He himself has been actively promoting the idea that fluoride is a neurotoxicant purely on the evidence of the metareview of Choi et al (2012). Yes he is a coauthor of that review and Choi is one of the reviewers of the Malin and Till paper. The review of Choi et al (2012) related to areas of mainly China where fluoride concentrations are higher than used in community water fluoridation. Areas where endemic fluorosis is common.
I have to wonder if Grandjean’s well-known position on fluoride and community water fluoridation was a consideration in choosing this journal for publication.
Others have commented that the journal Environmental Health is considered low-quality based on its low impact factor. I do not know the area well enough to pass judgement myself. However, I notice that the journal charges authors for publishing their paper (£1290/$2020/€1645 for each article accepted for publication.) This sort of charge, associated with poor quality peer-review makes me suspicious. I have commented on these sort of journal before in my post Peer review, shonky journals and misrepresenting fluoride science.
Conclusions
This is one example of peer-review and paper acceptance which brings into question the idea of using publication and peer-review as endorsement of a study’s quality. I am sure this is not an isolated case. Even with the best of intentions journal editors and reviewers are limited by their own areas of expertise. Journal publication and peer-review is a far from perfect process – even if it is preferable to current alternatives.
Unfortunately activists will promote poor quality studies like this by blindly using the study’s peer-review status.
The intelligent reader should beware of such blind endorsements. Knowing the human foibles which exist in the research and publication processes such a reader will consider the contents of the paper and not rely on peer-review status. They will consider the evidence and conclusions critically. And if they don’t have enough background to make their own critical assessment they will consider the views of others with the required expertise and not blindly accepting what political activists tell them.
Footnote
Just came across this article referring to peer-review problems in journals published by BioMed Central – Major publisher retracts 43 scientific papers amid wider fake peer-review scandal.
BioMed Central publishes the Journal Environmental Health discussed in this post. I am not suggesting this paper was part of the peer-review racket discussed in the article. But the news item does highlight the point I am making that intelligent readers need to consider published scientific papers carefully and critically and not blindly rely on “peer-review” endorsement.
Open Parachute 
The move to charge submitters is an alternative to charging viewers. Now so many institutions are going on line for journals, and restricting access to students and staff, former students and staff have to pay a tremendous amount of money to view a range or references even for one article.
The article was asking for more investigation. It came to you and you have studied and taken the investigation further. But if as you suggest it should not have been published our further investigation could not have happened.
Brian Sandle
LikeLike
Soundhill, it is very, very, common for reviewers to advise that a paper not be published. Often, but not always, they will add “not in its present form.”
I believe that this should have been the advice for this particular paper and the fact it wasn’t is another evidence of poor peer-review.
If it had been knocked back at the first stage and the authors had given proper consideration to confounders it could have been eventually published with a radically revised conclusion – that community water fluoridation does not have any statistically significant inlfuence on the prevalence of ADHD. That would have been a good contribution to the literature -especially as it would warn readers of the errors of incomplete statistical analyses.
Scientific journals should not be publishing poor quality papers with the excuse that it would encourage “further investigation.” They should be encouraging prospective authors to do their work properly before publishing.
At the moment this poor quality paper is being promoted by political activists with an agenda. The “further investigation” may never be done in the sense of seeing the light of day. I have already enquired about publishing a commentary on the paper in this journal (my on-line comment seems to have been not allowed). But the huge cost to me of their poublication charges may make this impossible.
The reasonable alternative for me may be to publish in a more reputable journal without such publication charges. This would take more work as it could not be related to a simple commentary or letter to the editor.
In the meantime I have offered the authors a right of reply here to encourage a more balanced discussion – I have yet to get a response. Perhaps you could put a word in with them for accepting my offer. 🙂
LikeLike
Ken if I contact them again I would like it to help go through the notions I put last articles on Openparachute. I wonder if they have read those. They will have the data already in the multiple regression programs, as you seem to have some of it.
Also firstly I would like to clarify with you about a simple correlation table, which lists correlations without taking into account other interactions, as opposed to a multiple regression table. Multiple regression works like partial correlation which I spoke of before. So those figures would be good to see.
But further it is needed to make sure the recipe used is not dependent on straight line relationships and normal distributions. I suppose the program may warn. If not then it is possible to put the values in order from greatest to smallest, that is rank them, and work with the ranking positions, 1st 2nd 3rd &c. and correlate which is then called a rank correlation. Then when partial correlation is further done it is called partial rank correlation, and can be given a significance figure.
Obviously I am not just writing for you, Ken.
Ken the financial barrier for you to get your message out is not the only barrier to a message getting out. As you said peer review also can and so can editorial policy, and the result is frequently dissemination bias. I am not sure if that can be called unethical therefore come under the Committee on Publication Ethics if the journal subscribes to that.
Brian Sandle
LikeLike
So there is a publication ethics committee – are they subjected to lobbying by parties with an axe to grind such as happens in the following example:
Federal records show that in 2009 the chemical industry spent more than $100 million lobbying Congress and the federal agencies.
In the first six months of 2010 the chemical industry spent more than $40 million on lobbying. Their combined campaign contributions so far in the current election cycle come to more than $10 million.
Individual chemical industry lobbyists were also generous. James Massie, who worked on reform for CropLife America, the main trade group for pesticide companies, donated $104,150, including $58,900 to the Democratic Congressional Campaign Committee. The American Chemistry Council donated more than $305,000 to federal candidates, plus another $50,000 to the Democratic Governors Association.
Koch Industries, whose subsidiary Georgia Pacific is struggling with proposals to regulate formaldehyde in its wood products in the face of increasing evidence of cancer and other health hazards, has donated $1.5 million to federal candidates so far this cycle, with 86 per cent going to Republican candidates.
Likewise, Exxon-Mobil has donated 83 per cent of its $762,099 to Republican federal candidates. Procter & Gamble, which is concerned about the Food and Drug Administration’s review of several of its products containing the anti-bacterial agent Triclosan, as well as reform of the toxic substances act and other issues, has covered both parties more evenly, giving 56 per cent of its $388,878 to the GOP.
BASF, which makes Triclosan, has donated more than $250,000 to federal candidates this year.
The generosity extends to congressional caucuses and their foundations. Last year, for example, Wal-Mart donated at least $500,000 to the Congressional Black Caucus Foundation. Wal-Mart lobbies on many issues, among them the revamping of chemical regulations. This fall’s Congressional Black Caucus Foundation annual legislative conference, the groups’ premier event, drew more than $250,000 from ExxonMobil, and at least $50,000 from Kraft Foods, a division of Altria (formerly known as Philip Morris Companies Inc.), which lobbied this year against a Senate measure to ban BPA in food and beverage containers. Wal-Mart gave $105,000 more so far this year to the Congressional Black Caucus Foundation and its legislative conference. Both Wal-Mart and ExxonMobil are also listed as annual $200,000 donors to the Congressional Hispanic Caucus Institute.
This is just a small segment of the totality spent in steering the pollies in the right direction, not it should be noted in the safest or most people friendly direction. 5 years ago a group of congressional Democrats vowed to overhaul the 34-year-old Toxic Substances Control Act (TSCA) to make it easier for EPA to take dangerous chemicals off the market and ensure that the substitutes are safe. But 5 years, six congressional hearings and 10 “stakeholder sessions” later, the bills are dead, a testament to the combined clout of $674 billion chemical industry, the companies that process their compounds into air fresheners, detergents, perfumes, cosmetics, toys, medical devices and other consumer goods, and the stores that sell them. Their campaign to block reform of the Toxic Substances Control Act won out over EPA’s support and an unprecedented campaign by public health advocates, fuelled by the industry’s own admissions that the current law does not fully protect public health.
Obviously financial ‘clout’ is the key to success for those who prosper from developing and marketing a wide range of products that are deemed safe, effective and beneficial but which may be anything but. Unfortunately there are still too many people who are prepared for a variety of reasons to sell their ethical/professional standards to the highest bidder and engage in vitriolic attacks on those who stand up and say enough is enough.
LikeLike
Trev, not sure how your little tirade against the well known lobbying of government and democratic institutions by big business relates to my article.
Are you suggesting, for example, that these researchers, the journal or the reviewers have been lobbied by the “natural”/alternative health industry? Could be they got encouragement from Mercola, via his financial support of Paul Connett’s FAN?
What is your evidence for this?
Or is it simply confirmation bias and emotional commitment to personal hypotheses?
>
LikeLike
Another unattributed cut and paste from Trevor Crosby.
The idiot that never learns.
LikeLike
Ken – you raised the issue of cost to access peer reviewed journals. All I was illustrating was that vested interests with wealth to burn are at an advantage. If you have a beef with having to fork out to access material you don’t need to look far to find the reason.
The same principle applied to the HCC referendum and subsequent legal action. Those with dough ended up with the bread whilst those without ended up with crumbs.
Richard – May I politely suggest you stick with the rules of engagement and stop name calling and spell my name correctly. Any fool can do what you do but you certainly do it well.
LikeLike
Trev – I do nto have a beef with the cost of accessing material at all. Read what I have written.
My comments related to 2 issues:
1: “Open access” journals which charge for publication are becoming notorius for poor-peer review – if any peer review at all. Already the publishers of this journal have had to withdraw large numbers of papers because of “peer-review” scandals. I am always suspicious of journals charging authors becuase it is encoiuraging people who have poor quality papers but know they can publish them without proper review. It is bad for science. Stephen Pekham’s second to last paper was a classic example of this.
That is my major bitch.
2: If I submit a paper to this journal commenting on the problems with Milan and Till’s paper I will have to confront the question of publication charges. However, the editorial office has told me I can ask for the costs to be waived – and I might have a chance – being retired and not having an institution to pay these bills. Alternatively I could submit to a higher quality journal – but that would require more work than a simple commentary on the offending paper.
In the meantime I have offered the authors a right of reply here which has the advantage of a more immediate exchange, which I think works quite well (evidenced by my exhange with Paul Connett). So far I have not had a response.
You comments on the Hamilton referendum are sour grapes. You guys lost becuase you did not have a case and people prefer to accept the advice of experts rather than activists who tomorrow can be seen protesting about chemtrails.
But you guys do not baulk at taking money from the “natural”/alternative health industry when it comes to legal action, do you? Big money went into the South Taranaki High Court actions and there is a paper trail to prove it.
LikeLike
Trevor, by now you have been pulled up on well over half a dozen occasions in regard to your habit of not citing material you use and for copy and pasting screeds without credit or source.
Initially, you were politely asked not to do so. Everybody gets a fair chance.
Your continued failure to modify your behaviour in this matter can best be explained by either stupidity or unbelievable arrogance and discourtesy. I suggest more of the former than the later. The description “idiot” stands.
As you have also used several sock puppet names in these comment sections, (including the highly apt moniker Trevor Nutter), you haven’t earned much of a right to complain if your name is misspelled.
LikeLike
Dear Richard – If you are incapable of identifying the source of material posted then I question your ability to distinguish between wood and trees. It is individuals like yourself who smart arse their contributions that give rise to people questioning the validity of the ‘science’ they present. Why do you defend historic decisions made on fluoridation that were based on science which recent declassified US government papers show was lacking in validity, integrity and ethics?
I’m not such an idiot that I cannot see through the stupid games you choose to play.
Ken – If you can put up a paper trail that links the funding of the campaign against the fluoride issue in Hamilton with the Natural Health industry that will settle that issue. Knowing that you cannot do that I expect an apology for your attack on my personal integrity and ethics.
LikeLike
Trev – I said “Big money went into the South Taranaki High Court actions and there is a paper trail to prove it.” And I think that “settles” the issue.
I suspect you were not successful in attempts to get the same funding for your little attempt in Hamilton. And that is why you withdrew. Why else would you withdraw, except for lack of funding? 🙂
Now what was that about your personal integrity and ethics??
LikeLike
Trevor, plagiarism is bad.
You are perfectly entitled to copy and paste to your little heart’s content, but it is expected that you provide a source when you are presenting somebody else’s work. Otherwise, people (perhaps people who lack personal experience with your behaviour) may well assume that this is your own work.
It’s also very handy for anybody wanting to check if you’ve chopped out the bits of the article which suit you and ignored other parts of the article which might contradict your viewpoint.
Quotation marks are also essential for telling where the other person’s work leaves off and your own commentary begins.
For others’ benefit, a source for your plagiarism:
http://www.politicsdaily.com/2010/10/13/reform-of-toxic-chemicals-law-collapses-as-industry-flexes-its-m/
LikeLike
Yes Trevor, blubber away, point the finger elsewhere.
In truth, you are being called out for your infantile, dishonest on-line behaviour.
I have no qualms about identifying such behaviour nor the probable reasons for it.
LikeLike