Industry-funded translation can introduce bias in selection of studies for scientific review

Image credit: Assessing and addressing bias in systematic reviews

The Fluoride Action Network (FAN), in the last decade, paid for translation of a lot of Chinese-language scientific papers linking high fluoride dietary intake to IQ deficits in children. They, of course, selected papers to fit their own ideologically-motivated bias. This is perfectly understandable for an activist group. But has this caused a bias in available English-language sources on this topic? And does this mean recent scientific reviews of this subject unintentionally suffer from selection bias?

I hadn’t considered this possibility before, but it is an issue raised in the recent US National Academies of Sciences (NAS) peer review of the US National Toxicity Program’s (NTP) review of possible neurotoxic effects of fluoride (see Another embarrassment for anti-fluoride campaigners as neurotoxic claim found not to be justified).

Use of FAN sources introduces biased study selection

The NAS peer reviewers are harshly critical of the NTP draft review. A central concern was the way the NTP evaluated the literature on the subject. The NAS peer reviewers say on page 3 of their report:

“The committee had substantive concerns regarding NTP’s evaluation of the human evidence as noted below. The strategy used for the literature search indicated that NTP used FAN as a source to identify relevant literature. The process by which FAN identified and selected studies is unclear, and that uncertainty raises the question of whether the process could have led to a biased selection of studies. Such a concern raises the need for a formal evaluation of any potential bias that might have been introduced into the literature-search process.”

OK, I am not impressed that the NTP used FAN as a source. FAN is hardly a reliable source and its “study tracker” certainly does not pick up anywhere near the full literature available (see Cherry-picking and ring-fencing the scientific literature). But, at first thought, I imagined that the FAN source simply produced a subset of anything that is picked up using a more reliable source like PubMed to do literature searches.

Injection of study bias into English-language scientific literature

But the NAS peer reviewers raise an important problem with reliance on FAN as a source and its effect on the available English-language scientific literature. On page 24 of their report they say:

“. . the process by which FAN identified and selected studies is not clear. FAN identified a number of studies published in Chinese language journals—some of which are not in PubMed or other commonly used databases—and translated them into English. That process might have led to a biased selection of studies and raises the question of whether it is possible that there are a number of other articles in the Chinese literature that FAN did not translate and about which NTP is unaware. NTP should evaluate the potential for any bias that it might have introduced into the literature search process. Possible ways of doing so could include conducting its own searches of the Chinese or other non–English-language literature and conducting subgroup analyses of study quality and results based on the resource used to identify the study (for example, PubMed vs non-PubMed articles). As an initial step in such evaluations, NTP should consider providing empirical information on the pathway by which each of the references was identified. That information would also improve understanding of the sources that NTP used for evidence integration and the conclusions drawn in the monograph.”

In a nutshell, FAN arranged and paid for translation of quite a large number of Chinese papers on this issue (fluoride intake and child IQ deficits). Naturally, they have selected papers supporting their political cause (the abolition of community water fluoridation) and ignored papers which they could not use to that end. It is therefore likely they have introduced into English-language scientific literature a biased selection of Chinese papers because FAN effectively “republished” the translated papers in the journal “Fluoride” – a well-known repository of anti-fluoride material.

Maybe I was wrong to assume anything from FAN would simply be a subset of what is available through more respectable searching sources. But, according to the peer reviewers, some of the translated papers may be picked up when FAN is used as a source of studies but not when PubMed or similar respected sources are used. A warning, though – many of the FAN-promoted translated studies have only been partly translated, maybe only the abstract is available. This is not sufficient for a proper scientific review (see Beware of scientific paper abstracts – read the full text to avoid being fooled).

I am not saying this bias introduction into the English-language scientific literature was intentional, but it is a likely end-result of their actions. Importantly, it is also a likely end-result of funding from big money sources (the “natural”/alternative health industry which funds FAN and similar anti-fluoride and anti-vaccination groups – see Big business funding of anti-science propaganda on health).

So, is this a way that big industry can inject their bias into the available scientific literature? A way to ensure that reviewers will, maybe unintentionally, convey this industry bais into their own summary of scientific findings?

Reviewers should make a critical assessment of studies

The FAN-promoted Chinese studies really do not contribute to any rational discussion of issues with CWF because they were all made in areas of endemic fluorosis. Ironically they often compare child IQ in villages where fluoride intake is high, with that in villages where the fluoride intake is low. It is the low -fluoride villages which are relevant to areas of CWF because their drinking water F concentrations are comparable.

In reality, these Chinese studies could be used to support the idea that CWF is harmless. Even if that is an inherent assumption for low fluoride intake in these studies.

So, perhaps the bias introduced to the literature by translation of the FAN-promoted studies really is of no consequence to the evaluation of CWF. However, consideration of reviews like the recent one by Grandjean (2019) indicates there is a tendency to simply extrapolate from high concentration studies to make unwarranted conclusions about CWF. In this case, the tendency is understandable as Grandjean is well known for his opposition to CWF and is often used by FAN to make press statements raising doubts about this health policy (see Special pleading by Philippe Grandjean on fluoride, Some fluoride-IQ researchers seem to be taking in each other’s laundry, and Fluoridation not associated with ADHD – a myth put to rest).

This was also a problem with the draft NTP review which produced the (unwarranted) conclusion “that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans.” The draft did actually mention that the conclusion “is based primarily on higher levels of fluoride exposure (i.e., >1.5 ppm in drinking water” and “effects on cognitive neurodevelopment are inconsistent, and therefore unclear”  for “studies with exposures in ranges typically found in the water distribution systems in the United States (i.e., approximately 0.03 to 1.5 ppm according to NHANES data).” But, of course, it is the unwarranted conclusion that gets promoted.

Conclusions

Reviewers need to be aware of this and other ways activist groups and big business can inject bias into the scientific literature.

This problem underlines the responsibility reviewers have of recognising all possible ways that biased selection of studies they consider can occur. It also means they should make every effort to include negative studies (not supporting the effect they may personally prefer) as well as positive studies. They also need to include all the findings (positive and negative) included in the individual studies they review.

In cases like the FAN-promoted Chinese studies, there is an obligation to at least note the possibility of bias introduced by activists and industry-funded translations. Even better, to ensure that the reviewer undertakes to independently search for all studies on the subject and arrange for translations where necessary.

Above all, reviewers should critically consider the quality of the studies they include in their reviews and not simply rely on their own confirmation bias.

Similar articles

Cultural and ideological bias in scientific literature reviews

 

The scientific community is international but it is surely dominated, and influenced by, something less international – the overwhelming predominance of  the use of the English language.

In the past when that dominance was less extreme scientists were more aware of the need to use non-English sources. Scientific degree courses often had a requirement for a degree of scientific fluency in an alternative language. I learned German and Russian and have often needed to translate scientific papers in both these languages. In fact, several times I can trace a breakthrough in my research to a Russian or German paper I translated.

So, I was often disappointed when searching the literature to come across papers in a less dominant language like Mongolian, Ukrainian, Byelorussian, Arabic, etc. Very often these were published in unattainable obscure journals or periodicals. While I was aware that such papers were perhaps of poor quality I still lamented my inability to check that out for myself. I felt I  might be missing something useful.

I can’t help thinking that scientific reviews, even systematic ones, may suffer from such omissions. After all the problems of expense and the probably poor quality of obscure papers usually mean that even systematic reviewers will not be able to obtain obscure papers and their translations.

But there is the other side. What happens if ideologically motivated groups able to make the financial investment obtain and translate obscure papers which fit their political or commercial biases? Can that not result in the “skewing” of readily available literature? Could it result ion the wider availability and influence of poor quality research?

One would hope scientific reviewers critically and intelligently appraise the papers they review. This would help prevent promotion of poor science. But not all scientific reviewers may be that objective. And certainly, such poor quality science can be promoted in the popular literature and used for political and ideological campaigns.

Bad science promoting the fluoridation/IQ myth

This is what has happened with the promotion of the fluoride/IQ myth by anti-fluoride campaigners. Several years ago the Fluoride Action Network (the main activist group campaigning against community water fluoridation internationally) paid for the searching out, and translation into English, obscure articles (mainly Chinese) suggesting a link between fluoride in drinking water and child IQ. These articles represented in almost all cases poor quality research mostly published in obscure journals, newsletters and conferences. Given the bias of FAN, preference was clearly given to articles supporting their party line.

The result is that these articles have now entered into English language scientific literature – via publication of the translations in the journal Fluoride – a poor quality, English language journal, linked organisationally and financially to FAN. This has introduced an ideologically promoted skewing of the available literature – on top of the well-understood publication bias (negative results are rarely published). Such skewing is important for reviewers such as those in the US  National Toxicology Program considering a new systematic review on (see Fluoride and IQ – another study coming up).

Don’t get me wrong. Despite the biases of this translation process I like to see such obscure studies, even poor quality one, made available to researchers. And sensible, conscientious reviewers will take the poor quality into account. But less objective researchers can opportunistically use such material to support their preferred hypothesis. This has happened with the fluoridation issue, for example, with the paper of Grandjean & Landrigan (2014) which is continually used by anti-fluoride campaigners (see Repeating bad science on fluoride). This is a secondary effect where the poor quality research that is selected, translated and published in a poor quality journal like Fluoride can then make its way into more reputable journals (like The Lancet) without any further evidence. Adding to the skewing of the available scientific literature.

And all of this provides ammunition to the unscrupulous ideologically motivated campaigners. Critical assessment of the research is the last thing on their minds and they are certainly not in  the habit of even reading, let alone critically understanding, the papers they promote in  their arguments.

Similar articles