Sometimes the on-line discussion of scientific issues looks like a citation battle. People take sides, battle lines are drawn and struggle commences. Each side fires barrages of citations “proving” their own argument.
The battle progresses in real-time – the proferred citations are immediately rejected and alternatives offered. One would think the other side would take time out to actually read the offered citations – but no they are usually quickly rejected as unreliable. I also get the impression that in many cases the side offering the citation has also not bothered to read it – usually relying on its use by an ally or its coverage in a friendly on-line magazine.
OK, it natural to be lazy but wouldn’t we all learn a lot more by actually reading the citations being thrown around. And doesn’t it discredit one’s position to reject a citation out of hand for unjustified reasons?
The Logic of Science recently posted an analysis of the bad reasons people use for rejecting citations – 12 bad reasons for rejecting scientific studies. It is well worth a read – we will recognise these 12 reasons and hopefully learn not to use them ourselves in future.
Here are the 12 bad reasons:
Bad reason #1: Galileo/Columbus
“When faced with results that they don’t like, many people will invoke Galileo or Columbus and claim that they defied the mainstream view and people thought that they were crazy, but they turned out to be right. . . [However] no one thought that Galileo was crazy. He presented facts and careful observations, not conspiracies and conjecture. He did not blindly reject the science of his day, rather he made meticulous observations and presented data that discredited the common views. That is not in any way shape or form the same as arrogantly and ignorantly rejecting a paper just because you disagree with it.”
Yes, the Galileo claim always come across to me as very arrogant and crazy – yet it’s a common excuse. An Australian climate change denial group even incorporated Galileo into its title – poor old Galileo must be turning in his grave.
Bad reason #2: science has been wrong in the past
“[P]eople often make the broad claim that science shouldn’t be trusted because it has been wrong before. . . . . First, it is true that science has been wrong, but it has always been other scientists who have figured out that it was wrong. Further, it is logically invalid to blindly assume that it is wrong just because it has been wrong before.
Additionally, although there have been plenty of minor hypotheses which have been discredited, there have been very few core ideas that have been rejected in the past century. In other words, ideas which are supported by thousands of studies have rarely been rejected, and very few central ideas have been overthrown in recent decades.
Finally, attacking science by asserting that it has been wrong before is utterly absurd because science is inherently a process of modifying our understanding of the world. In other words, science is self correcting. This is one of it’s greatest strengths. . . . . It constantly replaces erroneous ideas as new evidence comes to light (the same can’t be said for anti-science views which rigidly cling to their positions no matter how much evidence opposes them). Therefore, the fact that science has been wrong is actually a good thing, because if there were no instances where we had discovered that a previous idea was wrong, that would mean that science hadn’t advanced.”
Scientific knowledge is always incomplete – with time it becomes more and more correct in its description of reality, but there is always room for improvement, for deepening of specific knowledge and refinement of theories.
It seems to me very crass to use this inherent property of good science against science itself.
Bad reason #3: it’s all about the money
Ironically, this excuse is commonly used by people allied with movements funded by big business who are campaigning against scientific findings they feel challenged by.
“This is probably the most common response to papers on climate change, vaccines, GMOs, etc., and it’s often simply untrue. The scientific community is massive, and there are thousands of independent scientists doing research. Further, all scientific publications require authors to declare any conflicts of interest, so you can actually check and see if a paper was paid for by a major company, and if you did that, you would find that many of the papers supporting GMOs, vaccines, etc. have no conflicts of interest. Anti-scientists, of course, have no interest in actually looking at the paper. They would rather just assume that it was paid off because that fits with their world-view.
. . . even if a paper does have a conflict of interest, that doesn’t give you carte blanche to ignore it. The fact that someone works for a pharmaceutical company, for example, does not automatically mean that they biased or falsified their data. If a paper has a conflict of interest, then you should certainly give it extra scrutiny, and you should be suspicious if it disagrees with other papers or has questionable statistics, but you cannot automatically assume that it is flawed.”
Wise words. We should always read scientific papers critically and intelligently – especially when there may be a conflict of interest. But it is neither critical or intelligent to reject them out of hand in this way.
Bad reason #4: there are other results that I disagree with
Someone will say, “I reject the science of X because science also says Y and I disagree with Y.” We can rephrase this as, “I reject science because I reject science.” I would not, for example, accept water fluoridation as evidence that it’s ok to reject the science of vaccines unless I had already rejected the science of fluoridation. In other words, you have to justify your rejection of the science of Y before you can use it as evidence that we shouldn’t trust the science of X. Further, even if you could demonstrate that the science of Y (in this example fluoridation) was wrong, that still would not in any way shape or form prove that the science of X (in this example vaccines) is wrong. In fact, this entire line of reasoning is just a special case of the logical fallacy known as guilt by association. If are going to say that a scientific result is incorrect, you have to provide actual evidence that the specific result that you are talking about is incorrect.”
Yes, this tactic is a red-herring, often used as a diversionary device, and very lazy as it shows an unwillingness to consider properly the issue at hand.
Bad reason #5: gut feelings/parental instincts
” . . . . show someone the scientific evidence for vaccines, and they respond with, “well as a parent only I know what is best for my child.” Similarly, when I show people the evidence for GMOs, they often respond with something like, “well I just have a gut feeling that manipulating genes is bad.” I do not give a flying crap about your instincts or gut feelings. The entire reason that we do science is because instincts and feelings are unreliable. When someone presents you with a carefully conducted, properly controlled study, you absolutely cannot reject it just because you have a gut feeling that it’s wrong. Doing that makes no sense whatsoever. It is the most blatant form of willful ignorance imaginable. Don’t get me wrong, intuition is a good thing, and gut feelings can certainly help you in many situations, but they are not an accurate way to determine scientific facts.”
Our feelings and instincts are very strong and will often divert our attempts at rational considerations. I think such factors are often behind the rejection of scientific studies – even when this reason is not given. But:
“Gut feelings simply aren’t reliable. That’s why we do science.”
Bad reason #6: I’m entitled to my opinion/belief
“This is another very common response, and it is very similar to #5. Science deals with facts, not opinions or beliefs. When multiple scientific studies all agree that X is correct, it is no longer a matter of opinion. If you think that X is incorrect, that’s not your opinion, you’re just wrong. Think about the relationship between smoking and lung cancer again. What if someone said, “well everyone is entitled to their opinion, and my opinion is that it’s safe.” Do you see the problem? Scientists don’t have an opinion or belief that smoking is dangerous; rather, it is a scientific fact that it is dangerous, and if you think that it is safe, you are simply in denial. Similarly, you don’t get to have an “opinion” that the earth is young, or vaccines don’t work, or climate change isn’t true, or GMOs are dangerous, etc. All of those topics have been rigorously tested and the tests have yielded consistent results. It is a fact that we are changing the climate, a fact that vaccines work, a fact that the earth is old, etc. If you reject those, you are expressing willful ignorance, not an opinion or belief.”
Hear, hear!
Bad reason #7: I’ve done my research/an expert agrees with me
” . . . . . if your “research” disagrees with properly conducted, carefully controlled studies, then your research is wrong (or at the very least, must be rejected pending future data). There, it’s that simple. The only exception would be if your research is actually a large set of properly controlled studies which have directly refuted the study in question (e.g., if you have a meta-analysis vs. a single study, then, all else being equal, go with the meta-analysis). It’s also worth pointing out that having a few people with advanced degrees on your side does not justify your position (that’s a logically fallacy known as an appeal to authority). No matter what crackpot position you believe, you can find someone somewhere with an advanced degree who thinks you’re right.”
This appeal to authority is commonly used – nothing seems to offend an anti-fluoride campaigner more than to refer to their ideological leader, Paul Connett, without reference to his degree of former university title! Such people also often show the converse – refusing to use titles when referring to the work of someone they disagree with.
Bad reason #8: scientific dogma
“This response basically states that all scientists are forced to follow the “dogma” of their fields, and anyone who dares to question that dogma is quickly ridiculed and silenced. . . . . In short, that’s simply not how science works. Nothing makes a scientist happier than discovering that something that we thought was true is actually false. In fact, that is how you make a name for yourself in science. No one was ever considered a great scientist for simply agreeing with everything that we already knew. Rather, the great scientists are the ones who have shown that our current understanding is wrong and a different paradigm provides a better understanding of the universe. To be clear, if you are going to defeat a well established idea, you are going to have to have some very strong evidence.”
A related claim is that the “scientific establishment” prevents publication -often used to explain why many of the authorities used by people rejecting scientific studies do not have a credible publication record.
I would be the last person to deny human jealousies and defense of peer-reviewers and scientific editors can be a problem with specific journals – but there is many alternative journals willing to accept papers.
But this does raise another issue to be wary of – there are some journals which have incredibly poor peer review and often accept papers because of the authors’ willingness to pay a publication fee. Publication in such journals should definitely be seen as a warning sign – but as in all other cases judgment should be based on a critical and sensible analysis of the paper itself.
Bad reason #9: distrust of governments/media
” Many people, however, take it even a step further. On numerous occasions, I have shown someone a study which was not in anyway affiliated with a government agency, yet they still responded with a lengthy rant about corrupt governments or the media. The basic idea of their argument seems to boil down to, “the government/media agree with these results, therefore they must be false.” This line of reasoning is, however, clearly fallacious (in fact it’s a logical fallacy known as guilt by association). Governments and the media will lie to push their own agendas, I’m certainly not denying that, but that fact does not automatically mean that everything that they say is a lie. . . . . . . It’s fine to be skeptical of what you are told by the government/media. In fact it is a good thing, but when you are presented with scientific evidence, then it’s not a matter of trusting the government/media. Rather, it is a matter of whether or not you accept science. In other words, I don’t need to trust the government or media in order to accept the results of a carefully controlled study.”
A related reason is to imply that any scientific study is not independent because the researcher are paid. Rather silly, considering we all have to live and researchers are no different. These people will instead cite articles written by activists or journalists working for magazines financed by an industry like the “natural”/alternative health industry. Or claim the financing of activist organisation is by “donation” so it doesn’t count
Bad reason #10: it’s a conspiracy
“This one is very closely related to #8 and 9, but it takes things a step further. It proposes that there is a massive conspiracy and scientists are being paid by governments/big companies to falsify results. . . . . the scope of this conspiracy would be impossibly huge. The scientific community consists of millions of people from all over the world working out of thousands of universities, institutes, non-profits, corporations, agencies, etc. It includes people from countless religions, cultures, political ideologies, etc. There is no way that you could possibly get that many people to agree on a massive deception like this. Just think about what is being proposed here. Do you honestly think that nearly all of the world’s climate scientists have been bought off? . . . . . . Do you honestly think that all of those different organizations (many of whom compete with each other and have different goals and purposes) have all managed to come together to make one unified conspiracy? That’s just nuts. The same problems exist for governments. . . . . Honestly ask yourself the following question: which is more plausible, that countless governments, companies, non-profits, etc. have all come together to create the world’s largest conspiracy and buy off virtually every scientist on the planet, or that the thousands of independent scientists who have devoted their lives to science are actually doing real research?”
Personally I think this reason should be considered as an immediate acceptance that the commenter has lost or that they have disqualified themselves – like Godwin’s law for the first person to bring up Hitler or the Nazis.
Bad reason #11: anecdotes
“Anecdotes do not matter in science, because anecdotes don’t allow us to establish causation. Let me give an example. Suppose that someone takes treatment X and has a heart attack 5 minutes later. Can we conclude from that anecdote that treatment X causes heart attacks? NO! It is entirely possible that the heart attack was totally unrelated to the treatment and they just happened to coincide with one another. Indeed, I once heard a doctor describe a time where he was preparing to vaccinate a child, and while preparing the vaccine, the child began having a seizure (to be clear, he hadn’t vaccinate the child yet). He realized that if he had given the vaccine just 60 seconds earlier, it would have looked for all the world like the vaccine had caused the seizure when in fact the kid just happened to have a seizure at the same time that a vaccine was being administered.
. . . . it should be clear that anecdotes are worthless because they cannot establish causal relationships (in technical terms, using them to establish causation is a logical fallacy known as post hoc ergo propter hoc fallacies [i.e., A happened before B, therefore A caused B]). Properly controlled studies, however, do allow us to establish causation.”
Yet commenters again and again fall back on anecdotes – even after launching a citation attack the anecdotal evidence seems to have much more relevance than anything reported in scientific studies.
Bad reason #12: a scientific study found that most scientific studies are wrong
“This argument is fascinatingly ironic because it uses a scientific paper to say that we shouldn’t trust scientific papers, but let’s look closer because this argument actually has some merit. The paper being references is, “Why most published research findings are false” by John Ioannidis, and it is actually a very useful and informative work, but it often gets misused.”
The Ioannidis paper describes several reasons why individual papers may be wrong. Issues like small sample size and publication bias in its many forms.
Researcher who often search the literature are aware of these problems and they are aware of the advice to approach all papers critically and intelligently – even the ones which present results you find favourable.
But as the article points out:
“. . .all of that may sound very bleak, but it should not make you lose all confidence in the scientific process because of a very important component of scientific inquiry: replication. Ioannidis’s work applies mostly to single paper studies. . . . . So, this paper shouldn’t make you question the safety of vaccines, the effects we are having on the climate, etc. It should, however, make you skeptical of the one or two anti-vaccine papers that you occasionally see, or the one paper supporting some “miracle cure,” or the occasional paper on homeopathy, acupuncture, etc. Those studies almost always have tiny sample sizes and countless other studies have failed to replicate their results. This is why it is so important to look at the entire body of literature not just a single study.”
Conclusion
The Logic of Science article concludes:
” . . . no matter how you cut it, many of you wouldn’t be alive today if it wasn’t for science. Science clearly works and you need an extremely strong justification for rejecting scientific results.
To be fair, some scientists are corrupt and bad science does occasionally get published, but bad research tends to be identified and discredited by other researchers. In other words, there may be a high probability of a single paper being wrong, but when lots of different studies have all arrived at the same conclusion, you can be very confident in that conclusion. Perhaps most importantly, you cannot simply assume that a paper is bad just because you disagree with its results. You need to present actual evidence that it is flawed or biased before you can reject it.”
Good advice. When you enter a discussion you should actually read the citations you use – and insist you discussion partner readers theirs. And read them critically and intelligently.
. This provides the history of the Galileo Affair as it has been debated over the last almost 400 years. And these presentations have certainly been controversial. Partly because of limited access to documents in the early days. But also because of ideological positions (for and against the church).
“ becomes available over the next few months (see
Some religious apologists just can’t leave Galileo alone. They are unhappy about the fact that most people accept that the Church behaved badly in sentencing Galileo for heresy. (He got house arrest for the rest of his life and bans on his books, one of which was suppressed for 200 years). So in a manner which reminds me of modern day Stalinists trying to make excuses for the Stalin Terror, or to claim it wasn’t as bad as people believe, the apologists have been busy rewriting the history of the Galileo affair.
![Reblog this post [with Zemanta]](https://i0.wp.com/img.zemanta.com/reblog_e.png)
Open Parachute 