Tag Archives: Rationally Speaking

Sceptical humility and peer review in science

This follows on from my recent post about Rebecca Watson’s condemnation of evolutionary psychology (see Sceptical arrogance and evolutionary psychology). Rebecca has now delivered this lecture several times New Zealand. None of them local so I couldn’t personally check if she had taken criticism on board. However, she was interviewed this morning on Kim Hill’s Saturday Morning (here’s the mp3 link).

She has not withdrawn her overall criticism of evolutionary psychology. She makes clear in her interview that this, as well as pop psychology in the media, are her targets (I’ll come back to that below). In fact, I think she makes her position even more untenable by providing a very naive description of peer review in science.

I wish sceptics when they defend science would describe it more realistically. It doesn’t help when they describe a utopian version which doesn’t exist in reality.

What is peer review?

This is an important social process in science where scientists’ ideas, conference presentations, and publications are reviewed by others in their field. Their peers. This helps reduce, maybe even eliminate, the influence of biases and pet, but unsubstantiated, theories held by the author. I have pointed out before that we are all prone to such cognitive biases – it’s part of being human. And having a Ph. D. doesn’t eliminate this human foible.

Scientists are human. And individually it’s hard to escape from our biases. The social nature of science helps to reduce their effect. Hence the importance of peer review.

But I must stress, such review doesn’t just operate at the time of a paper’s publication as Rebecca asserted in the interview. And it’s not just carried out under the watch of the scientific journal. Peer review occurs at all stages.

Peer review is occurring when hypotheses and ideas are being floated with colleagues. In more formal settings like departmental seminars, ideas are presented and exposed to criticism and improvement. Effectively conferences do the same thing to presentations. But often preparation of a conference presentation will have been reviewed by institutional colleagues – formally and informally. (Many groups will even have practice “dry runs” where the scientific content may be considered by colleagues as well as the details of the presentation, speech and visual aids).

Now, getting around to publications (the only area Rebecca included in her naive description). Good scientific institutes will have procedures which ensure formal internal review well before the paper is sent to the journal. And good journals will also have formal procedures to ensure quality and scientific standards in what they accept. Philosopher Masimmo Pigliucci provided a details of the procedure he uses as a journal editor in one of his Rationally Speaking podcasts (see 57: Peer Review). If you aren’t familiar with the process it’s worth listening to. Often such review will involve three anonymous referees, with the requirement that authors respond to questions and recommendations and final decisions on acceptance are made by an editor.

But wait, that’s not all.

The scientific peer review has barely started. Once published the research and conclusions are exposed to a far greater audience of peers. There’s plenty of opportunity for acceptance or rejection by peers. Often journals well accept “Letters to the Editor” types of response. Other scientists will condemn or support those conclusions when they write discussions in their own publications. There is scope for independent people to repeat the work, or usually something similar rather than exactly the same, and publish different conclusions.

Science is dynamic – our knowledge improves all the time. Publications are not sacred – they are easily and often superseded. (And there is scope for withdrawal of published papers when mistakes or scientific fraud are found).

It’s a mistake to think a published paper is the final authoritative stamp of approval on scientific ideas. It isn’t. And that’s the mistake Rebecca makes in her naive reference to the concept of peer review in science.

Rebecca presented an idealistic version of peer review where all mistakes, particularly scientific one, are detected during prepublication review of a paper. She says such mistakes should never make it into the published version. Yet, she says, this is happening in evolutionary psychology and she gives specific example where she critiques research findings and not just media coverage.

Well, guess what Rebecca. Such mistakes are probably made to some degree in all scientific fields. We are human after all. Mistakes do get into published papers (one of mine has my own name spelt wrong – five times). And all publishing scientists are well aware that some journals have much lower standards than others. We have probably all had a paper accepted without any feedback or criticisms from reviewers. Maybe even just on the decision of the editor. I certainly downgrade my impression of the journal when it happens to me.

Those shonky studies

Personally, it think peer review during publication may be a particular problem in the “soft sciences.” At least, I have been surprised to see some ideas presented in this area without supporting evidence, or obviously selected references. Perhaps these weaker standards are inevitable in some areas. Perhaps this allows more scope for intrusion of “political correctness” and popular ideological positions. Or perhaps authors feel less need to justify ideas if they are consistent with the prevailing ideologies in their field or institutes. Maybe the ideological issues in these areas are just too harsh to handle objectively. I imagine this might be true for feminism in the US and race relations in New Zealand.

I am sure Rebecca can find evolutionary psychology research journals where the quality of review is poor or ideologically compromised. But I am sure she could also find, if she looked, journals and publications where the standard of peer review is much higher. Perhaps her interest in feminist ideology and preoccupation with sex-related research has soured her overall view. I wouldn’t like to make that judgement. But soured it is.

Evolutionary psychology is being targeted

Some US bloggers have defended Rebecca on this issue by claiming her criticisms were only of pop psychology and media presentation. They refuse to acknowledge her inclusion of the whole field of evolutionary psychology in her attacks. Or else they excuse it. Maybe that is just the humane propensity to defensiveness coming out. Those sceptics may just be guilty of motivated reasoning (I referred to this in Sceptical arrogance and evolutionary psychology). For their sake I include this slide from Rebecca’s talk where she specifically describes her version of evolutionary psychology and critiques it.

Evol-Psych

I understand evolutionary psychology in broad terms as the application of an evolutionary perspective to human and animal psychology. This doesn’t need that researchers assume that human evolution stopped in the Pleistocene – or any of the other bullet points she has.

Rebecca has set up a straw man version of evolutionary psychology. Maybe that’s because of limitations in her reading or understanding. Maybe just because of her preoccupation with feminism and gender issues. But a straw man nevertheless.

Peer review for Rebecca

Rebecca Watson would have benefited immensely from some peer review herself before finalising her presentation. And all is not lost. Her presentation is getting peer review now. Yes, some of it will be rubbish which she should ignore. But there are some excellent comments being made she would be wise to take on board.

See also:
Science denialism at a skeptic conference
Science Denialism? The Role of Criticism
Oh gob, evo psych again?
Evolving skeptic psychology
Responsible Reading
Responsible Writing
FTB Blogger Stephanie Zvan Makes A Small Mistake
Let’s Confirm Negative Stereotypes About Women
αEP: Shut up and sing!
Do You Need To Be An Expert To Criticize Science?

Similar articles

Free will – problems of definition

Some of the philosophically inclined readers have probably followed the recent internet discussion of “free will.” I am referring specifically to that between evolutionary biologist Jerry Coyne (blog: Why evolution is True) and philosopher  (blog: Rationally Speaking). It been interesting partly because the debate has also encompassed commenters on each blog. Regulars who might otherwise have lined up with the specific blogger but disagreed in this case.

I have no wish to get into debates on “free will” – I find them frustrating because people often argue past each other. And it seems to me that the debate often really boils down to how we define “free will.”  So I just want to restrict my comments here to the matter of definition.

I think much of what Jerry writes is good – but in this case I find his definition of “free will” too mechanical. And I think this leads him to doubtful conclusions. Here is how he defines “free will” in his USA Today article (see Why you don’t really have free will):

“. . .let me define what I mean by “free will.” I mean it simply as the way most people think of it: When faced with two or more alternatives, it’s your ability to freely and consciously choose one, either on the spot or after some deliberation. A practical test of free will would be this: If you were put in the same position twice — if the tape of your life could be rewound to the exact moment when you made a decision, with every circumstance leading up to that moment the same and all the molecules in the universe aligned in the same way — you could have chosen differently.”

It is the “consciously” which I wish to take issue with. It’s important because part of his argument refers to work indicating that decisions on an action may be taken by a person well before that person is conscious of the decision. As Jerry describes it:

“Recent experiments involving brain scans show that when a subject “decides” to push a button on the left or right side of a computer, the choice can be predicted by brain activity at least seven seconds before the subject is consciously aware of having made it. . . . “Decisions” made like that aren’t conscious ones. And if our choices are unconscious, with some determined well before the moment we think we’ve made them, then we don’t have free will in any meaningful sense.”

Consciousness is much over-rated

I think this work is interesting because it suggests an important role for the subconscious (or unconscious part of the brain) in decision-making which, on the surface, appears conscious. In a way this isn’t surprising because most of the work the brain does is, has to be, unconscious. Just imagine if all the ongoing work involved in homeostasis were controlled by conscious decisions. That you had to consciously decide how to respond to every incoming biological threat and then pass those conscious decision on to the immune system!

So, I don’t think it is wise to differentiate so sharply between the conscious and subconscious working of the mind/brain. I have written before about the role of the subconscious in moral decisions and think this stretches to many more areas of our decision-making than we might think. And that is a two-way street – our conscious mental deliberations also influence our subconsciousness brain – and that in return feeds back into later conscious decisions. Our academic and social learning involves, over time, constant interaction between the conscious and unconscious brain.

My suggestion is that when we “freely and consciously choose” this decision is not restricted to the conscious, self-aware brain. It also, and inevitably, involves the unconscious. Using the particular definition Jerry has, and limiting the process of decision-making to the conscious brain, is just too mechanical.

Similar articles