How to do a peer review of an article
This blog once again comes as a request from readers, and surprisingly, it is something I see again and again on social media forums.
In the past, when dinosaurs roamed the earth and I was a junior scholar just starting out, there was a thing called ‘mentoring’. Not the kind your institution writes about on a REF submission, but the real kind where senior scholars actively tried to help and ‘train’ more junior scholars into the mysteries of academic life. One of the things they might have instructed you in, is the secret art of how to do a peer review for journal articles.
It seems that time is long past (for many and various reasons I will not rehearse here). So, instead, I will provide some pointers on how (and crucially how not) to do a journal article peer review.
First principles
A decent and well run journal should send reviewers a review template which should guide your review. Not all do this, and perhaps when they do, these guidelines can be a bit wooly. In the absence of such a guide, there are some cardinal rules of good reviewing:
Don’t take too long to return it. You know what I mean - someone is desperately refreshing ScholarOne every 30 seconds to see what is happening to that paper. If you can’t do it briskly, don’t take it on.
How long should it take to do the review itself? Not long. 2 hours to read the paper (10,000 words) and construct feedback. Get on with it, you are not scouring the earth for the last Airbender. Move on with your life.
How long should your review be? Maximum 2 pages and preferably shorter. If you are writing more than this, you have fallen into asshole territory and should stop and check yourself. Nothing screams ‘insecure junior scholar who knows nothing’ more than someone who feels the need to write lengthy reviews to demonstrate the extent of their own brilliance. DON’T DO IT. Also know that senior editors are unimpressed by your posturing and you look like a fool.
Content rules
The question you are answering is: is this paper worth publishing?
Is it well researched? Are the appropriate scholarly articles/areas included? Is the referencing sufficient, careful and complete?
Does the author do what they claim to do - do the composite parts add up (abstract, introduction, middle and conclusions)?
Are any claims made properly substantiated?
Is it rigorous and does it display originality? Does it bring a new angle or way of thinking about something? Or is it addressing a pressing issue, or filling a gap by updating something?
Does it fit the brief of the journal (the editors should have already desk rejected anything that doesn’t, so this isn’t really your area to comment on unless something has gone very wrong).
If the answers to the above questions are YES, then you should recommend that the article is published.
What do you do if there are minor infelicities in the language or errors (I mean errors in the literal sense) or some of the things listed above are there, but could have been articulated in a clearer fashion?
Send it back with minor revisions pointing out exactly what could be better articulated with the above list in mind, and construct that advice with word count restrictions in mind.
Be concise and specific.
What do you do if there are major problems with the piece (i.e. the list above has not been met)?
You can send it back with major revisions, indicating what these revisions are. If this is ultimately a new paper, it is not a major revision.
If you do this, be prepared for a second round of reviewing. Do not indicate major revisions if you cannot be bothered to do this again. It is rude beyond belief and a massive pain for editors.
Finally, and exceptionally, you can reject the piece.
Reasons to reject a piece in good conscience:
It is not well researched. The appropriate scholarly articles/areas are not included (with the word count limit in mind). The referencing is insufficient, sloppy and incomplete.
The author does not do what they claim to do - the composite parts do not add up (abstract, introduction, middle and conclusions).
The claims made are not properly substantiated.
It is not rigorous and it does not display originality. It does not bring a new angle or way of thinking about something.
It is a partial piece of research that has been ‘salami sliced’ too many times from a bigger project so that it no longer really contributes something meaningful in its own right.
How do you construct your feedback?
Just like we are told to do for students, it should be a feedback sandwich. You know just how much effort that poor academic has put into that paper. They have wept, they have bled.
Start by saying how much you enjoyed reading it, and list some positive aspects of the paper and inquiry. It costs you nothing and it must have something positive about it or it would not have been sent out for review.
Then move on to any critique and weigh your words carefully. Imagine you were reading this review on your own paper. Phrase it professionally at all times. Don’t be an asshole (I’m looking at you: ‘this sounds more like a newspaper editorial than a scholarly paper’). Reader: I was published.
Finish the review with something positive about the project in general and why it is interesting.
Things you must never ever do
Reject a paper because it is not the paper you would have written. This is bullshit scholarly backstabbing and should not ever be entertained. Why don’t you go ahead and write that paper yourself, and leave this poor creature to publish the paper they have actually written?
Write endless reams of feedback to demonstrate just how smart you are. Yes, we are all smart. Get a life. Short and structured in deference to the word count is professional.
Request minor revisions but really you are asking for a whole new paper (a different paper, i.e. the one you would have written). This is impossible for the author to do in the word count available to them. If you want something included, why not point out the things that could be cut to make room. If you can’t find anything, you are being as asshole again. Everyone needs to respect the word limit, including reviewers.
Get personal. Don’t be snarky, snidey, supercilious, hectoring or patronising. Be professional. Someone has spent an awful lot of their (probably unpaid) time writing this piece of research. Respect the author.
Reviewer 2 is a thing because we make it one. Don’t be that person.