I was recently asked to review a few submitted articles for scholarly journals and I found myself wondering about what topics to include. A colleague shared the following insight:
Your evaluation, which does not need to be very long, should assess the submission based on 1) The importance of its contribution to the field (originality); 2) the soundness/rigour of its method (scholarship); and 3) the quality of its style (diction, grammar, structure, etc.). The evaluation should include one of the following recommended actions: 1) Accept; 2) Accept with revisions; 3) Revise and resubmit; 4) Reject.
I ran a quick Google search (because I deemed probabilistic authoritativeness as a valuable search strategy) and here are a few links to some guidelines and other sources on this topic:
The National Institutes of Health (US) has extensive resources on these issues, called Guidelines and Fill-able Templates for Reviewers. Of particular interest is the Review Criteria at a Glance – Research (PDF) (“Significance ; Investigator(s) ;Innovation ; Approach ; Environment“)
Elsevier’s Reviewer Guidelines first has you consider if you are fit to act as a reviewer (“Does the article you are being asked to review truly match your expertise? ; Do you have time to review the paper? ; Are there any potential conflicts of interest?“) and then, advises you on conducting the review (Originality, Structure, Previous Research, Ethical Issues)
On the McGraw-Hill Companies’ website, I found a free ebook from Marting Maner entitled The Research Process: A complete guide for writers. Section 4 covers Peer Review Guidelines and specifically, some review questions (“The thesis sentence ; Support for the thesis ; Organization ; Insight ; Overall quality ; Suggestions for revision“)
Nature.com’s Peer Review Policy (Provides strong evidence for its conclusions ; Novel ; Of extreme importance to scientists in the specific field ; Ideally, interesting to researchers in other related disciplines)
On the topic of peer-review, Murray Dineen’s piece in UniversityAffairs.ca is worth a read: Time to rethink peer review: Evaluating scholarly work in the Internet age (Dec 5 2012):
Anonymous peer review is rarely anonymous. By the time one’s research reaches the level of sophistication necessary to attract scholarly interest, one’s identity is known to peers. Nor is peer review always objective. Reviewers often hide behind anonymity to deliver unwarranted attacks. (And authors rarely have recourse to a vehicle by which to respond to the reviewer.) For these reasons, anonymous peer review has been called unjust and inhumane in some quarters.
It doesn’t have to be so. The Internet allows for timely and humane forms of exchange in scholarship. In the hands of an editor, peer review could become a form of colloquy, an exchange between author and reviewers. “Open peer review” and “open peer commentary” should become fully accepted practices of scholarly review. [Read more]
Also worth a read in UA is Rosanna Tamburri’s Opening up peer review in the April 2012 issue.