I was recently asked to review a few submitted articles for scholarly journals and I found myself wondering about what topics to include. A colleague shared the following insight:
Your evaluation, which does not need to be very long, should assess the submission based on 1) The importance of its contribution to the field (originality); 2) the soundness/rigour of its method (scholarship); and 3) the quality of its style (diction, grammar, structure, etc.). The evaluation should include one of the following recommended actions: 1) Accept; 2) Accept with revisions; 3) Revise and resubmit; 4) Reject.
I ran a quick Google search (because I deemed probabilistic authoritativeness as a valuable search strategy) and here are a few links to some guidelines and other sources on this topic:
The National Institutes of Health (US) has extensive resources on these issues, called Guidelines and Fill-able Templates for Reviewers. Of particular interest is the Review Criteria at a Glance – Research (PDF) (“Significance ; Investigator(s) ;Innovation ; Approach ; Environment“)
Elsevier’s Reviewer Guidelines first has you consider if you are fit to act as a reviewer (“Does the article you are being asked to review truly match your expertise? ; Do you have time to review the paper? ; Are there any potential conflicts of interest?“) and then, advises you on conducting the review (Originality, Structure, Previous Research, Ethical Issues)
On the McGraw-Hill Companies’ website, I found a free ebook from Marting Maner entitled The Research Process: A complete guide for writers. Section 4 covers Peer Review Guidelines and specifically, some review questions (“The thesis sentence ; Support for the thesis ; Organization ; Insight ; Overall quality ; Suggestions for revision“)
Nature.com’s Peer Review Policy (Provides strong evidence for its conclusions ; Novel ; Of extreme importance to scientists in the specific field ; Ideally, interesting to researchers in other related disciplines)
On the topic of peer-review, Murray Dineen’s piece in UniversityAffairs.ca is worth a read: Time to rethink peer review: Evaluating scholarly work in the Internet age (Dec 5 2012):
Anonymous peer review is rarely anonymous. By the time one’s research reaches the level of sophistication necessary to attract scholarly interest, one’s identity is known to peers. Nor is peer review always objective. Reviewers often hide behind anonymity to deliver unwarranted attacks. (And authors rarely have recourse to a vehicle by which to respond to the reviewer.) For these reasons, anonymous peer review has been called unjust and inhumane in some quarters.
It doesn’t have to be so. The Internet allows for timely and humane forms of exchange in scholarship. In the hands of an editor, peer review could become a form of colloquy, an exchange between author and reviewers. “Open peer review” and “open peer commentary” should become fully accepted practices of scholarly review. [Read more]
Also worth a read in UA is Rosanna Tamburri’s Opening up peer review in the April 2012 issue.
An interesting read, this post about the Open Repository conference in Edinburg. there is much talk of including raw data in IRs:
Just about everyone was discussing RDM, or Research Data Management. It has become clear that institutional repositories must not only manage scholarly publications, but the data that was created through observation and experimentation or collected and published, in order to support the “re-” activities: review, reuse, replicability and reproducibility. RDM platforms are needed to help researches capture and share and publish their datasets. The public-facing discovery infrastructure is but a small part of this effort: the greater need and effort is in capturing data from the original instruments and formats and the transfer and documentation of datasets in a reliable, documented way to support a forensic level of authenticity for future researchers. The Digital Curation Centre has a great blog post reviewing some of the sessions on this topic.
An interesting attempt to rething journal rankings, with open access publishing in mind:
March 2012 College & Research Libraries vol. 73 no. 2 134-145
Positioning Open Access Journals in a LIS Journal RankingJingfeng Xia