[AISWorld] Practical suggestions for improving journal cycle times

[Arto Lanamäki]

Hi,

I think that cycle time reduction is one of many aspects when considering peer review process improvement. But it is just one aspect, and probably not even the most important aspect. I would emphasize the developmental aspect of peer reviewing, in line of a recent AMR editorial: http://amr.aom.org/content/40/1/1.extract

With kind regards,
Arto Lanamäki
University of Oulu

-----Original Message-----
From: AISWorld [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of Paul Ralph
Sent: 2. huhtikuuta 2015 2:41
To: aisworld at lists.aisnet.org
Subject: [AISWorld] Practical suggestions for improving journal cycle times

Dear all,

Here are some ways to reduce review cycle times:

1) Give reviewers only two options: reject or accept with minor revisions.
2) Limit revisions to one cycle, i.e., manuscript, revision one, galle proofs, published. No revisions two and three.
3) Direct reviewers specifically to evaluate methodology and rigour, rather than respond to tone (see http://www.paulgraham.com/disagree.html).
4) For a 'minor revision' decision, insist on list of specific action items rather than a vague discussion.
5) Limit review periods to one month.
6) Officially suspend dysfunctional reviewers from authorship, i.e., if someone, fails to complete a review or does a terrible job, they lose the right to submit papers to that outlet for one year. Of course, this has to come with a limit, e.g., the right to refuse more than two reviews per year. It also has to be transparent; silently blacklisting people contributes to nepotism (see recommendation 12).
7) Flatten the editorial hierarchy - one paper doesn't need both an SE and an AE. One editor per paper is enough.
8) Limit editors to one month of decision time. Dismiss editors who can't make these deadlines and suspend their authorship privileges (see recommendation 6).
9) Abandon blind review. Blind review is supposed to free junior reviewers to reject the papers of their more powerful peers without repercussion.
This obviously isn't working. It's protecting bad reviewers from well-deserved backlash. Knowing your name is on a review encourages you to stick to actionable suggestions rather than name calling and quibbling about tone.
10) Stop peer-reviewing position papers. Peer review is a system for checking the methodological rigour of empirical research, not for analyzing essays. Treating a position paper as a "peer reviewed contribution" is absurd. Journals are for empirical science. If you want to share an opinion, start a blog.
11) Develop a clear set of desk-reject rules that allows more desk rejects.
Publish them, let them be challenged and continually evolve them. If these policies are regularly updated, they'll save everyone time and drive up research standards. For example, we might reject any interview-only study based on less than 10 hours of interviews.
12) Make no exceptions. Exceptions will inevitably apply more often to more powerful academics, increasing nepotism.

None of these suggestions are particularly novel or inventive. Common-sense improvements like these are only resisted because of the incorrect belief that anything that simplifies review will reduce quality. A simpler, more direct review process will encourage everyone to focus on key issues – methodology and results rather than framing, positioning and tone – increasing quality.

P.S. Long review cycles are not caused by poor reviewer incentives. This is a red herring, designed to divert criticism of the extraordinarily inefficient way we review papers, and the editors-in-chief who have the authority to improve it but choose not to.

—
Dr. Paul Ralph
Lecturer in Computer Science, University of Auckland http://paulralph.name _______________________________________________
AISWorld mailing list
AISWorld at lists.aisnet.org