[AISWorld] On quality of journals and quality of reviews

[Parsons, David]

I have been a journal editor in chief for the last 5 years and am increasingly annoyed by authors who criticise reviewers. Here are some excerpts from an email I received from an author who's paper I rejected after review:

"What kind of paper are you running down there?... we take the time and effort to develop the paper further, and now you are referring to technical issues of selection and limited space ??? Just disrespectful !!!....who is this third reviewer? This guy is obviously biased... this BS reviewer should be discarded as an outlier"

More recently I was pressured by a well-known author in my field to publish a paper that I rejected on the grounds that I felt its focus was wrong for the journal. I offered suggestions for refocusing the article so it could be considered but he kept coming back to me trying to argue that his paper was fine as it was.

The too-common perception seems to be that somehow editors and reviewers are paid lackeys of the publishers, rather than academics voluntarily trying to do their bit for their research community, usually in the evenings and at the weekends since these tasks are over and above our normal duties. I rely heavily on a relatively small core of reviewers to carry an increasingly heavy peer reviewing responsibility.

I wholeheartedly agree with Murray that "it seems everyone wants/has to author papers and few have the desire/time to review." All academics should review on a  regular basis. Just having your name listed on an editorial board is not enough.

Dave Parsons

**************************************************************

Message: 10
Date: Wed, 16 Oct 2013 13:43:18 +0000
From: William Tastle <tastle at ithaca.edu>
To: "aisworld at lists.aisnet.org" <aisworld at lists.aisnet.org>
Subject: Re: [AISWorld] On quality of journals and quality of reviews
	!
Message-ID:
	<18bf16e47fa44cd08c0e1d271bae1777 at BLUPR07MB387.namprd07.prod.outlook.com>
	
Content-Type: text/plain; charset="us-ascii"

I, too, am a former journal editor and the difficulties in securing proper reviewers is sometimes a bit of a challenge.  However, I consider it a responsibility of scholars to engage in a mentoring function with respect to paper reviewing.  In general, the best reviews are from the senior researchers and hence, they are thepeople most bombarded with review requests.  Those reviews are nurturing and guiding as they attempt to raise the quality of content (and writing style) such that the paper crosses the threshold of acceptability.  The worst reviews, again speaking very generally and without benefit of a scientific effort to confirm my experiential observations, are from the younger, or perhaps more insecure, scholars who seemingly engage in a "slap leather" attitude towards the review process.  Their reviews are more apt (again, a generalization) to be vicious in content and of little use to the paper author.

I recall a story from Ben Schneiderman about how his seminal paper on structure diagrams was severely trashed by some reviewers until it was finally published, only to become one of the most downloaded papers in the journal's history.  I heard a similar story, again first hand, from Lotfi Zadeh on his theory of fuzzy sets.  They persevered to success.  We have a responsibility to mentor our colleagues, just as we have (hopefully) been mentored ourselves.

However, this does not excuse a poorly written or researched paper.  That is sloppy work and papers of that sort should simply be returned to the authors without any review until the basic threshold of scientific literacy is reached.

Bill Tastle, Professor of IS, Ithaca College

From: aisworld-bounces at lists.aisnet.org [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of MurphJen at aol.com
Sent: Tuesday, October 15, 2013 8:42 PM
To: mmora at securenym.net; aisworld at lists.aisnet.org
Subject: Re: [AISWorld] On quality of journals and quality of reviews !

I'm not sure what the point of the below is but I'll add my perception as a journal editor in chief.  Quite frankly authors are lucky to get reviews.  I have seen a flood of submissions the last 2 years from all parts of the world where before I saw mostly US and European submissions.  I think this is good but, there are a lot of junior level papers and not enough senior level reviewers to go around.  The trend I do not like is that of authors feeling it is their right to get their reviews quick.  I'm not sure if this is an output of the open source journals that have promised fast turnarounds, but to get quality reviews takes time as there simply aren't enough quality reviewers who have the time to be almost full time reviewers.  My gripe (and I admit it is as an EiC) is that it seems everyone wants/has to author papers and few have the desire/time to review.  Also, unfortunately, new authors don't seem to understand the concept of a thorough literature review and seem to ha
 ve taken the debate on the ethics of recommending papers from the journal being submitted to, or of previous papers from senior reviewers as license to ignore them.

This leads to my major question: Why should a journal spend limited reviewing resources on doing thorough reviews of papers that do not meet the basic standard of scholarly research by grounding themselves in the literature.

I am tired of the excuse that the authors do not have access to the articles so they ignore them.  I see so much research that authors consider new but is at best a minor extension of something that has been published but the authors did not or could not get the article to know it.

I think we are at a crossroads of scholarly literature.  We argue about plagiarism but I think we are seeing a greater issue of authors not looking at the literature so that they can make the claim "there is little research" or that "this is new" as a justification for their paper.  To give credit we have to recognize the research that has been done, to build a body of knowledge we have to build on what has been done, not do it over and over again.

If authors want quality reviews they need to show the respect to the senior scholars by at least looking at their work and building on it rather than trying to waste their time by making them tell them what literature to look at rather than addressing the quality of the paper.

Also, I propose that we establish a new ethic: authors can only submit if they serve as reviewers.  I am also tired of hearing the cries of needing to get published to get tenure or promotion so they don't have time to review.  We need all universities to start recognizing the intellectual contribution of reviewing as being on equal or near equal par with authoring.  Do this and we will all have quality reviews.

sorry to rant but you can see this has touched a nerve that has been throbbing for a while.

Murray E. Jennex, Professor of MIS, San Diego State University

In a message dated 10/15/2013 5:05:48 P.M. Pacific Daylight Time, mmora at securenym.net<mailto:mmora at securenym.net> writes:
Dear colleagues,
In past weeks, a strong and relevant debate on the academic influence
of ISI listed journals was discussed in this forum. Well, on same ideas
I wonder whether there are studies on the quality of the reviewers (e.g.
level of seniority, level of expertise in the topic, level of expertise
on research methods, and in particular on the role played as peers
seeking to improve the advance of science and suggesting clear insights
rather some simple elaborations of flaws without any rational justification).
In summary, are we living in a spiral of hard reviewers and bad researchers?
or rather the opposite one is the reality? I am sure that Senior researchers
will have reviewed rare reviews from people with less expertise and seniority
level, so comments on it are welcome ! Of course, this non ethical practice
should be eliminated. As a funny real history, a Mexican top researcher
in Education received a strong critique on the null value of paper that
he wrote, and the suggestion was to use some papers writen by him in the
past (of course, the reviewers did know it). I know of other cases similar
cases. Well, sciences is about truthness but wrong reviews maybe are now a
real headache ! Cheers !
Manuel Mora
ACM Senior Member