[AISWorld] Sound conference desk rejection policy

[MurphJen at aol.com]

We actually had a discussion on the review process in 2016 including  
publications in CAIS. The problems with reviews are recognized and many  
proposals have been made to improve the process, however, reducing conference  
reviews and reviewing quality is not one of them, including conference desk  
rejection.  I won't say that conference desk rejection should never be  done, 
but I do suggest it be done rarely, perhaps only in those cases where the  
submission just does not fit the conference/track/minitrack theme.   Otherwise 
I can tell you that my suggestion in CAIS is that we put more emphasis  on 
the conference reviews and use the conference reviews to shorten journal  
review cycles and work load.  We are starting to see journals becoming more  
accepting of taking articles straight from the conference with little or no  
modification, this can only be done if we keep the conference review process  
rigorous.  I know this is not what you want to hear but I really think we  
have to be judicious with conference desk rejects.  In addition to not  
fitting the conference perhaps the only other times a desk reject is warranted  
is for a submission that is not complete, does not meet the conference  
submission guidelines, or is so bad that anyone looking at it would agree it is  
not worth sending out for review (a hard thing to quantify though and of 
course  if this is done the conference should publish that desk rejection is 
possible in  the cfp).  My CAIS article is:
 
 
Jennex,  M.E., (2016) "No Free Lunch: Suggestions for Improving the Quality 
of the Review  Process," Communications of the Association for Information 
Systems: Vol. 38,  Article 17. Available at: 
http://aisel.aisnet.org/cais/vol38/iss1/17
 
Note that I am editor in chief of the International Journal of Knowledge  
Management and co-editor in chief of the International Journal of Information 
 Systems for Crisis Response and Management so of course I do have my 
biases on  the review process (just saying this for full disclosure).  Given the  
growth in the number of conferences and the pressures of publishing I 
recognize  that there is a big growth in the number of submissions so the below 
issue of  people not being willing to review is one has to be addressed.  I  
personally believe that you should do two or three reviews for every 
submission  you make and that reviewing should be considered as much of a 
intellectual  contribution as writing.  So basically I do not believe any member of 
our  community who is publishing should refuse to review (and conferences 
should also  publish any expectation of authors reviewing in the cfp).  Just my 
2 cents  worth....murray jennex
 
Professor MIS, Fowler College of Business, San Diego State University
 
 
In a message dated 2/26/2017 5:48:54 P.M. Pacific Standard Time,  
aeh.chua at auckland.ac.nz writes:

What's  interesting about this post is that I have received several 
personal emails  rather than posts to ISWorld.  As some people have explicitly 
asked me  not to repost their mails, I won't do so.


The correspondence I have  received has been of the following forms:


(1) Interesting question.  Complex issue.

(2)​ War stories of how some editor had to ask many  people before someone 
would finally review for them.

(3) Comments about  how some people write a lot of papers, but won't review.

(4) I used to  oppose desk rejections until the day I became an editor.

(5) The issue  of trust- if you appoint editors you need to trust them to 
desk reject  appropriately.

(6) Recommendation that journals adopt fee-based  submission policies and 
actually pay reviewers.

(7) Requests for an  update on the conversation.


I do think we should bring the issue  out in the open and discuss it.  It 
affects all of us, and a public  discussion would better inform policy.


Cecil Chua


On  Fri, Feb 24, 2017 at 1:37 PM -0500, "Cecil Eng Huang Chua"  
<aeh.chua at auckland.ac.nz<mailto:aeh.chua at auckland.ac.nz>>  wrote:


I would like to start a discussion on desk rejections at  conferences.  I 
want to start this, because I am editing/have edited for  two conferences now 
where the editor instructions were “no desk  rejects.”  A desk rejection 
occurs when either the track chair or  associate editor rejects a paper 
without sending it to reviewers.

I  suspect a proper policy is not “no desk rejects.”  It isn’t “give 
editor  all the power to desk reject they want” either.  I am hoping that we 
will  make some of the issues transparent so we can develop good policy.

An  argument in favor of desk rejection is the total uncompensated manpower 
 required for conference reviewing.  The typical conference structure is  
track chair, associate editor, and two reviewers.  So, each fully  reviewed 
paper receives 4 man-review units of effort.  If there are 50  papers per 
track (yes, some tracks have hundreds of papers- bear with me),  that’s 200 
man-review units in each track.  If there are 15 tracks,  that’s 3000 
man-review units.  At just 5 major conferences  (ICIS,  AMCIS, ECIS, PACIS, ACIS), 
that’s 15000 man-review units, and we have more  than just 5 major 
conferences.  Desk rejections can shave a lot of  uncompensated man-review units from 
this.

A counter argument is that  one can get authors of a track to review for 
the track.  I would note  that if the editor feels a paper is of desk 
rejection quality, that the  authors may not be competent to review.

An argument against desk  rejection is conferences are about providing 
feedback to authors.   However, this requires uncompensated time from reviewers. 
 In many of our  premier conferences, there’s a pre-submission game where 
the track chairs try  to “reserve” as many reviewers/AEs as possible.  We 
wouldn’t have that  game if there wasn’t a shortage of reliable reviewers.  
Indeed, there  probably aren’t more that 5000 active researchers in the IS 
field at any one  time- see above 15000 man-review units.  Everyone gets 
involved in the  reviews.  It is unfair to favor authors who benefit from reviews 
rather  than reviewers who are harmed (by having to spend uncompensated tim
e) when  they have to do reviews.

Frankly, our community seems to  underappreciate reviewers.  For the above 
conservative 15000 man-review  units, we maybe give 5 best reviewer awards, 
which are often paper  certificates with no money attached.  Best paper 
authors receive shiny  plaques and often a cheque.  We could argue reviewer 
competence is  recognized because reviewers are invited to become track chairs 
and  editors.  To this, I would note the pyramidal structure of  
conference/journal organization-many reviewers, few board positions and the  fact that 
appointment to boards is not explicitly based on reviewing  competence and is 
often based on getting papers accepted.  Also, it is  not clear to me that 
being appointed to boards is necessarily a  reward.

So, that’s my discussion seed.  I’d like to hear your  thoughts.

Cecil  Chua
_______________________________________________
AISWorld mailing  list
AISWorld at lists.aisnet.org

_______________________________________________
AISWorld  mailing  list
AISWorld at lists.aisnet.org