[AISWorld] Sound conference desk rejection policy

Sun Feb 26 21:30:56 EST 2017

The issue of reviewing not being valued is a real one. My institution has a finely drawn workload allocation model (I got asked to account for a shortfall of 50 hours over the year!) and reviewing is not counted.

I'm seeking to fill approx 600 reviewing slots per annum as our usual journal workload. This is a significant imposition on the goodwill of colleagues.

But I digress ...

Michael's proposals regarding nurturing vs rigorous is a valuable one. Conferences have a greater value than simple exposure of research proposals or results. The less formal networking potential of conferences is a huge opportunity, particularly for those commencing a research career.

Cheers
John

Undergraduates should learn to use the library;
Masters students should use the library; and
Doctoral students should add new knowledge to the library.

-----Original Message-----
From: AISWorld [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of Michael Cuellar
Sent: Monday, 27 February 2017 1:07 PM
To: jartz at gwu.edu
Cc: aisworld at lists.aisnet.org
Subject: Re: [AISWorld] Sound conference desk rejection policy

I want to support John’s concept of nurturing vs. prestigious conferences and provide an extension.

First, we have a problem because reviewing is not valued. And by that I mean it is not an explicit part of the reward system for academics. It should be made an explicit requirement in the annual evaluation and tenure and promotion as well to review a certain number of papers. Reviewing awards should be counted (as highly?) as best paper awards towards your performance. This will go a ways toward resolving the unwillingness to review and to review properly.

Second, Conferences should identify whether they will be nurturing or “rigorous” (may I change the word?). Nuturing conferences don’t reject any papers except those out of scope and send the rest for review to provide feedback. “Rigorous” conferences might subject the papers to an initial screen to eliminate those “out of scope” papers and additionally, those what in the perspective of the program/track chairs aren’t good enough to withstand a rigorous review.

MC
----------------------------------------------
Michael Cuellar, PhD, PMP
Assistant Professor, Information Systems Georgia Southern University Information Systems Department PO Box 7998 Statesboro, GA  30460-7998
email: mcuellar at georgiasouthern.edu
phone: (404)-405-4510

Managing Editor, Journal of the Southern AIS Senior Editor, JISE Secretary, AIS Project Management SIG VP, Education, PMI Savannah Chapter

> On Feb 26, 2017, at 5:28 PM, John Artz <jartz at gwu.edu> wrote:
>
> Dear Colleagues,
>
> I would like to argue both sides of this issue. On one hand, I think
> desk rejections are not only the right but the responsibility of the
> editor. I tend to readily accept review requests and have often sent
> the editor a strongly worded note for wasting my time with a junk
> paper that should have never gotten past the editor's desk. Getting
> too many junk papers makes a reviewer less likely to accept the task
> of reviewing a paper thus making reviewers even more scarce.
>
> On the other hand, conferences can be a different story. I have sent
> papers to "nurturing" conferences with ideas that are "not ready for prime time"
> in order to get feedback on an idea in progress. I think we need
> outlets like this for emerging ideas and budding researchers. Having
> said that, I also think that the conference committee should decide
> whether a conference is to be "prestigious" or "nurturing" and make
> sure everybody knows its intent.
>
> John
>
>
>
>
>
>
> *John M. Artz, PhDWebpages: http://home.gwu.edu/~jartz
> <http://home.gwu.edu/%7Ejartz>Email: jartz at gwu.edu <jartz at gwu.edu>*
> *It is not the strongest of the species that survives, nor the most
> intelligent, but the ones most responsive to change. - Charles Darwin*
>
> On Sun, Feb 26, 2017 at 2:29 PM, Manuel Mora
> <dr.manuel.mora.uaa at gmail.com>
> wrote:
>
>> Dear colleague Cecil Huang,
>> This is a great topic. My two cents contribution: 1) IT area has
>> exploded in topics and subtopics and there is a scarcity of
>> reviewers; 2) asking to authors from the same track for being
>> reviewers plus final and fair decision from track chair can help to
>> cope with scarcity problem; and 3) desk rejection is still valid for
>> really low-quality papers that track chair can identify and avoid
>> wasted time from reviewers. Thanks, Manuel Mora On Feb 26, 2017 1:05
>> PM, "Cecil Eng Huang Chua" <aeh.chua at auckland.ac.nz>
>> wrote:
>>
>>> What's interesting about this post is that I have received several
>>> personal emails rather than posts to ISWorld.  As some people have
>>> explicitly asked me not to repost their mails, I won't do so.
>>>
>>>
>>> The correspondence I have received has been of the following forms:
>>>
>>>
>>> (1) Interesting question. Complex issue.
>>>
>>> (2) War stories of how some editor had to ask many people before
>>> someone would finally review for them.
>>>
>>> (3) Comments about how some people write a lot of papers, but won't
>> review.
>>>
>>> (4) I used to oppose desk rejections until the day I became an editor.
>>>
>>> (5) The issue of trust- if you appoint editors you need to trust
>>> them to desk reject appropriately.
>>>
>>> (6) Recommendation that journals adopt fee-based submission policies
>>> and actually pay reviewers.
>>>
>>> (7) Requests for an update on the conversation.
>>>
>>>
>>> I do think we should bring the issue out in the open and discuss it.
>>> It affects all of us, and a public discussion would better inform policy.
>>>
>>>
>>> Cecil Chua
>>>
>>>
>>> On Fri, Feb 24, 2017 at 1:37 PM -0500, "Cecil Eng Huang Chua" <
>>> aeh.chua at auckland.ac.nz<mailto:aeh.chua at auckland.ac.nz>> wrote:
>>>
>>>
>>> I would like to start a discussion on desk rejections at
>>> conferences.  I want to start this, because I am editing/have edited
>>> for two conferences now where the editor instructions were “no desk
>>> rejects.”  A desk
>> rejection
>>> occurs when either the track chair or associate editor rejects a
>>> paper without sending it to reviewers.
>>>
>>> I suspect a proper policy is not “no desk rejects.”  It isn’t “give
>> editor
>>> all the power to desk reject they want” either.  I am hoping that we
>>> will make some of the issues transparent so we can develop good policy.
>>>
>>> An argument in favor of desk rejection is the total uncompensated
>> manpower
>>> required for conference reviewing.  The typical conference structure
>>> is track chair, associate editor, and two reviewers.  So, each fully
>> reviewed
>>> paper receives 4 man-review units of effort.  If there are 50 papers
>>> per track (yes, some tracks have hundreds of papers- bear with me),
>>> that’s
>> 200
>>> man-review units in each track.  If there are 15 tracks, that’s 3000
>>> man-review units.  At just 5 major conferences  (ICIS, AMCIS, ECIS,
>> PACIS,
>>> ACIS), that’s 15000 man-review units, and we have more than just 5
>>> major conferences.  Desk rejections can shave a lot of uncompensated
>>> man-review units from this.
>>>
>>> A counter argument is that one can get authors of a track to review
>>> for the track.  I would note that if the editor feels a paper is of
>>> desk rejection quality, that the authors may not be competent to review.
>>>
>>> An argument against desk rejection is conferences are about
>>> providing feedback to authors.  However, this requires uncompensated
>>> time from reviewers.  In many of our premier conferences, there’s a
>>> pre-submission game where the track chairs try to “reserve” as many
>>> reviewers/AEs as possible.  We wouldn’t have that game if there
>>> wasn’t a shortage of reliable reviewers.  Indeed, there probably
>>> aren’t more that 5000 active researchers in the IS field at any one
>>> time- see above 15000 man-review units.  Everyone gets involved in
>>> the reviews.  It is unfair to favor authors who benefit from reviews
>>> rather than reviewers who are harmed (by having to spend uncompensated time) when they have to do reviews.
>>>
>>> Frankly, our community seems to underappreciate reviewers.  For the
>>> above conservative 15000 man-review units, we maybe give 5 best
>>> reviewer
>> awards,
>>> which are often paper certificates with no money attached.  Best
>>> paper authors receive shiny plaques and often a cheque.  We could
>>> argue
>> reviewer
>>> competence is recognized because reviewers are invited to become
>>> track chairs and editors.  To this, I would note the pyramidal
>>> structure of conference/journal organization-many reviewers, few
>>> board positions and
>> the
>>> fact that appointment to boards is not explicitly based on reviewing
>>> competence and is often based on getting papers accepted.  Also, it
>>> is
>> not
>>> clear to me that being appointed to boards is necessarily a reward.
>>>
>>> So, that’s my discussion seed.  I’d like to hear your thoughts.
>>>
>>> Cecil Chua
>>> _______________________________________________
>>> AISWorld mailing list
>>> AISWorld at lists.aisnet.org
>>>
>>> _______________________________________________
>>> AISWorld mailing list
>>> AISWorld at lists.aisnet.org
>> _______________________________________________
>> AISWorld mailing list
>> AISWorld at lists.aisnet.org
> _______________________________________________
> AISWorld mailing list
> AISWorld at lists.aisnet.org

_______________________________________________
AISWorld mailing list
AISWorld at lists.aisnet.org

Important Notice: The contents of this email are intended solely for the named addressee and are confidential; any unauthorised use, reproduction or storage of the contents is expressly prohibited. If you have received this email in error, please delete it and any attachments immediately and advise the sender by return email or telephone.

Deakin University does not warrant that this email and any attachments are error or virus free.