[AISWorld] More on journal cycle times

Wed Apr 1 05:56:51 EDT 2015

There are flaws in the below logic, yes a good review can be done in a few  
hours but unfortunately we are all limited by time and what we can focus 
our  attention on.  I'm an editor in chief of two journals and I can say that 
we  have more people wanting to submit articles than we have people wanting 
to  review articles.  It is simple economics of publish or perish with 
little  to no reward for reviewing.  I think I see only occasionally an email 
that  mentions the best reviewer but see many emails on best papers or 
announcing new  issues being published.  A few hours seems like a little, but how 
much time  do we spend doing our jobs?  Many of us have to teach as well as do 
 research, and of course there is always service.  The last several years 
of  poor budgets has raised the teaching and service loads.  We do give 
course  reductions as rewards.  I have never, ever seen a course reduction given  
because of reviewing but do see it frequently for publishing articles.   
Also, how many do great reviews?  I get new reviewers and many submit  simple 
one paragraph reviews, not helpful to me or the authors.  So what do  we do? 
 We punish bad reviewers by not asking them to review and reward  good 
reviewers by giving them more and more reviews.  Doesn't work, good  reviewers 
get to the point where they refuse further reviews, many are burned  out, and 
who can blame them?  I have seen many complaints on this list  about long 
review times and poor reviews but I have yet to see anyone push  changing the 
publishing economics (yes we do respond to the complaints with this  
suggestion but then we hear how we just need to work harder and we owe it to  
authors to get their papers processed).  

I agree we need to change the review process, but please, lets quit  
complaining about the editors and reviewers and instead suggest ways to make the  
publishing process manageable and rewarding to all those who make it work.   
I am getting a break next year because of being a editor in chief, but I've 
also  been told it won't happen again, I need to publish my own research if 
I want any  more reductions.  Guess what, I have several papers stacked up 
waiting for  me to write them and analyze the data, but they are on hold as 
I service other  authors in getting their work published.  Please remember 
next time you  want to complain about review times all the editors and 
reviewers who are  volunteering their time to try and make this process work.

If you want to address a large part of the problem lets look at conference  
reviewing.  How many conference announcements do we see? (hundreds) and  
each one of these conferences has to review their papers and of course most  
universities don't want to send their faculty to present at a conference that 
 accepts all papers, they want a 50% acceptance rate, so to get that how 
many  papers do we need to fill in all the tracks, minitracks, and sessions?   
ICIS boasts a very low acceptance rate an is such is considered the best, 
so  what happens?  We get dozens pre and post conferences around ICIS because 
 again, most of our universities won't send us to a conference unless we 
have a  paper accepted.  So how much reviewing time is spent on conference  
papers?  And what is the reward for having a conference paper  accepted?  For 
most universities it is very little yet we spend amazing  amounts of 
reviewer resources on these papers.

If you want faster review times for journals fix the conference reviewing  
issue.  That is my best suggestion short of making reviewing a part of  
tenure and promotion.

Thanks and sorry for the rant....murray jennex

In a message dated 3/31/2015 10:15:55 A.M. Pacific Daylight Time,  
mmora at securenym.net writes:

AIS  colleagues:
First of all, I appreciate totally comments from senior  researchers who
know "the real inside situation" lived in top tier journals  in our
discipline. I agree with Professor Stafford, who has a vast  expertise as
EiC of top tier journals on this undesired situation, where it  seems there
is not an easy solution for this messy academic problem in our  discipline.
On plausible feasible solutions, awards for reviewers should be  also
provided, but it is assumed implicitly for belonging to ERB of  top
journals. In summary, it seems there is not a near and feasible  solution
in our discipline, which runs the risks to be a closed system  (papers
published only for gaining a PhD or a tenure track position or  being read
mandatory by our graduate students) but rarely transferred to  main
stakeholders (IT customers, IT users, IT community)). Of course, there  are
exceptions in our discipline like Communications of the ACM, like one  of
the best examples of top tier journals. Thanks, more comments are  welcome!
Manuel Mora
PS. High quality reviewers should be able to  qualify a paper in few hours,
like medical experts can establish a critical  diagnosis on few ones (even
in minutes). Are our papers (in our discipline)  so difficult to be
understood regarding the research papers in Chemistry,  Health Sciences or
Biological Sciences?

On Mon,  March 30, 2015 9:06 pm, Thomas Stafford (tstaffor) wrote:
> Prof Mora  raises interesting issues in his response to my response to his
>  well-put points on publication queues  :-)
>
>>>>  judges for accepting a paper are the reviewers, and they are  not
>>>> valued.>>>
>
> I agree. As long  as reviewing is considered unimportant by department
> chairs, Deans,  and tenure and promotion committees, reviewers are going
> to give first  priority to the work that gets them promoted and tenured.
> Who would  not?  Until we find a solution to this, much else that we can
> do  is only a bandaid solution. Incented reviewers are critical to the
>  process.
>
> There is a pragmatic yet informal quid pro quo at  journals:  "those who
> wish to enjoy the peer review services  should contribute to the peer
> review services as well." It is not  actually stated as a requirement, but
> everybody understands. Not  everybody honors it, but it is known. However,
> that still leaves  reviewing, which the Dean does not view with approval
> at tenure  decision time, as a "last in the queue" work  activity.
>
>>>> the effective time for review a paper  can be estimated on 2-3 hours
>>>> in 2-3 iterations, so the  problem is the scheduling of the paper
>>>> assigned by the  reviewer, so it is very likely that the paper is in
>>>> stand  by for months. This is the process review problem. None of  us
>>>> can believe that the review process lasted several  months because
>>>> reviewers read the paper all of this  time.>>>
>
> Well, yes and no. For no, see above:   reviewing is a last-in-line
> activity owing to how it is (not) incented  by the bodies who decide rank
> and tenure. For yes, we have to question  the assumption of time to
> initiation. There is the simple Poisson  problem of time to point of
> processing in waiting queues, but these  problems are always analyzed
> ceteris paribus. Plainly put, we can't  easily assume that a reviewer
> attends immediately to an assignment  once received, so even if we can
> optimize the queue to assignment, the  wait time for processing once
> assigned also depends on myriad factors.  Generally, we scholars who
> regularly review teach our classes and  attend our committee meetings
> before we turn to our outside service  work. There is a wait time.
>
> But, in all, I still agree with  the Professor:  it only takes a few hours
> to render a quality  evaluation of a paper. The slowdown is in the
> intake/assignment  process, if one puts aside the notion that reviewers
> should jump  immediately to assignments when received.
>
>>>> Biology,  Chemistry, Medicine have faster
>>>>  review-acceptance/rejection time frames and they have zero  fee
>>>> costs for publishing
>
> Stay tuned. This  is exactly where I intend to do my own benchmarking.
> Fortunately,  quite fortunately, indeed, I have on my Senior Board a
> credentialed MD  who also holds the Ph.D. in operations. He is a treasure,
> since we get  lots of healthcare papers. He also edits for and publishes
> in the  medical community, and he and I just shared a time on the phone
>  talking about these things. The defining factor I discern in the
>  fast-as-lighting turnaround times at the preeminent medical journals  is
> *professional staff* because publications like JAMA and NEJM are  run by a
> professionals who don't have to teach classes or publish  their own
> research. Nothing gets work done like paying people to do  it, I say.
>
> There is another journal in the Sciences legendary  for quick turnaround:
> PLOS, the Public Library of Science. However, it  is a business,
> unabashedly. Each accepted article carries a $1300 fee  for acceptance,
> and the revenue stream from that supports an  operational staff of 15 who
> see to it quite effectively that the  trains run on time. They do have
> some technological innovations  (amazing what you can do with money, isn't
> it?), which are things that  have been in my mind lately, as well.
> Searchable database of reviewer  expertise and current assignment loads,
> for example, where you can  instantly search out just the right set of
> credentials for reviewing a  given paper and then check to see if that
> chosen individual is not  already over-assigned or not.
>
> I predict we are at the cusp of  a paradigm shift in peer review of
> scholarly research. Emerging  business models are worth considering, and
> science-as-business is also  a notion not to be dismissed. Most of what
> keeps us alive and healthy  in the world of medicine resides in some point
> on the profit motive of  somebody somewhere in the medical food chain. It
> results in  lightning-fast research reviews for publication, though, and
> that gets  my attention.
>
> On the side of constituent pressure for quicker  queues to acceptance for
> articles, there is the equally important and  increasingly popular issue
> of what I will call "publication bounties."  When scholars are paid
> interestingly large amounts of money as a bonus  for successful
> publication in premier journals -- well, of course they  get anxious to
> find out what has happened with a given submission as  quickly as
> possible. Big money is riding on that decision. Yet, it  remains that the
> folks upon whom they rely and become increasingly  impatient with for
> results are not given bounties for reviewing the  work that will be
> subsequently rewarded when published. This is a  business scholarship
> issue to think about; we appear to be handsomely  incentivizing the
> production of scholarly research, but not its  review. Marx would have
> field day with that economic  equation.
>
> I am so glad folks are noticing this issue. We won't  get anywhere on the
> top-level problem, which is incentivizing  reviewers, without widespread
> awareness of the issue at a general  level, I think.
>
> Humbly submitted for your further  consideration,
>
>
> Tom Stafford
> Editor, Decision  Sciences  Journal
>
>
>

_______________________________________________
AISWorld  mailing  list
AISWorld at lists.aisnet.org