[AISWorld] Fwd: Re: On quality of journals and quality of reviews !

[mmora at securenym.net]

Dear colleagues in AIS:

Motivated by the recent debate on the quality of journals and the
influence of ISI listed ones, I kindly raised the logical previous
step regarding the quality of reviews. Many interesting comments
have been shared by other colleagues. In particular the recommendations
from A.S.Lee (a past EiC of MIS Quarterly) on 15 key points that
should be addressed by anyone who self-named an ETHICAL REVIEWER
(posted by John Lamp) are excellent. Link is:
http://www.people.vcu.edu/~aslee/referee.htm

On it, while the problematic of the existence of pseudo-scientific
journals with fee rates for publishing is real, I believe that it is not
our main
concern. I mean, we must be worried about well-trained PhD people
that conducting a rather emotional than rational review, and that
by their novelty in research topics, their reviews are more flawed
that the paper on review. It regrettably happens with just graduated
PhDs (from any Institution to be honest) given by a plausible academic
pressure for being "hard". A PhD training is just a training to know
1 or 2 research methods (from the available 25+) and an expertise
of 1 journal and 2-3 conference papers.

In summary, the ethic concern that we should formalize/legitimatize
are the reviewers' requirements (e.g. the expected years of experience
(after the PhD completion), the self-evaluation of the topics to be
reviewed, and the willingness to follows -at least- some of the excellent
recommendations provided by A.S. Lee and reminded to us by John Lamp).

The anecdotal real history on core contributions by Schneirderman,
and Zadeh, on serius journals suggest a real problem. Furthermore,
the awarded 1978 Economy Nobel Prof. Herbert A. Simon reported in
his biographic book, that one of their first papers on statistics
methods was wrongly rejected at first (he gained his PhD in Political
Sciences) but fortunately this mistake was corrected.



On Thu, October 17, 2013 9:37 am, Galletta, Dennis wrote:
> As a journal editor, these stories are always frightening. But to provide
> a little levity, and a few smiles, you MUST read the following blog entry
> so you can see the hilarious clues about the paper’s authenticity, or
> lack thereof:
> http://retractionwatch.wordpress.com/2013/09/23/a-serbian-sokal-authors-s
> poof-pub-with-ron-jeremy-and-michael-jackson-references/
>
> It will lead you to try and read a little of the paper, only to recoil in
> pain and agony.
>
> In that otherwise hilarious blog, some were questioning the actual
> existence of the journal. I am unable to find the journal in the EBSCO
> system under Engineering, so the jury still seems to be out given some of
> their links are to EBSCO that purport to defend its existence. This
> recalls a famous similar hoax in 1996 by Sokal that has been covered
> extensively elsewhere (outlined nicely in Wikipedia at
> http://en.wikipedia.org/wiki/Sokal_affair).
>
>
> I believe the editorial process will stand up to these jokes. The
> marginal journals are the most vulnerable. But in any event, reviewers
> and editors obviously should read the papers they agree to review.
>
> DG
>
>
> -------------------------------------------------------------------------
> ---
> Dennis F. Galletta                      Professor of Business
> Administration
> University of Pittsburgh                 and Director, Katz Doctoral
> Program
> 282a Mervis Hall                            Katz Graduate School of
> Business
> Phone +1 412-648-1699                                  Pittsburgh, PA
> 15260
> E-mail: galletta @                                       Fax +1
> 412-648-1693
> katz.pitt.edu                       homepage:
> www.pitt.edu/~galletta<http://www.pitt.edu/~galletta>
> --------------------------------------------------------------------------
> --
>
>
> From: aisworld-bounces at lists.aisnet.org
> [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of Ejub Kajan
> Sent: Thursday, October 17, 2013 10:03 AM
> To: aisworld at lists.aisnet.org
> Cc: ekajan at ieee.org
> Subject: [AISWorld] Fwd: Re: On quality of journals and quality of reviews
> !
>
>
> Wake up people,
>
>
>
>
> ------- Forwarded message -------
> From: "Ejub Kajan" <ejubkajan at sbb.rs<mailto:ejubkajan at sbb.rs>>
> To: aisworld at lists.aisnet.org<mailto:aisworld at lists.aisnet.org>
> Cc: ekajan at ieee.org<mailto:ekajan at ieee.org>
> Subject: Re: [AISWorld] On quality of journals and quality of reviews !
> Date: Thu, 17 Oct 2013 15:27:25 +0200
> My dear friends and colleges,
>
>
> Attached is an article that addressed several important issues on papers
> reviews, where the misserable of review process *or not at all) has
> occurred . Authors have  proved , with a lot of carriage (the paper has
> made intentionally), that everything is possible if there no control over
> it, that so-called scientific people may reach full professor positions
> due that administration.
>
> I would like to pay attention to reference section where Walt Disney
> actor has been cited, Laplas has given his oppinion on the Web and many
> other stpitides,
>
> Actually, there is no reviews, authors paid for paper appearance,
> conditions applied are cited articles in that journal and paid fee. WHAT
> ISI (THOMSON REUTERS) yet wait to send this journal into history.
>
>
> I think that authors of this  paper should be reworded by many thanks by
> scientific community.
>
> Thanks,
> Ejub
>
>
>
> On Thu, 17 Oct 2013 00:25:57 +0200, John Lamp
> <john.lamp at deakin.edu.au<mailto:john.lamp at deakin.edu.au>> wrote:
>
>
> When dealing with new reviewers, you could not really do better than to
> direct them to Allen Lee’s note on reviewing a manuscript for
> publication. It was originally published in the Journal of Operations
> Management, but is online at:
> http://www.people.vcu.edu/~aslee/referee.htm
>
>
> Fifteen simple (well maybe) criteria to address to create a decent
> review. I still refresh my mind from it when doing a review.
>
> Cheers
> John
>
>
> From:
> aisworld-bounces at lists.aisnet.org<mailto:aisworld-bounces at lists.aisnet.or
> g> [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of William Tastle
>  Sent: Thursday, 17 October 2013 12:43 AM
> To: aisworld at lists.aisnet.org<mailto:aisworld at lists.aisnet.org>
> Subject: Re: [AISWorld] On quality of journals and quality of reviews !
>
>
> I, too, am a former journal editor and the difficulties in securing
> proper reviewers is sometimes a bit of a challenge.  However, I consider
> it a responsibility of scholars to engage in a mentoring function with
> respect to paper reviewing.  In general, the best reviews are from the
> senior researchers and hence, they are thepeople most bombarded with
> review requests.  Those reviews are nurturing and guiding as they attempt
> to raise the quality of content (and writing style) such that the paper
> crosses the threshold of acceptability.  The worst reviews, again
> speaking very generally and without benefit of a scientific effort to
> confirm my experiential observations, are from the younger, or perhaps
> more insecure, scholars who seemingly engage in a “slap leather”
> attitude towards the review process.  Their reviews are more apt (again,
> a generalization) to be vicious in content and of little use to the paper
> author.
>
> I recall a story from Ben Schneiderman about how his seminal paper on
> structure diagrams was severely trashed by some reviewers until it was
> finally published, only to become one of the most downloaded papers in
> the journal’s history.  I heard a similar story, again first hand, from
> Lotfi Zadeh on his theory of fuzzy sets.  They persevered to success.  We
> have a responsibility to mentor our colleagues, just as we have
> (hopefully) been mentored ourselves.
>
>
> However, this does not excuse a poorly written or researched paper.  That
> is sloppy work and papers of that sort should simply be returned to the
> authors without any review until the basic threshold of scientific
> literacy is reached.
>
> Bill Tastle, Professor of IS, Ithaca College
>
>
> From:
> aisworld-bounces at lists.aisnet.org<mailto:aisworld-bounces at lists.aisnet.or
> g> [mailto:aisworld-bounces at lists.aisnet.org] On Behalf Of
> MurphJen at aol.com<mailto:MurphJen at aol.com>
> Sent: Tuesday, October 15, 2013 8:42 PM
> To: mmora at securenym.net<mailto:mmora at securenym.net>;
> aisworld at lists.aisnet.org<mailto:aisworld at lists.aisnet.org>
> Subject: Re: [AISWorld] On quality of journals and quality of reviews !
>
>
> I'm not sure what the point of the below is but I'll add my perception as
> a journal editor in chief.  Quite frankly authors are lucky to get
> reviews.  I have seen a flood of submissions the last 2 years from all
> parts of the world where before I saw mostly US and European submissions.
> I think this is good but, there are a lot of junior level papers and not
> enough senior level reviewers to go around.  The trend I do not like is
> that of authors feeling it is their right to get their reviews quick.
> I'm not sure if this is an output of the open source journals that have
> promised fast turnarounds, but to get quality reviews takes time as there
> simply aren't enough quality reviewers who have the time to be almost
> full time reviewers.  My gripe (and I admit it is as an EiC) is that it
> seems everyone wants/has to author papers and few have the desire/time to
> review.  Also, unfortunately, new authors don't seem to understand the
> concept of a thorough literature review and seem to have taken the debate
> on the ethics of recommending papers from the journal being submitted to,
> or of previous papers from senior reviewers as license to ignore them.
>
> This leads to my major question: Why should a journal spend limited
> reviewing resources on doing thorough reviews of papers that do not meet
> the basic standard of scholarly research by grounding themselves in the
> literature.
>
> I am tired of the excuse that the authors do not have access to the
> articles so they ignore them.  I see so much research that authors
> consider new but is at best a minor extension of something that has been
> published but the authors did not or could not get the article to know
> it.
>
> I think we are at a crossroads of scholarly literature.  We argue about
> plagiarism but I think we are seeing a greater issue of authors not
> looking at the literature so that they can make the claim "there is
> little research" or that "this is new" as a justification for their
> paper.  To give credit we have to recognize the research that has been
> done, to build a body of knowledge we have to build on what has been
> done, not do it over and over again.
>
> If authors want quality reviews they need to show the respect to the
> senior scholars by at least looking at their work and building on it
> rather than trying to waste their time by making them tell them what
> literature to look at rather than addressing the quality of the paper.
>
> Also, I propose that we establish a new ethic: authors can only submit if
> they serve as reviewers.  I am also tired of hearing the cries of needing
> to get published to get tenure or promotion so they don't have time to
> review.  We need all universities to start recognizing the intellectual
> contribution of reviewing as being on equal or near equal par with
> authoring.  Do this and we will all have quality reviews.
>
> sorry to rant but you can see this has touched a nerve that has been
> throbbing for a while.
>
> Murray E. Jennex, Professor of MIS, San Diego State University
>
>
> In a message dated 10/15/2013 5:05:48 P.M. Pacific Daylight Time,
> mmora at securenym.net<mailto:mmora at securenym.net> writes:
> Dear colleagues,
> In past weeks, a strong and relevant debate on the academic influence
> of ISI listed journals was discussed in this forum. Well, on same ideas I
> wonder whether there are studies on the quality of the reviewers (e.g.
> level of seniority, level of expertise in the topic, level of expertise on
> research methods, and in particular on the role played as peers seeking to
> improve the advance of science and suggesting clear insights rather some
> simple elaborations of flaws without any rational justification). In
> summary, are we living in a spiral of hard reviewers and bad researchers?
>  or rather the opposite one is the reality? I am sure that Senior
> researchers will have reviewed rare reviews from people with less
> expertise and seniority level, so comments on it are welcome ! Of course,
> this non ethical practice should be eliminated. As a funny real history, a
> Mexican top researcher
> in Education received a strong critique on the null value of paper that he
> wrote, and the suggestion was to use some papers writen by him in the past
> (of course, the reviewers did know it). I know of other cases similar
> cases. Well, sciences is about truthness but wrong reviews maybe are now a
>  real headache ! Cheers ! Manuel Mora
> ACM Senior Member
>
>
>
> _______________________________________________
> AISWorld mailing list
> AISWorld at lists.aisnet.org<mailto:AISWorld at lists.aisnet.org>
>
>
> Important Notice: The contents of this email are intended solely for the
> named addressee and are confidential; any unauthorised use, reproduction
> or storage of the contents is expressly prohibited. If you have received
> this email in error, please delete it and any attachments immediately and
> advise the sender by return email or telephone.
>
> Deakin University does not warrant that this email and any attachments
> are error or virus free.
>
>
> --
> Ejub Kajan
> State University of Novi Pazar
> Editor of Electronic Business Interoperability: Concepts, Opportunities
> and Challenges Editor of Handbook of Research on E-Business Standards and
> Protocols
>
>
> Using Opera's revolutionary email client: http://www.opera.com/mail/
>
>
>
> --
> Ejub Kajan
> State University of Novi Pazar
> Editor of Electronic Business Interoperability: Concepts, Opportunities
> and Challenges Editor of Handbook of Research on E-Business Standards and
> Protocols
>
>
> Using Opera's revolutionary email client: http://www.opera.com/mail/
> _______________________________________________
> AISWorld mailing list
> AISWorld at lists.aisnet.org