[AISWorld] Downside of impact factors: Scientists engaging in 'citation stacking'

Robert Johnston robert.johnston at ucd.ie
Mon Sep 23 03:17:40 EDT 2013


I would like to point out that research metrics including impact factors 
are not simply objective measuring devices. They are highly political 
and ideological. They are an attempt to turn research activity into some 
kind of product that can be measured and managed by administrators who 
in many cases know little about research (at least the research in 
question).

Most of us would agree that research can only be evaluated effectively 
by knowledgeable academic peers. However, it is not in the interests of 
academic administrators to have research value assessed by such 
knowledgeable persons as research group leaders or unit heads, because 
then these roles become a barrier to managerial control over the means 
of academic production. Hence, the reification of metrics as a 
substitute for judgement.

Cheers Johno

Dr Robert B. Johnston
  
Mail: 81 Hunter St. Richmond, VIC, Australia. 3121
Email: Robert.Johnston at ucd.ie <Robert.Johnston at ucd.ie>
Mobile: +61 414 493 255
Skype: johno.home

On 23/09/2013 7:37 AM, Michael Cuellar wrote:
> John and Leon:
>
> Impact factors definitely have their problems. Mike Gallivan has made 
> an incisive criticism of them:
>
> Gallivan, M. J. (2009). Unpacking the Journal "Impact Factor" and Its 
> Effect on IS Research: Does it Do More Harm than Good?". Proceedings 
> of the America's Conference on Information Systems, San Francisco, CA, 
> Association for Information Systems.
>
> The whole current system of evaluating research by publication in 
> ranked journals has been recognized as having issues. At the last 
> ICIS, at the senior scholars forum the senior scholars seemed to decry 
> the evaluation of research by publication in ranked journals. 
> Additionally, Varun Grover and Kalle Lyytinen at the SIG Philosophy 
> workshop last year, decried the "script" by which we produce research. 
> This script, reinforced by our journals, is very good at producing our 
> current research but cuts out creative investigation of new theories 
> without data or exploration of data without theories.
>
> A better way might be evaluation of the scholar's Hirsch statistics. 
> These metrics consider both productivity and influence and are less 
> subject to manipulation than impact factors. Different varieties all 
> you to control for such things as recency, number of publications, 
> number of coauthors, a "megabit" articles.
>
> Best regards,
>
> Mike Cuellar
> *----------------------------------------------*
> *Michael Cuellar*
> Assistant Professor, Information Systems
> Georgia Southern University
> Information Systems Department
> PO Box 7998
> Statesboro, GA  30460-7998
> email: mcuellar at georgiasouthern.edu <mailto:mcuellar at georgiasouthern.edu>
> phone: (404)-405-4510
>
> Managing Editor, JSAIS
> Secretary, AIS Project Management SIG
> Member, Board, Southern Assoc. for Information Systems
> Member, JISE and BISE Editorial Boards
>
> On Sep 22, 2013, at 2:16 PM, John Artz <jartz at gwu.edu 
> <mailto:jartz at gwu.edu>> wrote:
>
>> Leon,
>>
>> This is an important point. My research is original and way out of 
>> the mainstream. Hence impact factors do not serve me well. I was 
>> complaining to one of our Deans a few years ago the use of impact 
>> factors. His response was something along the lines of "How can I 
>> evaluate your research if I don't look at impact factors?" My 
>> response to that was "If you don't understand my research, you really 
>> shouldn't be trying to evaluate it." I got one of those looks that I 
>> always get when I point out the follies of academia.
>>
>> Regards,
>>
>> John
>>
>>
>> On Sun, Sep 22, 2013 at 1:22 PM, Kappelman, Leon 
>> <Leon.Kappelman at unt.edu <mailto:Leon.Kappelman at unt.edu>> wrote:
>>
>>     Some of our institutions have adopted or are considering the use
>>     of impact factors as a metric for faculty performance.  Certainly
>>     our current faculty performance measures are less than perfect so
>>     it seems to be a suggestion worthy of consideration.  But impact
>>     factors are not without controversy and apparently prone to
>>     manipulation and questionable behaviors.  So before embracing
>>     impact factors, consider that at least to some extent they are a
>>     measure of what some might call "incestuous citation behaviors."
>>      Not surprising since most all of us understand that when it
>>     comes to human behavior, you get what you measure.  The big
>>     question is: Do the positives of using impact factors, or any
>>     other metric or combination of measure for that matter,
>>     sufficiently outweigh the negatives?
>>
>>     Here is some of the "food for thought" that raised my eyebrows
>>     enough to send this note:
>>
>>     http://www.nature.com/news/brazilian-citation-scheme-outed-1.13604
>>
>>     http://occamstypewriter.org/scurry/2012/08/13/sick-of-impact-factors/
>>
>>     http://retractionwatch.wordpress.com/2013/08/30/journal-retracts-two-papers-after-being-caught-manipulating-citations/
>>
>>     http://www.naturalnews.com/042152_citation_stacking_scientific_journals_dishonesty.html
>>
>>     Best wishes,
>>     Leon Kappelman
>>
>>     *"Only a virtuous people are capable of freedom." -- Benjamin
>>     Franklin *
>>
>>     ----------------------------------------------------------------------------------------------------------------------------------------------
>>     Leon A. Kappelman, Ph.D.
>>       Professor of Information Systems
>>       Director Emeritus, Information Systems Research Center
>>       Fellow, Texas Center for Digital Knowledge
>>         College of Business, University of North Texas
>>         Voice: 940-565-4698 <tel:940-565-4698>   Email: kapp at unt.edu
>>     <mailto:kapp at unt.edu>
>>     Founding Chair, Society for Information Management's Enterprise
>>     Architecture Working Group
>>     ----------------------------------------------------------------------------------------------------------------------------------------------
>>     Please consider the environment before printing this e-mail
>>
>>
>>     _______________________________________________
>>     AISWorld mailing list
>>     AISWorld at lists.aisnet.org <mailto:AISWorld at lists.aisnet.org>
>>
>>
>>
>>
>> -- 
>> *John M. Artz, PhD
>> Webpages: http://home.gwu.edu/~jartz <http://home.gwu.edu/%7Ejartz>
>> Email: jartz at gwu.edu <mailto:jartz at gwu.edu>
>>
>> *
>> There are only 10 kinds of people in the world - those who think in 
>> binary and those who don't.
>> **
>> _______________________________________________
>> AISWorld mailing list
>> AISWorld at lists.aisnet.org <mailto:AISWorld at lists.aisnet.org>
>
>
>
> _______________________________________________
> AISWorld mailing list
> AISWorld at lists.aisnet.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.aisnet.org/pipermail/aisworld_lists.aisnet.org/attachments/20130923/03dd4ed8/attachment.html>


More information about the AISWorld mailing list