<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><base href="x-msg://1391/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I think we have seen in this post that Impact Factors should not be used for evaluation of scholarly productivity given their volatility and ability to manipulated.<div><br></div><div>That is why we have advocated using the Hirsch indices as a measure of the "scholarly influence" of the scholar in the field. Depending on how you define the field, you can use the Hirsch indices to assess impact on the scholarly community, or the practitioner community or both or the social world. </div><div><span style="text-indent: -36px; "><br></span></div><div style="text-indent: 0px;"><span style="text-indent: -36px; ">The Hirsch indices do this by assessing both productivity (how many papers did you get somebody to publish) and impact (how many people have cited your work). Thus it gets beyond the viewpoint of a few editors and reviewers and lets the field determine what is important. This is what the field does anyway. Most of the most highly cited articles are not to be found in the "top journals" of the field. See </span><span style="text-indent: -36px; font-size: 12px; ">Singh, G., K. M. Haddad, et al. (2007). "Are Articles in "Top" management Journals Necessarily of Higher Quality." </span><span style="text-indent: -36px; font-size: 12px; text-decoration: underline; ">Journal of Management Inquiry</span><span style="text-indent: -36px; font-size: 12px; "> </span><b style="text-indent: -36px; font-size: 12px; ">16</b><span style="text-indent: -36px; ">(4): 319-331 for the situation in management. They are also less subject to manipulation since once you get beyond the lowest levels, they are very difficult to manipulate by self-citation. They are also able to account for the influence of recent papers, big hit papers, numbers of co-authors, and number of publications.</span></div><div style="text-indent: 0px;"><span style="text-indent: -36px; "><br></span></div><div style="text-indent: 0px;"><span style="text-indent: -36px; ">See these papers for more discussion of influence and the hirsch indices.</span></div><div style="text-indent: 0px;"><span style="text-indent: -36px; "><br></span></div><div style="text-indent: 0px;"><div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; ">Truex III, D. P., M. J. Cuellar, et al. (2011). "The Scholarly Influence of Heinz Klein: Ideational and Social Measures of His Impact on IS Research and IS Scholars." <span style="text-decoration: underline">European Journal of Information Systems</span> <b>20</b>(4).</div><div><div style="margin: 0px 0px 0px 36px; text-indent: -36px; font-size: 12px; ">Truex III, D. P., M. J. Cuellar, et al. (2009). "Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse." <span style="text-decoration: underline">Journal of the Association of Information Systems</span> <b>10</b>(7): 560-594.</div></div><div><div style="margin: 0px; font-size: 12px; "><div style="margin: 0px 0px 0px 36px; text-indent: -36px; ">Takeda, H., D. P. Truex III, et al. (2012). "Evaluating Scholarly Influence Through Social Network Analysis: the Next Step in Evaluating Scholarly Influence." <span style="text-decoration: underline">The International Journal of Social and Organizational Dynamics in Information Technology</span> <b>2</b>(1).</div><div style="margin: 0px 0px 0px 36px; text-indent: -36px; "><br></div><div style="margin: 0px 0px 0px 36px; text-indent: -36px; ">Best Regards,</div><div style="margin: 0px 0px 0px 36px; text-indent: -36px; "><br></div><div style="margin: 0px 0px 0px 36px; text-indent: -36px; ">Mike Cuellar</div><div><br></div></div></div></div><div><div><div>On Sep 23, 2013, at 6:58 PM, John Lamp <<a href="mailto:john.lamp@deakin.edu.au">john.lamp@deakin.edu.au</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div bgcolor="white" lang="EN-AU" link="blue" vlink="purple" style="font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="WordSection1" style="page: WordSection1; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I don’t know whether this attachment is too big (<1Mb) but the Australian Government recently published a discussion paper on the wider definition of research impact. What will come of this following the change of government is moot.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Essentially it looks beyond internal impact (within the originating academic discipline) to suggest three dimensions of external impact:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -18pt; "><span style="font-size: 11pt; font-family: Symbol; color: rgb(31, 73, 125); "><span>·<span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-family: 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Academic impact (on researchers outside the originating discipline)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -18pt; "><span style="font-size: 11pt; font-family: Symbol; color: rgb(31, 73, 125); "><span>·<span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-family: 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Professional Impact (on practitioners)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -18pt; "><span style="font-size: 11pt; font-family: Symbol; color: rgb(31, 73, 125); "><span>·<span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-family: 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Social impact (broadly defined.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">If the attachment is stripped, email me directly for a copy.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-family: Calibri, sans-serif; color: rgb(31, 78, 121); ">Cheers<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-family: Calibri, sans-serif; color: rgb(31, 78, 121); ">John<o:p></o:p></span></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div><div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: windowtext; ">From:</span></b><span lang="EN-US" style="font-size: 11pt; font-family: Calibri, sans-serif; color: windowtext; "><span class="Apple-converted-space"> </span><a href="mailto:aisworld-bounces@lists.aisnet.org" style="color: purple; text-decoration: underline; ">aisworld-bounces@lists.aisnet.org</a><span class="Apple-converted-space"> </span>[mailto:aisworld-<a href="mailto:bounces@lists.aisnet.org" style="color: purple; text-decoration: underline; ">bounces@lists.aisnet.org</a>]<span class="Apple-converted-space"> </span><b>On Behalf Of<span class="Apple-converted-space"> </span></b>Ilia Bider<br><b>Sent:</b><span class="Apple-converted-space"> </span>Monday, 23 September 2013 5:16 PM<br><b>To:</b><span class="Apple-converted-space"> </span><a href="mailto:aisworld@lists.aisnet.org" style="color: purple; text-decoration: underline; ">aisworld@lists.aisnet.org</a><br><b>Subject:</b><span class="Apple-converted-space"> </span>Re: [AISWorld] Downside of impact factors: Scientists engaging in 'citation stacking'<o:p></o:p></span></div></div></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><o:p> </o:p></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hossein,<br><br>Sadly enough :-(, no academic measurement, including the Impact Factor, is directed at measuring the impact of research on real life, only its influence on the academic life. Publishing in a highest ranking journal does not guarantee that a paper will be picked up by the industry, or other real life sector for implementation. Part of the problem here is the academic papers being written in such a style that a "normal" person, most probably, does not understand. What's more, IMHO, the high ranking journals are among those that promote this style in no lesser degree than the less ranking journals. On the positive side, this problem is started to be understood, and, hopefully, the situation will change in the nearest future :-).<br><br>Hope, my message will take away some of your frustration :-).<br><br>Best regards/Ilia<br><br>On 2013-09-23 04:38, Hossein Seif Zadeh wrote:<o:p></o:p></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Not currently employed as an academic, I have just prepared a fresh jar of coffee and I am embracing for an "interesting" philosophical debate without any tangible, actionable, outcome; but alas here we go again... <o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Just to share one personal experience; I successfully completed a research project a few years ago and I received a letter of commendation (on federal government letterhead) specifically saying my research was of "national importance" and "impact". Later that year when I sought feedback on whether I should include the letter in my upcoming promotion pack, I was told it would contribute "zilch" as it was not peer-reviewed, not a recognized publication, and it was "just one letter by one person"!! Oh, well... Chalked it up as a lesson learnt; original research of national importance has no place in today's academia. Is this really the message we want to convey to junior, aspiring, academics?<o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Coffee mug in hand, feet up, ready for the onslaught of emails.... on second thought, I might need a glass of red instead of coffee :-) Should be an interesting 48 hours or or so before the debate winds down.<o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Cheers,<o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Hossein <o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br>Sent from my iPhone<o:p></o:p></div></div><div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br>On 23/09/2013, at 3:22 AM, "Kappelman, Leon" <<a href="mailto:Leon.Kappelman@unt.edu" style="color: purple; text-decoration: underline; ">Leon.Kappelman@unt.edu</a>> wrote:<br><br><br><o:p></o:p></div></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Some of our institutions have adopted or are considering the use of impact factors as a metric for faculty performance. Certainly our current faculty performance measures are less than perfect so it seems to be a suggestion worthy of consideration. But impact factors are not without controversy and apparently prone to manipulation and questionable behaviors. So before embracing impact factors, consider that at least to some extent they are a measure of what some might call “incestuous citation behaviors.” Not surprising since most all of us understand that when it comes to human behavior, you get what you measure. The big question is: Do the positives of using impact factors, or any other metric or combination of measure for that matter, sufficiently outweigh the negatives? <o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Here is some of the “food for thought” that raised my eyebrows enough to send this note:<o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "><a href="http://www.nature.com/news/brazilian-citation-scheme-outed-1.13604" style="color: purple; text-decoration: underline; ">http://www.nature.com/news/brazilian-citation-scheme-outed-1.13604</a><o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "> <o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "><a href="http://occamstypewriter.org/scurry/2012/08/13/sick-of-impact-factors/" style="color: purple; text-decoration: underline; ">http://occamstypewriter.org/scurry/2012/08/13/sick-of-impact-factors/</a><o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "> <o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "><a href="http://retractionwatch.wordpress.com/2013/08/30/journal-retracts-two-papers-after-being-caught-manipulating-citations/" style="color: purple; text-decoration: underline; ">http://retractionwatch.wordpress.com/2013/08/30/journal-retracts-two-papers-after-being-caught-manipulating-citations/</a><o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "> <o:p></o:p></p><p class="MsoPlainText" style="margin-right: 0cm; margin-left: 36pt; font-size: 12pt; font-family: 'Times New Roman', serif; margin-bottom: 0.0001pt; "><a href="http://www.naturalnews.com/042152_citation_stacking_scientific_journals_dishonesty.html" style="color: purple; text-decoration: underline; ">http://www.naturalnews.com/042152_citation_stacking_scientific_journals_dishonesty.html</a><o:p></o:p></p><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div><p class="MsoNormal" style="margin: 0cm 0cm 6pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">Best wishes,<br>Leon Kappelman<o:p></o:p></p><p class="MsoNormal" style="margin: 6pt 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><b>“Only a virtuous people are capable of freedom.” – Benjamin Franklin</b><o:p></o:p></p><p class="MsoNormal" style="margin: 6pt 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">----------------------------------------------------------------------------------------------------------------------------------------------<br>Leon A. Kappelman, Ph.D. <br> Professor of Information Systems <br> Director Emeritus, Information Systems Research Center<br> Fellow, Texas Center for Digital Knowledge<br> College of Business, University of North Texas<br> Voice: <a href="tel:940-565-4698" style="color: purple; text-decoration: underline; ">940-565-4698</a> Email:<a href="mailto:kapp@unt.edu" style="color: purple; text-decoration: underline; ">kapp@unt.edu</a><br>Founding Chair, Society for Information Management's Enterprise Architecture Working Group<br>----------------------------------------------------------------------------------------------------------------------------------------------<br>Please consider the environment before printing this e-mail <o:p></o:p></p><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "> <o:p></o:p></div></blockquote><blockquote style="margin-top: 5pt; margin-bottom: 5pt; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">_______________________________________________<br>AISWorld mailing list<br><a href="mailto:AISWorld@lists.aisnet.org" style="color: purple; text-decoration: underline; ">AISWorld@lists.aisnet.org</a><o:p></o:p></div></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><br><o:p></o:p></div><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">_______________________________________________<o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">AISWorld mailing list<o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; "><a href="mailto:AISWorld@lists.aisnet.org" style="color: purple; text-decoration: underline; ">AISWorld@lists.aisnet.org</a><o:p></o:p></pre></blockquote><div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; "><br><br><br><o:p></o:p></div><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">-- <o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">===============================================<o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">Dr. Ilia Bider <o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">Process- och systemutvecklingskonsult at <a href="http://ibissoft.se">ibissoft.se</a><o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">Lektor & Forskare at <a href="http://DSV.su.se">DSV.su.se</a><o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; "><a href="mailto:ilia@ibissoft.se" style="color: purple; text-decoration: underline; ">ilia@ibissoft.se</a> +46 (0)8 164998<o:p></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: 'Courier New'; ">Design science in action ... <a href="http://slidesha.re/Uq3RTC" style="color: purple; text-decoration: underline; ">http://slidesha.re/Uq3RTC</a><o:p></o:p></pre></div><span style="font-size: 9pt; font-family: Calibri; "><em><strong><br>Important Notice:</strong><span class="Apple-converted-space"> </span>The contents of this email are intended solely for the named addressee and are confidential; any unauthorised use, reproduction or storage of the contents is expressly prohibited. If you have received this email in error, please delete it and any attachments immediately and advise the sender by return email or telephone.<br><br>Deakin University does not warrant that this email and any attachments are error or virus free.</em></span><span class="Apple-converted-space"> </span><span><Research Impact Discussion Paper.pdf></span>_______________________________________________<br>AISWorld mailing list<br><a href="mailto:AISWorld@lists.aisnet.org" style="color: purple; text-decoration: underline; ">AISWorld@lists.aisnet.org</a></div></blockquote></div><br></div></body></html>