[AISWorld] On top journals with low acceptance rate and value delivering to IT professional community

Tue Jun 7 15:05:21 EDT 2022

Colleagues Joao, David, and Michael,

I estimate we have similar paths - everyone in our respective national context- gaining a BSc on Informatics in the early 80's, and about 60 years old. In my case, I arrived late to research (early 2000 year). However, my general concern about top research and top journals in Informatics (Management Information Systems) is their bias in the last decades to study micro-problems using sophisticated concepts far away of the mindset of the final IT users - CIOs, CTOs, CKMs, CISOs, managers, developers, and so on).  The analogy with the medical field or with systems engineering fields was done from a natural perception of studied topics, where the title and abstracts of most papers make sense for practitioners. For our discipline, when you review title and abstract of papers published in top journals in the 70s, 80s and yet 90s, and compare them with current ones, the natural relevance for practitioners is lost. It could be supported with an experimental research. Just one example from a top#1 journal:

Compeau, D. R., & Higgins, C. A. (1995). Application of social cognitive theory to training for computer skills. Information Systems Research, 6(2), 118-143.

This title attracts to practitioners without the need to have a PhD in MIS, because it includes natural IT agents (IT people demanding computer training) and a natural problem (how to improve computer skills), and a top theory ! (social cognitive theory). This paper has over 2,000 citations.

The key idea here is that in the current top research, the IT artifact and IT agents involved are not the important things to be studied in the research but else other complementary issues.

Well, cheers colleagues!
Manuel

------------------------------------------------------------
Prof. Dr. José Manuel Mora Tavarez
Depto. de Sistemas de Información
Centro de Ciencias Básicas
Universidad Autónoma de Aguascalientes
Ave. Universidad 940
Aguascalientes, AGS. México 20131
Email: jose.mora at edu.uaa.mx
<https://www.researchgate.net/profile/Manuel_Mora>
ResearchGate Weblink<https://www.researchgate.net/profile/Manuel_Mora>
<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
Scholar Google Weblink<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
Linkedin Weblink<https://www.linkedin.com/in/manuel-mora-engd-37b03a1/>
SCOPUS Weblink<https://www.scopus.com/authid/detail.uri?authorId=25823339800>
------------------------------------------------------------

________________________________
From: Prof. David G. Schwartz <David.Schwartz at biu.ac.il>
Sent: Tuesday, June 7, 2022 11:44 AM
To: João Alvaro Carvalho <jac at dsi.uminho.pt>
Cc: JOSE MANUEL MORA TAVAREZ <jose.mora at edu.uaa.mx>; Michael Myers <m.myers at auckland.ac.nz>; AISWorld at lists.aisnet.org <aisworld at lists.aisnet.org>
Subject: Re: [AISWorld] On top journals with low acceptance rate and value delivering to IT professional community

Dear João and all,

I have for years been advocating for the adoption of certain (positive) aspects of the medical discipline for our own research and education practices.
See: Schwartz, D. G. (2014). Research commentary—the disciplines of information: Lessons from the history of the discipline of medicine. Information Systems Research, 25(2), 205-221. https://doi.org/10.1287/isre.2014.0516<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1287%2Fisre.2014.0516&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170881921330%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XQ7dFgXBx3KizINZ1%2Bbgd1MnIUqp%2BFORRc8A7tLQiBg%3D&reserved=0>

In addition to the excellent points that you raise regarding three types of research, and the important synergies between hospital/clinic/patient/MD in the organizational context, the medical field also benefits from a clear definition of core knowledge (that which all MDs are required to study and know), and wide ranging specializations (which allow for growth of the field from the common core). This did not occur overnight and is the result of a 100+ year process of unification and specialization which had tremendous disciplinary benefits.

Medical specialization was the result of a number of motivators and environmental factors. It was the pragmatic outgrowth of the evolution of the field and the impacts of environment and technology. One might even be tempted to say that it was a sociotechnical result. But it was also the result of a conceptual leap taken by John Morgan, founder of university medical education in the United States, in his discourse of 1765 (Gelfand 1976). It would take almost 100 years for the ideas crystallized by Morgan to significantly alter the medical landscape. We don’t have that long. Despite some clear differences between the discipline of medicine and the Disciplines of Information (DI) at scientific, academic, and practical levels, there remains a strong and clear parallel between the factors that led to today’s conceptualization of the medical field. Following Freidson (1988) the identification and analysis of these factors makes it clear that we have much to learn from the history of medicine. Much work remains to be done in characterizing what will become the core and specialties of DI; what institutional ecology will result; and how specialization would be determined, accredited, and maintained.

Perhaps the time has come for a working committee to systematically try and determine what aspects of the medical disciplinary structure we would be wise to adopt.

Refs:
Freidson E (1988) Profession of Medicine: A Study of the Sociology of Applied Knowledge (University of Chicago Press, Chicago).
Gelfand T (1976) The origins of a modern concept of medical specialization: John Morgan’s Discourse of 1765. Bull. Hist. Medicine 50(4):511–535.

--David
-----------------------------------------------
David G. Schwartz
Professor of Information Systems
Graduate School of Business
Bar-Ilan University, Israel

[https://ci3.googleusercontent.com/mail-sig/AIorK4wvUhZ2TTtz6UbjonmKYE0vNDTUXOnZ5TuVTuLozp2HFsEo04NKvKjiojId_2_cyCH3524Ha7o]

On Tue, Jun 7, 2022 at 7:07 PM João Alvaro Carvalho <jac at dsi.uminho.pt<mailto:jac at dsi.uminho.pt>> wrote:
Hi José and Michael,

In the first message, José compares IS research with medical research.

It called my attention because I have been looking into medical research, trying to establish the parallel with IS.

In medical research there is a well established distinction among 3 types of research: basic/blue skies research; translational research; and clinical research: the first is driven by curiosity; the second aims at creating new “treatments"; the third aims at establishing the effectiveness of “treatments”.
Clinical research typically involves practitioners. They get involved in clinical research with their own patients or involving the patients of the hospital they work in. Thus, practitioners are also producers of research reports. Not mere consumers.
I would say that medical practitioners (ate least some) look into reports of translational research when looking for novel treatments for some disease they are interested in, and they look into reports of clinical research looking for evidence about the effectiveness of the existing/alternative treatments for a disease.
This is particularly true when practitioners follow evidence-based medicine practices.

In the case of IS we miss several of the components I mentioned above.
A lot of (most) IS research is basic/blue skies research.
DSR takes us close to translational research. However, a few differences can be mentioned.

We seldom have clinical research.
We lack a systematic catalogue of diseases.
We lack a systematic catalogue of treatments, with all the elements we found in the leaflet of any medicament we get from a pharmacy.
Furthermore, evidence-based practice is far from being established.
I recognise that medicine is much better organised. OK, it is much mature. After all it is one of the oldest professions in the world… ;-)

But I believe that we can learn a lot from it.
Not just in the research facet. I also mean the way research is entangled with practice. And the way practitioners get involved in teaching.
And the way students get involved into practice, since early in their programs, and at the final stage of their training.
I understand that the differences between IS and medicine, prevent us from “importing” all the package of medicine.
But several things could be done. As soon as possible.

Regards,

João

> On 7 Jun 2022, at 01:45, JOSE MANUEL MORA TAVAREZ <jose.mora at edu.uaa.mx<mailto:jose.mora at edu.uaa.mx>> wrote:
>
> Prof. Ioannadis is a top cited researcher (over 400,000 citations). This next paper has over 11,000 citations:
>
> Ioannidis JP. Why most published research findings are false. PLoS Med. 2005; 2(8):e124. PMID: 16060722
>
> There are interesting replies on this top paper clarifying the previous strong assertion at:
>
> Stroebe, W. (2016). Are most published social psychological findings false?. Journal of Experimental Social Psychology, 66, 134-144.
> "Ioannidis´ claim is valid only for one-shot studies without replication and with a low a priori probability that the tested hypothesis is true."
>
>
> Lydersen, S., & Langaas, M. (2021). What proportion of published research findings are false?. Tidsskrift for Den norske legeforening.
> "Ioannidis produced a model which was based on a number of assumptions in various study designs. The assumptions may appear realistic, but it is a weakness that they to not build on data."
>
>
> Diekmann, A. (2011). Are most published research findings false?. Jahrbücher für Nationalökonomie und Statistik, 231(5-6), 628-635.
> "This short note sketches the argument and explores under what conditions the assertion holds. The “positive predictive value” (PPV) is lower than 1/2 if the a priori probability of the truth of a hypothesis is low. However, computation of the PPV includes only significant results. If both significant and non-significant results are taken into account the “total error ratio” (TER) will not exceed 1/2 provided no extremely large publication bias is present. "
>
> but most interesting recommendation from Ioannadis (2016) on what is useful research:
>
> "Many of the features that make clinical research useful can be identified, including those relating to problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency" (Ioannadis, 2016; p.1).
>
> From a practitioner perspective, many medical practitioners consult medical research journals to search for insights and clues and it is impossible at present in IS top research.
>
>
>
> ------------------------------------------------------------
> Prof. Dr. José Manuel Mora Tavarez
> Depto. de Sistemas de Información
> Centro de Ciencias Básicas
> Universidad Autónoma de Aguascalientes
> Ave. Universidad 940
> Aguascalientes, AGS. México 20131
> Email: jose.mora at edu.uaa.mx<mailto:jose.mora at edu.uaa.mx>
> <https://www.researchgate.net/profile/Manuel_Mora<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FManuel_Mora&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170881921330%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KiUIuD1rxVWqSAAV%2FBOFLGqFC6nslh2%2BWlcJPj6E3iw%3D&reserved=0>>
> ResearchGate Weblink<https://www.researchgate.net/profile/Manuel_Mora<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.researchgate.net%2Fprofile%2FManuel_Mora&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170881921330%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KiUIuD1rxVWqSAAV%2FBOFLGqFC6nslh2%2BWlcJPj6E3iw%3D&reserved=0>>
> <https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fscholar.google.com.mx%2Fcitations%3Fuser%3D97rTgbkAAAAJ%26hl%3Den%26oi%3Dsra&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nE%2FLqhe%2Fq%2BNPWQgcpA4Ic3EHjj8ODzZVodcQ74R2PWY%3D&reserved=0>>
> Scholar Google Weblink<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fscholar.google.com.mx%2Fcitations%3Fuser%3D97rTgbkAAAAJ%26hl%3Den%26oi%3Dsra&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nE%2FLqhe%2Fq%2BNPWQgcpA4Ic3EHjj8ODzZVodcQ74R2PWY%3D&reserved=0>>
> Linkedin Weblink<https://www.linkedin.com/in/manuel-mora-engd-37b03a1/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fmanuel-mora-engd-37b03a1%2F&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=P7szpdugwTPyxFHDiJZ3kvzdEq7oKcrH999yB7YdTA0%3D&reserved=0>>
> SCOPUS Weblink<https://www.scopus.com/authid/detail.uri?authorId=25823339800<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.scopus.com%2Fauthid%2Fdetail.uri%3FauthorId%3D25823339800&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tYNN%2FA%2FHZ%2B4654eBRNiimA3lfz3i6UiagSXhvFa%2FehE%3D&reserved=0>>
> ------------------------------------------------------------
>
>
> ________________________________
> From: Michael Myers <m.myers at auckland.ac.nz<mailto:m.myers at auckland.ac.nz>>
> Sent: Monday, June 6, 2022 4:57 PM
> To: AISWorld at lists.aisnet.org<mailto:AISWorld at lists.aisnet.org> <aisworld at lists.aisnet.org<mailto:aisworld at lists.aisnet.org>>
> Cc: JOSE MANUEL MORA TAVAREZ <jose.mora at edu.uaa.mx<mailto:jose.mora at edu.uaa.mx>>
> Subject: RE: On top journals with low acceptance rate and value delivering to IT profes
> sional community
>
>
> Most medical research is not very useful either. For example, some estimates suggest that only about 10% of drug trials are successful. For more info, see:
>
>
>
> Citation: Ioannidis JPA (2016) Why Most Clinical Research Is Not Useful. PLoS Med 13(6): e1002049. https://doi.org/10.1371/journal.pmed.1002049<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pmed.1002049&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6GFkm6ikTq2PxgSoPTAQc7lXdfviCyplqakAlN9byGY%3D&reserved=0><https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pmed.1002049&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7C9b9e6cc6409d4c28fa3408da48078d92%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637901494532715543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8VaAPO%2BtyfAC%2F8wVQfsMw5zYSEQfM9TZnUY4hK7TG8Q%3D&reserved=0<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pmed.1002049&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7Cd736c77ab61542604e4508da48a507c6%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637902170882077554%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6GFkm6ikTq2PxgSoPTAQc7lXdfviCyplqakAlN9byGY%3D&reserved=0>>
>
>
>
> But until you actually conduct some research, how can you find out?
>
>
>
>
>
> From: AISWorld <aisworld-bounces at lists.aisnet.org<mailto:aisworld-bounces at lists.aisnet.org>> On Behalf Of JOSE MANUEL MORA TAVAREZ
> Sent: Tuesday, 7 June 2022 9:28 AM
> To: AISWorld at lists.aisnet.org<mailto:AISWorld at lists.aisnet.org>
> Subject: [AISWorld] On top journals with low acceptance rate and value delivering to IT professional community
>
>
>
> It is not expected that practitioners from any discipline were usual readers of scientific journals, but it should be ethically expected that scientific journals in any discipline produce knowledge that can be used - after adaptations or simplifications - in real interventions by practitioners, like medical research. Top MIS journals in the 70s and 80s achieved this mission but at present days, is it true? Multiple micro-problems are studied and selectively published (about 5% of acceptance) in top journals but I estimate the same percentage of 5% will be useful for practitioners in the next 5 years. Then, are we delivering value for the real population of interest in our discipline?
>
> --------------------------------------------------------------------------
> Manuel Mora, EngD.
> Full-time Professor and Researcher Level C
> ACM Senior Member / SNI Member
> Department of Information Systems
> Autonomous University of Aguascalientes
> Ave. Universidad 940
> Aguascalientes, AGS
> Mexico, 20131

_______________________________________________
AISWorld mailing list
AISWorld at lists.aisnet.org<mailto:AISWorld at lists.aisnet.org>