[AISWorld] On top journals with low acceptance rate and value delivering to IT professional community

Tue Jun 7 12:02:19 EDT 2022

Hi José and Michael,

In the first message, José compares IS research with medical research.

It called my attention because I have been looking into medical research, trying to establish the parallel with IS.

In medical research there is a well established distinction among 3 types of research: basic/blue skies research; translational research; and clinical research: the first is driven by curiosity; the second aims at creating new “treatments"; the third aims at establishing the effectiveness of “treatments”.
Clinical research typically involves practitioners. They get involved in clinical research with their own patients or involving the patients of the hospital they work in. Thus, practitioners are also producers of research reports. Not mere consumers.
I would say that medical practitioners (ate least some) look into reports of translational research when looking for novel treatments for some disease they are interested in, and they look into reports of clinical research looking for evidence about the effectiveness of the existing/alternative treatments for a disease.
This is particularly true when practitioners follow evidence-based medicine practices.

In the case of IS we miss several of the components I mentioned above.
A lot of (most) IS research is basic/blue skies research.
DSR takes us close to translational research. However, a few differences can be mentioned. 

We seldom have clinical research.
We lack a systematic catalogue of diseases.
We lack a systematic catalogue of treatments, with all the elements we found in the leaflet of any medicament we get from a pharmacy.
Furthermore, evidence-based practice is far from being established.
I recognise that medicine is much better organised. OK, it is much mature. After all it is one of the oldest professions in the world… ;-)

But I believe that we can learn a lot from it.
Not just in the research facet. I also mean the way research is entangled with practice. And the way practitioners get involved in teaching.
And the way students get involved into practice, since early in their programs, and at the final stage of their training.
I understand that the differences between IS and medicine, prevent us from “importing” all the package of medicine.
But several things could be done. As soon as possible.

Regards,

João

> On 7 Jun 2022, at 01:45, JOSE MANUEL MORA TAVAREZ <jose.mora at edu.uaa.mx> wrote:
> 
> Prof. Ioannadis is a top cited researcher (over 400,000 citations). This next paper has over 11,000 citations:
> 
> Ioannidis JP. Why most published research findings are false. PLoS Med. 2005; 2(8):e124. PMID: 16060722
> 
> There are interesting replies on this top paper clarifying the previous strong assertion at:
> 
> Stroebe, W. (2016). Are most published social psychological findings false?. Journal of Experimental Social Psychology, 66, 134-144.
> "Ioannidis´ claim is valid only for one-shot studies without replication and with a low a priori probability that the tested hypothesis is true."
> 
> 
> Lydersen, S., & Langaas, M. (2021). What proportion of published research findings are false?. Tidsskrift for Den norske legeforening.
> "Ioannidis produced a model which was based on a number of assumptions in various study designs. The assumptions may appear realistic, but it is a weakness that they to not build on data."
> 
> 
> Diekmann, A. (2011). Are most published research findings false?. Jahrbücher für Nationalökonomie und Statistik, 231(5-6), 628-635.
> "This short note sketches the argument and explores under what conditions the assertion holds. The “positive predictive value” (PPV) is lower than 1/2 if the a priori probability of the truth of a hypothesis is low. However, computation of the PPV includes only significant results. If both significant and non-significant results are taken into account the “total error ratio” (TER) will not exceed 1/2 provided no extremely large publication bias is present. "
> 
> but most interesting recommendation from Ioannadis (2016) on what is useful research:
> 
> "Many of the features that make clinical research useful can be identified, including those relating to problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency" (Ioannadis, 2016; p.1).
> 
> From a practitioner perspective, many medical practitioners consult medical research journals to search for insights and clues and it is impossible at present in IS top research.
> 
> 
> 
> ------------------------------------------------------------
> Prof. Dr. José Manuel Mora Tavarez
> Depto. de Sistemas de Información
> Centro de Ciencias Básicas
> Universidad Autónoma de Aguascalientes
> Ave. Universidad 940
> Aguascalientes, AGS. México 20131
> Email: jose.mora at edu.uaa.mx
> <https://www.researchgate.net/profile/Manuel_Mora>
> ResearchGate Weblink<https://www.researchgate.net/profile/Manuel_Mora>
> <https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
> Scholar Google Weblink<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
> Linkedin Weblink<https://www.linkedin.com/in/manuel-mora-engd-37b03a1/>
> SCOPUS Weblink<https://www.scopus.com/authid/detail.uri?authorId=25823339800>
> ------------------------------------------------------------
> 
> 
> ________________________________
> From: Michael Myers <m.myers at auckland.ac.nz>
> Sent: Monday, June 6, 2022 4:57 PM
> To: AISWorld at lists.aisnet.org <aisworld at lists.aisnet.org>
> Cc: JOSE MANUEL MORA TAVAREZ <jose.mora at edu.uaa.mx>
> Subject: RE: On top journals with low acceptance rate and value delivering to IT profes
> sional community
> 
> 
> Most medical research is not very useful either. For example, some estimates suggest that only about 10% of drug trials are successful. For more info, see:
> 
> 
> 
> Citation: Ioannidis JPA (2016) Why Most Clinical Research Is Not Useful. PLoS Med 13(6): e1002049. https://doi.org/10.1371/journal.pmed.1002049<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1371%2Fjournal.pmed.1002049&data=05%7C01%7Cjose.mora%40edu.uaa.mx%7C9b9e6cc6409d4c28fa3408da48078d92%7Ce1e2e29221d64849b7104d47d9578ad0%7C0%7C0%7C637901494532715543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8VaAPO%2BtyfAC%2F8wVQfsMw5zYSEQfM9TZnUY4hK7TG8Q%3D&reserved=0>
> 
> 
> 
> But until you actually conduct some research, how can you find out?
> 
> 
> 
> 
> 
> From: AISWorld <aisworld-bounces at lists.aisnet.org> On Behalf Of JOSE MANUEL MORA TAVAREZ
> Sent: Tuesday, 7 June 2022 9:28 AM
> To: AISWorld at lists.aisnet.org
> Subject: [AISWorld] On top journals with low acceptance rate and value delivering to IT professional community
> 
> 
> 
> It is not expected that practitioners from any discipline were usual readers of scientific journals, but it should be ethically expected that scientific journals in any discipline produce knowledge that can be used - after adaptations or simplifications - in real interventions by practitioners, like medical research. Top MIS journals in the 70s and 80s achieved this mission but at present days, is it true? Multiple micro-problems are studied and selectively published (about 5% of acceptance) in top journals but I estimate the same percentage of 5% will be useful for practitioners in the next 5 years. Then, are we delivering value for the real population of interest in our discipline?
> 
> --------------------------------------------------------------------------
> Manuel Mora, EngD.
> Full-time Professor and Researcher Level C
> ACM Senior Member / SNI Member
> Department of Information Systems
> Autonomous University of Aguascalientes
> Ave. Universidad 940
> Aguascalientes, AGS
> Mexico, 20131