[AISWorld] Reminder: P-Hacking Study - Survey

Thu Apr 30 13:23:48 EDT 2020

Prof. Galletta,
Your concerns are really important on the "new additional rules" for publishing in JCRs indexed journals in our discipline. We should learn from the colleagues in Natural Sciences (Biology and related areas), where short research papers (4-6 pages with 15-20 references) and with supported and non-supported hypotheses are welcome and worthy to be published with review-correction-publishing times from 2-3 months. I mean, for them it is very important report if expectations can be supported (i.e. failed to reject null hypotheses) or not. Both results are welcome! On relevance, for them all units of study in any part of the Earth planet are welcome. In our area, if the study is not conducted with  important organizations in the world, the data are "poor". My record of JCRs is low (about 12 papers published) but I and other Senior colleagues are experimenting similar rejections automatically in the screening phase (maybe done by Machine Learning methods). DORA acta should be consulted in our discipline.

https://sfdora.org/
[https://sfdora.org/'https://sfdora.org/wp-content/uploads/2018/01/banner02.jpg']<https://sfdora.org/>
DORA – San Francisco Declaration on Research Assessment (DORA)<https://sfdora.org/>
Improving how research is assessed Join the organizations and individuals who have signed the Declaration on Research Assessment. Sign the declaration Read the full declaration » Follow us on twitter Latest news About The Declaration on Research Assessment (DORA) recognizes the need to improve the ways in which the outputs of scholarly research are evaluated.
sfdora.org

Thanks. Prof. Mora
SCOPUS Weblink<https://www.scopus.com/authid/detail.uri?authorId=25823339800>
Scholar Google Weblink<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>

------------------------------------------------------------
Prof. Dr. José Manuel Mora Tavarez
Depto. de Sistemas de Información
Centro de Ciencias Básicas
Universidad Autónoma de Aguascalientes
Ave. Universidad 940
Aguascalientes, AGS. México 20131
Email: jose.mora at edu.uaa.mx
<https://www.researchgate.net/profile/Manuel_Mora>
ResearchGate Weblink<https://www.researchgate.net/profile/Manuel_Mora>
<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
Scholar Google Weblink<https://scholar.google.com.mx/citations?user=97rTgbkAAAAJ&hl=en&oi=sra>
Linkedin Weblink<https://www.linkedin.com/in/manuel-mora-engd-37b03a1/>
SCOPUS Weblink<https://www.scopus.com/authid/detail.uri?authorId=25823339800>
------------------------------------------------------------

________________________________
From: AISWorld <aisworld-bounces at lists.aisnet.org> on behalf of Galletta, Dennis <GALLETTA at pitt.edu>
Sent: Thursday, April 30, 2020 8:51 AM
To: AISWorld at lists.aisnet.org <AISWorld at lists.aisnet.org>
Subject: [AISWorld] Reminder: P-Hacking Study - Survey

This is a reminder in case you were not able to respond earlier. Thanks to those several dozen who have responded (anonymously, of course)!

Dear Colleagues,

You might conclude that the "bar has been rising" on research over the years. Journal lists did not exist when I was hired, and now there is a fixed, tiered system in place at many schools. If a journal arrives on the "short list" such as the Financial Times 50, submission rates tend to increase while acceptance rates tend to decrease. Papers with many non-significant results are often unpublishable.

Along the way, some researchers have adopted practices that could raise cautions if they were disclosed. A study with six supported hypotheses and four unsupported hypotheses would look stronger if the latter four were dropped. Sometimes a review panel requests those to be dropped. Sometimes authors will omit one or more unsupported hypothesis, on their own, before submission. Sometimes researchers will find that omitting a few subjects will help reach p<.05. Maybe running through dozens of packages will, after a long weekend, result in significant findings. Some desperate researchers who are facing a difficult tenure process will answer their own questionnaire several times to help nudge significance higher. Finally, some researchers completely falsify all their data.

Notice the slippery slope in just these examples: from culling hypotheses as directed, to removing hypotheses on your own, to cutting a few data points, to fiddling with statistical packages until one test actually works, to entering a few falsified records, to fabricating entire data sets. As the amount of desperation increases, the researcher becomes nudged down the slope with increasing force.

This study aims to understand the attitudes, beliefs, and performance regarding p-hacking in our field. Of course, data will be collected with complete anonymity by my colleagues at University of Nevada Las Vegas, and it is approved by the UNLV Office of Research Integrity (their IRB).

The study was designed by Han-Fen Hu, Greg Moody, and myself.

We believe you will find the survey to be quite interesting and we look forward to sharing the results at the end. We will not alter the data in any way other than removing any empty or suspicious records. And we will disclose any omissions of those records!

Please navigate to https://unlv.co1.qualtrics.com/jfe/form/SV_0HxwqHho8jAXbYp to participate in the study.

DG

----------------------------------------------------------------------------
Dennis F. Galletta                      Professor of Business Administration
University of Pittsburgh                     Director, Katz Doctoral Program
282a Mervis Hall                                          AIS LEO and Fellow
Phone +1 412-648-1699                       Katz Graduate School of Business
Fax +1 412-624-3633                                    Pittsburgh, PA  15260
E-mail: galletta @ pitt.edu                 homepage: www.pitt.edu/~galletta<https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.pitt.edu%2F~galletta&data=01%7C01%7Cgalletta%40katz.pitt.edu%7C6c38ed63ff2044c895ff08d59e9c6746%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=mn%2FzK1thKeKZbxjEYBTPliXPQVEgsHl%2FsmrE3YU9V6g%3D&reserved=0<http://www.pitt.edu/~galletta<https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.pitt.edu%2F~galletta&data=01%7C01%7Cgalletta%40katz.pitt.edu%7C6c38ed63ff2044c895ff08d59e9c6746%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=mn%2FzK1thKeKZbxjEYBTPliXPQVEgsHl%2FsmrE3YU9V6g%3D&reserved=0>>
----------------------------------------------------------------------------

_______________________________________________
AISWorld mailing list
AISWorld at lists.aisnet.org