[AISWorld] Reminder: P-Hacking Study - Survey

[Galletta, Dennis]

This is a reminder in case you were not able to respond earlier. Thanks to those several dozen who have responded (anonymously, of course)!

Dear Colleagues,

You might conclude that the "bar has been rising" on research over the years. Journal lists did not exist when I was hired, and now there is a fixed, tiered system in place at many schools. If a journal arrives on the "short list" such as the Financial Times 50, submission rates tend to increase while acceptance rates tend to decrease. Papers with many non-significant results are often unpublishable.

Along the way, some researchers have adopted practices that could raise cautions if they were disclosed. A study with six supported hypotheses and four unsupported hypotheses would look stronger if the latter four were dropped. Sometimes a review panel requests those to be dropped. Sometimes authors will omit one or more unsupported hypothesis, on their own, before submission. Sometimes researchers will find that omitting a few subjects will help reach p<.05. Maybe running through dozens of packages will, after a long weekend, result in significant findings. Some desperate researchers who are facing a difficult tenure process will answer their own questionnaire several times to help nudge significance higher. Finally, some researchers completely falsify all their data.

Notice the slippery slope in just these examples: from culling hypotheses as directed, to removing hypotheses on your own, to cutting a few data points, to fiddling with statistical packages until one test actually works, to entering a few falsified records, to fabricating entire data sets. As the amount of desperation increases, the researcher becomes nudged down the slope with increasing force.

This study aims to understand the attitudes, beliefs, and performance regarding p-hacking in our field. Of course, data will be collected with complete anonymity by my colleagues at University of Nevada Las Vegas, and it is approved by the UNLV Office of Research Integrity (their IRB).

The study was designed by Han-Fen Hu, Greg Moody, and myself.

We believe you will find the survey to be quite interesting and we look forward to sharing the results at the end. We will not alter the data in any way other than removing any empty or suspicious records. And we will disclose any omissions of those records!

Please navigate to https://unlv.co1.qualtrics.com/jfe/form/SV_0HxwqHho8jAXbYp to participate in the study.

DG

----------------------------------------------------------------------------
Dennis F. Galletta                      Professor of Business Administration
University of Pittsburgh                     Director, Katz Doctoral Program
282a Mervis Hall                                          AIS LEO and Fellow
Phone +1 412-648-1699                       Katz Graduate School of Business
Fax +1 412-624-3633                                    Pittsburgh, PA  15260
E-mail: galletta @ pitt.edu                 homepage: www.pitt.edu/~galletta<https://na01.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.pitt.edu%2F~galletta&data=01%7C01%7Cgalletta%40katz.pitt.edu%7C6c38ed63ff2044c895ff08d59e9c6746%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=mn%2FzK1thKeKZbxjEYBTPliXPQVEgsHl%2FsmrE3YU9V6g%3D&reserved=0>
----------------------------------------------------------------------------