Quarterly Journal of Political Science > Vol 3 > Issue 3

Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals

Alan Gerber, ISPS, Yale University, USA, alan.gerber@yale.edu Neil Malhotra, Graduate School of Business, Stanford University, USA, neilm@stanford.edu
 
Suggested Citation
Alan Gerber and Neil Malhotra (2008), "Do Statistical Reporting Standards Affect What Is Published? Publication Bias in Two Leading Political Science Journals", Quarterly Journal of Political Science: Vol. 3: No. 3, pp 313-326. http://dx.doi.org/10.1561/100.00008024

Published: 24 Oct 2008
© 2008 A. Gerber and N. Malhotra
 
Subjects
 

Article Help

Share

Download article
In this article:
Method and Data
Results
Discussion
Appendix: Procedures for Collecting Studies from the APSR and the AJPS
References

Abstract

We examine the APSR and the AJPS for the presence of publication bias due to reliance on the 0.05 significance level. Our analysis employs a broad interpretation of publication bias, which we define as the outcome that occurs when, for whatever reason, publication practices lead to bias in the published parameter estimates. We examine the effect of the 0.05 significance level on the pattern of published findings using a "caliper" test, a novel method for comparing studies with heterogeneous effects, and find that we can reject the hypothesis of no publication bias at the 1 in 32 billion level. Our findings therefore raise the possibility that the results reported in the leading political science journals may be misleading due to publication bias. We also discuss some of the reasons for publication bias and propose reforms to reduce its impact on research.

DOI:10.1561/100.00008024

Replication Data | 100.00008024_supp.rar

This file contains the data that is required to replicate the data on your own system.

DOI: 10.1561/100.00008024_supp