Arel-Bundock et al 2022 (political science)

Note

To download only this data file: ArelBundock.rds (368 KB)

To download all BEAR datasets, click here.

Arel-Bundock et al 2022 (political science)

Reference: Arel-Bundock et al. (2022).

Research question: assess statistical power in political science research

Data availability: replication package (zip file with date 20241010) can be found at https://osf.io/fgdet. All data are in public domain.

Data description and source: Authors searched for, and contacted authors of, meta-analysis articles across 141 journals in political science. The resulting source dataset contains 16,649 hypothesis tests, grouped in 351 meta-analyses and reported in 46 peer-reviewed meta-analytic articles.

Notes: This is a collection of meta-analysis inputs rather than a direct extraction of all results from primary articles. The source combines a political-science search with an earlier Doucouliagos work, meaning that some of this dataset will overlap with Askarov et al. However, the two articles cover different fields and therefore BEAR keeps the two datasets separate.

Data processing: using the same code as original authors (replication package that runs on makefiles), but skipping some additional processing that they’ve done for their paper. BEAR uses question_id as metaid (351 meta-analysis/research question groups) Signed z-values were computed as reported estimates divided by their standard errors. We verified that the distributions of z-values matched with the article.

Additional variables used: BEAR release includes meta-analytic article (46 papers). The dataset also includes sample field records whether a row comes from the Doucouliagos data collection or the later Briggs/Arel-Bundock data collection.

Model of z-values

The fitted mixture model is shown over the empirical distribution of absolute z-values. The solid line is a mixture of half-normals, with selection. The dashed line shows the distribution without selection. If there are inequalities (e.g. studies reporting p < 0.05) the histogram resamples values from the appropriate set.

Arel-Bundock et al: political science mixture model plot

References

Arel-Bundock, Vincent, Ryan C. Briggs, Hristos Doucouliagos, Marco Mendoza Aviña, and Tom D. Stanley. 2022. “Quantitative Political Science Research Is Greatly Underpowered.” I4R Discussion Paper Series, no. 6.