psymetadata and Nuijten et al
To download only this data file: psymetadata.rds (279 KB)
To download all BEAR datasets, click here.
Description
Reference: Set of psychology datasets compiled by Rodriguez and Williams (2022). We also disaggregate a large meta-meta-analysis of intelligence research (Nuijten et al. 2020).
Snapshot date: Jan 2026
Research question: psymetadata is an R package that curates multiple open datasets from published meta-analyses in psychology for teaching and demonstrations. Nuijten et al. (2020) is a large meta-meta-analysis of intelligence research, which focused on power and small study effects.
Data availability: 22 datasets are distributed with the psymetadata R package (CRAN). We do not use its Many Labs 2 dataset, which we obtain from the original source and include separately. After excluding Many Labs 2 and separately representing the intelligence meta-meta-analysis, the remaining psychology datasets vary from 65 rows of data to over 1,000. The package is GPL-3; individual datasets originate from the cited source papers.
Data processing: minimal. We used regular expressions to extract year labels from titles. We dropped some invalid rows (missing, non-finite, or non-positive standard errors). Signed z-values were computed as effect sizes divided by their standard errors.
Additional grouping variable: name of the original psymetadata dataset.
Model of z-values
This documentation page covers more than one fitted dataset, so the fitted models are shown separately.
| Characteristic | Estimate |
|---|---|
| Probability of significance | 32% |
| Relative probability of publication for |z| < 1.96 | 0.66 |
| Successful replication for |z| > 1.96 | 74% |
| Correct sign for |z| > 1.96 | 96% |
| Characteristic | Estimate |
|---|---|
| Probability of significance | 48% |
| Relative probability of publication for |z| < 1.96 | 0.85 |
| Successful replication for |z| > 1.96 | 75% |
| Correct sign for |z| > 1.96 | 99% |
What do these terms mean?
- Probability of significance
- The reported value is the assurance: the proportion of significant results adjusted for publication bias.
- Relative probability of publication
- The relative probability of observing a result below the |z| = 1.96 threshold rather than above it. Values below one indicate lower observation probability below the conventional two-sided significance threshold.
- Successful replication
- The probability that an exact replication has the same sign and |z| greater than 1.96, conditional on the original result having |z| greater than 1.96.
- Correct sign
- The probability that the observed effect has the same direction as the true effect, conditional on an original result with |z| greater than 1.96.
psymetadata: psychology
Nuijten et al: intelligence
