Sladekova et al. 2023

Tags:meta analysismetascience paperHover over the tag text for details.

Domainpsychology

Data3.5k studies in 406 meta-analyses

Note

To download only this data file: Sladekova.rds (229 KB)

To download all BEAR datasets, click here.

Description

Reference: Sladekova et al. (2023).

Research question: how much do meta-analytic effect-size estimates change after applying publication-bias adjustment methods in psychology datasets?

Data availability: Data and materials are available through OSF.

Data description and source: The authors reanalysed 433 meta-analytic datasets from 90 psychology papers. BEAR represents the imported meta-analytic datasets as row-level effect sizes, with approximate study groupings.

Data processing: We use rows with available effect estimates and sampling variances. Effect sizes on the correlation scale are transformed to Fisher’s z scale, with standard errors derived from the reported variances. Signed z-values were computed as transformed effect sizes divided by their standard errors.

Study characteristics:

Study ID: we use DOI where available, then a descriptive label and year, then the label, and finally a row-specific ID. Overall labelling of studies in this dataset is problematic.
Meta-analysis ID: we use the source filename to distinguish meta-analyses.
The designs of the primary studies are not classified.

Model of z-values

Characteristic	Estimate
Probability of significance	54%
Relative probability of publication for \|z\| < 1.96	0.82
Successful replication for \|z\| > 1.96	83%
Correct sign for \|z\| > 1.96	99%

What do these terms mean?

Probability of significance: The reported value is the assurance: the proportion of significant results adjusted for publication bias.
Relative probability of publication: The relative probability of observing a result below the |z| = 1.96 threshold rather than above it. Values below one indicate lower observation probability below the conventional two-sided significance threshold.
Successful replication: The probability that an exact replication has the same sign and a |z| greater than 1.96, conditional on the original result having |z| greater than 1.96.
Correct sign: The probability that the observed effect has the same direction as the true effect, conditional on an original result with |z| greater than 1.96.

Sladekova et al: psychology mixture model plot

References

Sladekova, Martina, Lois E A Webb, and Andy P Field. 2023. “Estimating the Change in Meta-Analytic Effect Size Estimates After the Application of Publication Bias Adjustment Methods.” Psychological Methods 28 (3): 664.