Costello and Fox 2022; Yang et al 2023, 2024
To download only this data file: CostelloFox.rds (2.3 MB); Yang.rds (255 KB)
To download all BEAR datasets, click here.
Description
Reference: these two datasets come from the same replication package and shared data processing pipeline in Yang et al. 2024 (Yang et al. 2024). Original references for the datasets are Yang et al. 2023 and Costello and Fox 2022 (Yang et al. 2023; Costello and Fox 2022).
Research question: decline effects in ecology (Costello and Fox) and publication bias and performance of empirical research (Yang et al)
Data collection: Costello and Fox look at 466 meta-analyses obtained through searching Web of Science. Reproducibility dataset includes 88,000 rows among 466 analyses in 232 papers. Subsequently, Yang et al removed duplicated studies and zeroes, which meant discarding about 20% of rows. Yang et al collect 102 meta-analyses published 2010-2019 in relevant journals, de-duplicated against Costello and Fox dataset.
Data availability: two datasets may be downloaded from https://github.com/Yefeng0920/replication_EcoEvo_git/. Available under CC BY 4.0 licence.
Data processing: no additional data processing other than cleaning up study years. Signed z-values were computed as effect sizes divided by their standard errors. Note that the replication package for Yang et al does not include meta-analysis indicators.
Additional variables used: we retained a measure variable (“lnRR”, “SMD”, “Zr” or “uncommon”)
Model of z-values
This documentation page covers more than one fitted dataset, so the fitted models are shown separately.
| Characteristic | Estimate |
|---|---|
| Probability of significance | 39% |
| Relative probability of publication for |z| < 1.96 | 0.74 |
| Successful replication for |z| > 1.96 | 74% |
| Correct sign for |z| > 1.96 | 98% |
| Characteristic | Estimate |
|---|---|
| Probability of significance | 37% |
| Relative probability of publication for |z| < 1.96 | 0.73 |
| Successful replication for |z| > 1.96 | 73% |
| Correct sign for |z| > 1.96 | 98% |
What do these terms mean?
- Probability of significance
- The reported value is the assurance: the proportion of significant results adjusted for publication bias.
- Relative probability of publication
- The relative probability of observing a result below the |z| = 1.96 threshold rather than above it. Values below one indicate lower observation probability below the conventional two-sided significance threshold.
- Successful replication
- The probability that an exact replication has the same sign and |z| greater than 1.96, conditional on the original result having |z| greater than 1.96.
- Correct sign
- The probability that the observed effect has the same direction as the true effect, conditional on an original result with |z| greater than 1.96.
Costello and Fox: ecology & evolution
Yang et al: ecology & evolution
