What Works Clearinghouse
Description
Large database of education effect sizes created by Institute of Education Sciences (Institute of Education Sciences 2025).
Snapshot date: 17 May 2025
Data availability: flat files are available at https://ies.ed.gov/ncee/wwc/studyfindings. Per permissions and disclaimers section on that website: “Unless stated otherwise, all information on the U.S. Department of Education’s IES website … is in the public domain and may be reproduced, published, linked to, or otherwise used without permission from IES.”
Data collection: The WWC source data have 13,054 findings (data rows) across 1,908 reviews. Each row is one finding, so a single source paper can contribute multiple rows. The dataset includes effect sizes, but not standard errors, as well as p-values reported by the studies and p-values re-calculated by the WWC reviewers. After BEAR processing and filtering, the main BEAR dataset retains 7,080 findings.
Data processing: We retained only RCTs and quasi-experimental studies (98% of data altogether). We removed a few rows where the results were flagged as not meeting WWC standards or as ineligible for review. We also removed rows marked by WWC as subgroup measures to avoid treating subgroup analyses as independent focal findings. Where available, we preferred effect sizes and p-values calculated by WWC to the ones reported by the studies. We replaced p-values that were equal to zero (this occurred in 3% of the dataset) with truncated p-value (i.e. assumed \(p < 10^{-16}\)). We calculated z-values from these p-values under assumption of two-sided tests. We then applied the sign of the reported effect size, so z-values are signed where the effect size is non-zero.
Additional grouping variable: BEAR uses a citation-derived author-year identifier as studyid, so findings from the same source paper share a study ID. Outcome domain is retained as subset.
Model of z-values
| Characteristic | Estimate |
|---|---|
| Probability of significance | 34% |
| Relative probability of publication for |z| < 1.96 | 0.92 |
| Successful replication for |z| > 1.96 | 70% |
| Correct sign for |z| > 1.96 | 97% |
What do these terms mean?
- Probability of significance
- The reported value is the assurance: the proportion of significant results adjusted for publication bias.
- Relative probability of publication
- The relative probability of observing a result below the |z| = 1.96 threshold rather than above it. Values below one indicate lower observation probability below the conventional two-sided significance threshold.
- Successful replication
- The probability that an exact replication has the same sign and |z| greater than 1.96, conditional on the original result having |z| greater than 1.96.
- Correct sign
- The probability that the observed effect has the same direction as the true effect, conditional on an original result with |z| greater than 1.96.
