What Works Clearinghouse
Description
Large database of education effect sizes created by the Institute of Education Sciences (Institute of Education Sciences 2025).
Snapshot date: 17 May 2025
Data availability: Flat files are available at https://ies.ed.gov/ncee/wwc/studyfindings. The Permissions and Disclaimers section of that website states: “Unless stated otherwise, all information on the U.S. Department of Education’s IES website … is in the public domain and may be reproduced, published, linked to, or otherwise used without permission from IES.”
Data collection: The WWC source data have 13,054 findings (data rows) across 1,908 reviews. Each row is one finding, so a single source paper can contribute multiple rows. The dataset includes effect sizes, but not standard errors, as well as p-values reported by the studies and p-values re-calculated by the WWC reviewers. After BEAR processing and filtering, the main BEAR dataset includes 7,080 findings.
Data processing: We include only RCTs and quasi-experimental studies (98% of data altogether). Majority of these (about 80%) are RCTs.
We removed a few rows where the results were flagged as not meeting WWC standards or as ineligible for review. We also removed rows marked by WWC as subgroup measures to avoid treating subgroup analyses as independent focal findings. Where available, we preferred effect sizes and p-values calculated by WWC to the ones reported by the studies. We replaced p-values that were equal to zero (this occurred in 3% of the dataset) with a truncated p-value (i.e. assumed \(p < 10^{-16}\)). We calculated z-values from these p-values under the assumption of two-sided tests. We then applied the sign of the reported effect size, so z-values are signed where the effect size is non-zero.
Study characteristics:
- Study ID: we use citation-derived author and year; we use the original
WWCStudyas fall-back in some cases. - Topic: WWC dataset includes some useful topic classifications, but these are not included in
BEAR.rdsas they are either too granular or are overlapping flags for study categories
Model of z-values
| Characteristic | Estimate |
|---|---|
| Probability of significance | 34% |
| Relative probability of publication for |z| < 1.96 | 0.92 |
| Successful replication for |z| > 1.96 | 70% |
| Correct sign for |z| > 1.96 | 97% |
What do these terms mean?
- Probability of significance
- The reported value is the assurance: the proportion of significant results adjusted for publication bias.
- Relative probability of publication
- The relative probability of observing a result below the |z| = 1.96 threshold rather than above it. Values below one indicate lower observation probability below the conventional two-sided significance threshold.
- Successful replication
- The probability that an exact replication has the same sign and a |z| greater than 1.96, conditional on the original result having |z| greater than 1.96.
- Correct sign
- The probability that the observed effect has the same direction as the true effect, conditional on an original result with |z| greater than 1.96.
