Randomized controlled trials (RCTs) are widely regarded as the most credible method of evaluating social programs, but these studies can cost millions of dollars and are often considered to be too expensive for routine use. We are advancing a model that makes it possible for researchers to conduct large-scale RCTs over multiple years at a fraction of the normal cost. Rather than perform expensive original data collection through interviews or tests, researchers analyze key outcomes using privacy-protected data that is already captured by government agencies. This so-called administrative data includes things like student test scores on state exams, criminal arrest records, and wage records.
As of late 2017, we have funded 17 evaluations through our Low-Cost RCT Competition. These studies have large samples — typically hundreds or thousands of individuals — and focus on areas such as K‑12 education, postsecondary education, workplace safety, workforce development, and substance abuse prevention. They measure policy-important outcomes over a sustained period of time, typically between two and eight years. At a total combined cost of less than $3 million across all 17 evaluations, these studies are helping to more rapidly build the body of evidence about effective social interventions and represent a scalable way to reliably measure their impact.
In addition to directly funding RCTs, our Evidence-Based Policy team systematically monitors published evaluations of public programs. We find and report on those shown in high-quality studies to produce important impacts. Through this process, we have selected roughly 15 programs backed by strong scientific evidence of meaningful improvements in people’s lives to target for deliberate expansion. Our Moving the Needle Competition is helping to expand such programs to more locations and evaluate whether they produce consistent results across diverse communities.
Highly effective programs are rare, and even those that have a positive impact in one community may not produce the hoped-for effects in another. This can lead policymakers to implement programs that don’t work and can divert attention and funding away from those that do. As a way to help address this problem, this year, we launched Straight Talk on Evidence. The website reviews important, policy-relevant research and explains whether the results are credible and accurately reported.
The total combined cost of 17 evaluations funded through our Low-Cost RCT Competition.
