Grantee: The Rector and Visitors of the University of Virginia.
Description of the Intervention: This project was a long-term follow-up of an RCT of Core Knowledge Charter Schools in Denver, Colorado. Core Knowledge schools are public charter schools serving students in grades K‑8. They use an integrated curriculum that sets out, in a clear grade-by-grade sequence, the specific knowledge that students should gain in each grade, rather than simply providing teachers with broad goals and objectives about what topics to cover. The Core Knowledge curriculum is implemented in over 1500 schools (charter and non-charter) across 41 states and the District of Columbia.
Study Design: The study used randomized lotteries to allocate offers of kindergarten admission to 1,831 applicants to nine oversubscribed Core Knowledge schools in Denver. 1 The 688 lottery winners (treatment group) were offered admission; the 1,143 lottery losers (control group) were not. The sample was approximately 50% female, 8% low-income, and 8% Black or Hispanic. The primary pre-specified outcome measure was English language arts achievement (ELA), as measured by the Colorado state test, in 6th grade (i.e., the longest-term follow-up, seven years after school entry). The study also measured ELA on the state test in 4th and 5th grade.
Impact on the Primary Outcome: The study found no statistically significant effect on the primary outcome – ELA scores in 6th grade (the non-significant effect size was 0.04). The study found modest positive effects on ELA scores in 4th grade (effect size 0.09, not statistically significant) and 5th grade (effect size 0.12, 2 statistically significant). 3
The above effects, based on the study’s pre-registered primary analysis, are contained in the study team’s final grant report to Arnold Ventures. A public study report contains additional, exploratory analyses and findings.
Study Quality: Based on careful review, we believe this RCT was generally well-conducted and produced valid findings. 4
-
1
The study sample for primary analysis comprised the 1,831 children whose families applied to just one school lottery, as opposed to multiple lotteries.
↩︎ -
2
An effect size of 0.12 means that the program would have moved the average student in the control group from the 50th to the 55th percentile, had he or she been assigned to the treatment group.
↩︎ -
3
These are intent-to-treat effects – that is, the effects of winning the school lottery and being offered admission. However, only 45% of lottery winners accepted the offer and enrolled in a Core Knowledge school. The effects on those who actually enrolled – i.e., the treatment-on-treated effects – are roughly twice as large as the intent-to-treat effects (e.g., the treatment-on-treated effect in 6th grade is 0.09 and not statistically significant).
↩︎ -
4
For example, the school lotteries were monitored to ensure adherence to random assignment, and the study used valid analysis methods that were publicly pre-registered. A study limitation is the moderate-to-high level of sample attrition – 37% in 6th grade, 31% in 5th grade, and 28% in 4th grade – with attrition rates 4 to 5 percentage points higher in the control group than the treatment group in each grade. These rates are within Works Clearinghouse thresholds for acceptable levels of potential bias under optimistic, but not cautious, assumptions.
↩︎
