Grantee: Research Alliance for New York City Schools at New York University. The full study report is posted here.
Description of the Intervention: This RCT evaluated the effect of CTE programs in New York City high schools that were oversubscribed and used randomized lotteries to determine student admissions. The CTE programs encompassed a diverse array of programmatic approaches (e.g., different occupational focus areas, course sequences, work-based learning experiences, and whole-school models versus mixed academic/CTE school models).
Study Design: The study sample comprised 19,995 students who, between 2007 and 2013, applied to enter ninth grade in one of 76 oversubscribed public high schools hosting a total of 138 CTE programs. As part of New York City’s school choice system, these students were randomly assigned via lottery to (i) a treatment group that was eligible to attend their first-choice school, or (ii) a control group that was not.1 The sample was 47% Hispanic and 37% Black; 41% spoke a language other than English at home.
The study’s two primary outcomes, measured 18 months after students’ scheduled high school graduation, were rates of (i) on-time high school graduation and (ii) college persistence, defined as remaining enrolled in college for two consecutive years after on-time high school graduation. Outcomes were measured using administrative data from the New York City Department of Education, City University of New York, and the National Student Clearinghouse.
Impact on the Primary Outcomes: The study found a small, statistically significant adverse effect on on-time high school graduation (49.5% of treatment group students graduated on-time versus 51.6% of control group students). The study found no effect on college persistence (18.0% of treatment group students persisted in college versus 18.1% of control group students, a difference that was not statistically significant).
Importantly, the study estimated the average effect of many diverse CTE programs, and these average results may well mask positive effects for some specific CTE programs and adverse effects for others.2 In addition, the study did not measure CTE programs’ long-term effects on workforce earnings, which is a one of the key targeted outcomes of CTE.
In their report, the study authors discuss possible explanations for the observed adverse effect on graduation (e.g. suggestive evidence that the adverse effects were concentrated among CTE programs within high schools that were not exclusively dedicated to CTE and high schools that were discontinued).
Study Quality: Based on careful review, we believe this is a well-conducted RCT that produced valid findings.3
-
1
34% of the students who lost the lottery for their first-choice school with a CTE program ended up attending a school with a CTE program that was not their first choice (e.g., through a subsequent stage of the lottery process). The study appropriately kept these “cross-over” students in the control group when estimating CTE’s effects, consistent with an intention-to-treat approach.
↩︎ -
2
The authors are conducting additional studies to examine a more recent sample of CTE programs and to learn about the conditions under which CTE may be most effective.
↩︎ -
3
For example, the study had successful random assignment (as evidenced by highly similar treatment and control groups), low and non-differential sample attrition, and valid analyses that were publicly pre-registered and, among other things, appropriately accounted for different random assignment ratios across different lotteries.
↩︎
