Grantee: Saga Education and the University of Chicago. The full study report is forthcoming.
Description of the Intervention: In this pilot program, the University of Chicago partnered with Saga Education to hire and train recent college graduates to tutor first graders in reading at two LEARN charter schools on the West Side of Chicago serving students from low-income backgrounds. Tutors provided 45 minutes of tutoring per day, in ratios of two students to one tutor (2:1), for the full first-grade school year. The tutoring took place during the latter half of students’ 90-minute reading periods, so that the tutoring sessions supplanted 45 minutes of usual classroom reading instruction. All students in first grade were eligible for inclusion in the study, regardless of prior ability level. The study found that the program was well implemented in close adherence to the program model.
Study Design: The study sample comprised 299 first graders at two high-poverty charter schools in Chicago. Students were randomized over two school years: 2016 – 17 and 2017 – 18. 98% of the students in the sample were Black and 95% were eligible for free and reduced lunch. Students in the sample performed slightly above the national average at baseline (57th percentile in reading, 63rd percentile in math). The students were randomly assigned to a treatment group that was offered Saga tutoring, versus a control group that received usual classroom reading instruction from a classroom teacher. Importantly, the control group was a smaller cohort than typical “business as usual” given that students were pulled out of tutoring for the latter half of a 90-minute reading block.
Impact on the Primary Outcome: The study found that the program modestly increased reading scores on the Chicago Public Schools district test at the end of the first grade school year. However, the effect did not quite reach statistical significance, so is suggestive but not yet reliable. Specifically, the treatment group scored at the 52nd percentile nationally, versus 48th percentile for the control group – a difference that was just shy of statistical significance (p=0.06). A slightly different outcome measure – the standardized effect size on the same test – was 0.13 and not statistically significant (p=0.15).
At the end of second grade – one year after the tutoring program ended – the effect size on the district test was roughly the same but, again, not statistically significant. Specifically, the treatment group scored at the 54th percentile nationally, versus 49th percentile for the control group (p=0.11, not statistically significant), and the standardized effect size was 0.09 (p=0.30, not statistically significant).
Study Quality: Based on a careful review, we believe the study was well conducted and produced valid findings. 1
-
1
For example, the study had successful random assignment (as evidenced by highly similar treatment and control groups), low sample attrition, and valid analyses that were publicly pre-registered and used an intention-to-treat approach (i.e., measured outcomes for all students assigned to the treatment group regardless of the extent of their program participation).
↩︎
