Grantee: American Institutes for Research. The full study report is posted here.
Description of the Intervention: Literacy First provides tutoring in reading to K‑2 students from underserved communities in the central Texas area. Students are selected to participate in the program if they read below grade level at the beginning of the school year, and participating students receive daily, one-on-one, 30-minute lessons over the full school year with a highly trained AmeriCorps volunteer. These lessons are tailored to match the student’s literacy level and are available in either Spanish or English based on the student’s preferred language. Program features reflect the research-based practices recommended by the National Reading Panel and the Institute of Child Health and Human Development, and identified as effective in recent meta-analytic evidence reviews.
Study Design: The study sample comprised 798 students entering second grade in fall 2017 or 2018 at 22 Austin Independent School District elementary schools, who were identified as reading below grade level. These students were randomly assigned to a treatment group that received Literacy First tutoring in second grade, or a control group that received usual classroom instruction. Sample members were approximately 88% Hispanic and 93% low income, and 69% had limited English proficiency. The study’s primary pre-registered outcomes were reading and science achievement, as measured on the state test at the end of fifth grade. The study also measured, as an interim outcome, reading comprehension at the end of second grade, using well-established tests (Iowa Test of Basic Skill and its Spanish counterpart, Lorgramos).1
Impact on the Primary Outcomes: The study found no statistically significant effects on the primary outcomes – reading or science achievement at the end of fifth grade (the non-significant effect sizes were 0.04 and ‑0.08 standard deviations, respectively). Although no effects were discernible in fifth grade, the study did find a statistically significant interim effect on reading comprehension at the end of second grade (i.e., the tutoring year). The effect size was 0.19 standard deviations, representing approximately a 20% improvement over the annual gain otherwise expected for second graders.
Study Quality: Based on a careful review, we believe the study was generally well-conducted and likely produced valid findings.2
-
1
The study originally intended to measure third and fourth grade achievement outcomes for the full sample, but was only able to obtain these outcomes for a portion of the sample due to cancellation of state achievement tests in the wake of the COVID-19 pandemic.
↩︎ -
2
For example, the study had successful random assignment (as evidenced by similar treatment and control groups) and valid analyses that were publicly pre-registered. One study limitation is fairly high sample attrition (47%) for fifth grade achievement outcomes – a limitation partly mitigated by the fact that attrition rates were nearly identical in the treatment and control groups. For fifth grade reading, attrition rates fall within Works Clearinghouse boundaries for acceptable levels of potential bias under optimistic, but not cautious, assumptions. For fifth grade science, attrition rates fall within these boundaries under both optimistic and cautious assumptions.
↩︎
