Grantee: ServeMinnesota. The full study report is linked here.
Description of the Intervention: Minnesota Math Corps is a program that enlists and trains AmeriCorps volunteers to provide structured one-on-two tutoring in math to students in grades 4 – 8 whose math test scores fall below proficiency levels. A previous short-term RCT found that the program increased student math achievement over the course of one semester. In the current study, students in the treatment group received an average of 71 minutes of tutoring per week for 23 weeks, and the tutoring lessons generally adhered to the program model. The program costs approximately $850 per student in 2020 dollars.
Study Design: The study randomly assigned 750 4th to 8th grade students struggling with math from 20 schools across Minnesota to either a treatment group that received the tutoring intervention during the school year or a control group that did not. 64% of students in the sample were white, 14% were African American, 9% were Hispanic, and 7% were Asian. 45% were eligible for free or reduced price lunch.
Impact on the Primary Outcome: At the end of the school year, the study found that Minnesota Math Corps did not produce a statistically significant effect on student math achievement as measured by the state proficiency test. The treatment group’s test scores were modestly higher than the control group’s, but the lack of statistical significance means this difference could well be due to chance and so is only suggestive.1
Study Quality: While the study was well conducted in many respects,2 it also had weaknesses that somewhat reduce confidence in its findings.3
-
1
The effect size was 0.11 standard deviations. If this is a true (as opposed to chance) finding, it means that the program would have moved the average student in the control group from the 50th to the 54th percentile, had he or she been assigned to the treatment group. This effect was slightly smaller than the statistically significant 0.15 standard deviation effect found in the earlier RCT of the program.
↩︎ -
2
For example, random assignment created treatment and control groups that were highly similar in demographic characteristics (i.e., gender, race, and grade level); the study used an objective outcome measure (state test scores); and it had valid analyses that were publicly pre-registered.
↩︎ -
3
The primary weakness was a flaw in the study’s random assignment procedures that inadvertently resulted in a sizable difference between the treatment group and control group in their pre-program math scores, with the control group having substantially higher scores on average (the difference was 0.22 standard deviations, which the study analysis appropriately sought to correct for using pre-specified statistical adjustments). The study was also unable to obtain test score outcomes for 11% of the treatment group and 3% of the control group. These differential rates of sample attrition, while modest, could have partly undermined the equivalence of the two groups and thereby reduced the accuracy of the study findings.
↩︎
