Grantee: Bottom Line. The study’s publicly-available report is posted here.
Description of the Intervention: This project was a randomized controlled trial (RCT) of Bottom Line, a program that provides one-on-one guidance to help low-income students get into and graduate from college. The program serves students beginning in their senior year of high school and continuing for up to six years of college. Full-time trained advisors help students identify colleges where they are likely to be successful, apply to those schools, complete financial aid applications, and select a college or university that best meets their needs. Once enrolled in college, students who attend a college affiliated with Bottom Line are offered ongoing support from campus-based advisors in several areas including course selection, adjusting to college social life, and managing financial aid. The program costs approximately $4,000 per student.
Study Design: The study randomly assigned 2,422 students at three sites — Boston, MA; Worcester, MA; and New York City, NY — to a treatment group that was offered Bottom Line or a control group that was not (but had access to usual school and community services). Seventy percent of sample members were female and 64% were Black or Hispanic. At the start of the study, 81% of the sample were children of parents who did not attend college, and the average household income for students in the sample was approximately $23,000. The study is measuring college enrollment, persistence, and completion outcomes over a seven-year period, using administrative data from the National Student Clearinghouse. The report discussed in this summary presents findings for the full sample for whom six-year outcome data are currently available.
Impact on the Primary Outcome: On the study’s pre-registered primary outcome, the analysis found that Bottom Line produced a sizable increase of 8 percentage points in the likelihood of earning a bachelor’s degree six years after random assignment (i.e. five years after expected high school graduation). Specifically, 55% of the Bottom Line group had received a bachelor’s degree vs. 47% of the control group. This effect was statistically significant in the full sample (p<0.01), and consistent across the study’s three implementation sites.1
Over the same period, Bottom Line also produced a 5 percentage point increase in any college enrollment (87% of the Bottom Line group had enrolled in any college vs. 82% of the control group) and a 9 percentage point increase in enrollment in a four-year college (79% of the Bottom Line group had enrolled in a four-year college vs. 70% of the control group). Both effects were statistically significant (p<0.01).2
Study Quality: Based on a careful review, we believe this study is well-conducted, and that the findings are valid.3
-
1
The effects were between 7 and 8 percentage points at each site and statistically significant (p<0.05) at two of the three sites (Boston and New York). The effect at the third site (Worcester) did not reach statistical significance, because it had a smaller sample of youth than the other sites. These site-specific impacts were requested directly from the study authors for the purposes of this summary.
↩︎ -
2
Bottom Line produced statistically significant reductions in (i) enrollment in two-year degree programs (9% of the Bottom Line group had ever enrolled in a two-year degree program vs. 13% of the control group), and (ii) receipt of an associate’s degree (10% of the Bottom Line group earned an associate’s degree vs. 13% of the control group). Taken as a whole, the study’s findings suggest that Bottom Line may have shifted some students into four-year degree programs who otherwise would have pursued two-year degree programs.
↩︎ -
3
For example, the study had successful random assignment (as evidenced by highly similar treatment and control groups), no sample attrition, and valid analyses that were publicly pre-registered.
↩︎
