What’s Broken and What Works in Online Learning

The pandemic has upended a lot of things about life, not the least of which is education. Many colleges and universities moved their full curriculum online — and most of those that didn’t had to invent hybrid options. So, from an evidence-based viewpoint of higher education, what kind of educational harms did you see?

I think particularly when you use the phrase ​“evidence-based” — we didn’t see a lot of evidence, we don’t have a lot of evidence. What we saw in the past year and a half was a lot of people trying to figure it out, and doing it kind of blind. While there have been schools that were online for a lot longer, I don’t think we’ve ever seen this kind of mass transition and disruption in higher education. We’ve heard anecdotally that there were a lot of bad experiments and that things did not go great in a lot of places. But we’ve also heard, again anecdotally, that people are going to continue to go to school online. That makes it even more urgent to find ways to make sure that students can be successful while they’re online.

Looking at the raw numbers, a lot of what we’re seeing is very concerning. We saw big drops in enrollment last year, particularly at community colleges and among low-income students. Students at high-poverty high schools were far less likely to go directly to college last year. Data from July shows that we even saw a big drop in persistence last year, which is not surprising if you’ve heard what students’ experiences were like. New America also did a really interesting survey that showed a lot of students weren’t happy with the quality of the online education they received in the last year and a half. That said, we also saw a lot of those students are actually more likely to enroll online or in a hybrid setting in the future, maybe because they’ve built up some familiarity even though it’s been so difficult. 

Was there a silver lining to the move to online? Did you see any particularly promising or creative programs or implementations?

The thing that I hold onto the most is that if we can figure out how to do online education well, there are a huge number of potential students in our country for whom, because of family responsibilities or work responsibilities, in-person, full-time school doesn’t really make sense and isn’t really practical in the way our system is currently set up. So the silver lining for me is that when we get this right, when we figure this out, we can open up opportunities to so many more people who have been locked out of our system so far. The challenge of course is getting it right, and supporting students in that.

The last year has seen a lot of crisis education in which people have had to adapt things very quickly. But one silver lining is that some institutions, organizations, and providers have taken this opportunity to more intentionally approach some of the work they’re doing with students online, and figure out how to help these students succeed, to persist, to make the most out of their education online. 

Did your own experiences working remotely and online in the past 18 months give you any insights or theories into what you might find in this research? 

One of the things I’ve been thinking about is how much easier it is to ask for help when I can just pop over to somebody’s desk really quickly. If I have to search through our directory and try to figure out who the right person is to send an email to — that’s exhausting! And I’m not going to do it. When I think about especially students who are first-generation college students and are experiencing all of this for the first time, it’s intimidating to figure out who to ask for help. My guess is what we’ll see is that programs that are more proactive, programs that are more flexible to meet people where they are, those seem to be working in-person best, so I’m guessing that we’ll see that online. 

It’s easy to focus on the downsides. But in this RFP you’re certainly seeking programs that may have proof they actually have upsides. Beyond flexibility for students, what are some of the potential upsides for students, faculty, taxpayers?

The biggest upside is overcoming logistical challenges that face students. They can have more flexibility to do school at home. I think there is still this perception in our country that when we talk about college students we’re talking about 18-year-olds just leaving high school, moving into the dorms at residential school, and that’s just not the norm anymore. The majority is now either returning students, adult students, people who are coming back for specific skills or training, and I think we really need to adapt our collective thinking around that. Hopefully that will mean opening up access and making our schools more welcoming, more accessible, and also with better outcomes for students who aren’t the stereotypical college student.

One other thing, for students with disabilities who may have accessibility challenges on campus, online education can be a great resource for them.

Talk a little bit about the kinds of programs and projects you’re hoping to hear from in this open RFP. What are the traits of an ideal candidate?

Our focus with this RFP is really to find some of the most promising student success programs that help students succeed in online education. We’re especially interested in student support interventions like novel advising, tutoring, and other forms of outreach and engagements for students.

And to have a project that neatly sets up a larger-scale randomized evaluation. We hope to see applicants who already have the basic contours of the program worked out — though it might need some honing, and figuring out how to take the next step for research. There are so many things to answer before you go to this full-scale evaluation: Do we have the data we need? Are enough students going to want to participate? What does it look like if we’re going to deliver these services to a much bigger group of students? And we are open to applications from anyone. Any type of school, any type of provider, whether it’s the school themselves, a nonprofit or some other provider working with campuses. We are really, really open. 

What exactly are you trying to learn from the research?

We know that we need to support students better. But what we know is our schools aren’t equipped to actually support students. They are incentivized by enrollment; they are not incentivized by success, largely. So what we are trying to learn is how are some schools and some programs shifting that narrative? How are some schools making sure that they are taking responsibility for their online students and their success — changing both the narrative and the outcomes for students who are online. And also recognizing that students are whole people, with whole challenges, with whole opportunities, with whole strengths, with whole everything, and we need to recognize that and align our system with those needs. That is what I’m looking to learn: What are the leading schools and programs doing that we can test, validate, and then spread more broadly so that we continue this narrative shift of the importance of setting students up for success.

Your team focuses on accountability and the return on investment of the cost and experience of higher education. Overall, what are you most hopeful about right now?

Quite a few things! A couple of really exciting things are happening. For the first time we are seeing really intentional, proactive policies around student success at the federal level. At the same time, we are seeing a new Department of Education that is taking steps to better protect consumers, to help make students whole who have been preyed upon by fraudulent institutions, and really trying to set up a system in which student and taxpayer dollars are not flowing to the schools that are going to leave students worse off. We are seeing a sea change in the approach right now, to make sure that we are providing the best opportunities for students. I’m really excited about how all those things fit together. 

And on the flip side, what has you most concerned or outraged?

The biggest thing right now is that we’re seeing unscrupulous for-profits try to sweep in. We’re seeing enrollment drops at community colleges but we’re actually seeing enrollment increases at for-profits, because they are promising, in some cases, outright lies about job placement and earnings. They are preying on the people who have been hurt the worst in this pandemic. That makes me really angry. 

One other concern is that some nonprofit and public institutions are also cashing in on their good name to offer programs mostly outsourced to third-party providers that don’t actually provide quality outcomes and without transparency to students — and maybe offered in tandem with exploitative private financing arrangements that leave students on the hook over the long term.

Let’s end on a positive note. The Biden administration has proposed a $62 billion College Completion Fund that would be slated for evidence-based programs. By doing this research is it possible you’ll unearth an online program that would be eligible for some of those funds?

That’s the goal. The goal of all of this is to build the evidence and identify what works so we can more intentionally direct taxpayer dollars. The goal of all of our research is to build the bench of options for schools to leverage as they try to improve student outcomes. And we’re excited about the timing of that as well.

Finally, is there anything else you’d want RFP applicants to know?

I would say we are enthusiastic about applicants who have not worked with Arnold Ventures in the past. If you’re a new researcher, new institution, new provider, please do apply. We’d love to see an application. And do feel free to reach out to me with any questions. My email address is listed on the RFP.

I’d underline that and say, don’t waste time guessing what we might think. Just send us an email!