Marrying Evidence to Legislation

Your work revolves around getting an ​“evidence-based” standard into legislation. Can you explain more about what ​“evidence-based” means? 

In the simplest of terms, ​“evidence-based” describes a program that has been rigorously evaluated and shown to improve important life outcomes for the people it serves. While relatively few higher education and workforce programs currently are backed by this kind of evidence, the list is growing as more rigorous evaluations are conducted. 

We want to help ensure that programs that have been proven to work have the opportunity to scale up in Colorado and thus help more people, and we also want to grow the menu of proven programs by helping promising local programs engage in evaluations that can credibly measure their effectiveness. 

One of the things we find astonishing at CEEMI is, overwhelmingly, states are not using rigorous evidence as a meaningful directive for how to spend federal stimulus dollars through the American Rescue Plan Act (ARPA). And that is particularly true when it comes to workforce job training and postsecondary support services for students. 

However, after a lot of work in advocacy and coalition building, we are excited that Colorado has become one of the first states to add meaningful evidence standards in the legislation that is guiding how to spend on training programs, workforce, and higher education.

Can you tell me more about how you were able to incorporate an evidence standard in House Bill 22 – 1350, the Regional Talent Development Initiative Grant Program legislation?

House Bill 22 – 1350 is a major legislative initiative, signed into law in May, which uses $91 million from the American Rescue Plan Act (ARPA) to invest in work and talent development programs. CEEMI worked with legislators from both parties to get language into that bill that prioritizes funding for programs that are backed by strong evidence. 

The portion we added to House Bill 22 – 1350 says programs that are at, near to, or clearly advancing toward the top of the evidence continuum get to go to the front of the line for ARPA funding. 

To that end, about two-thirds of the ARPA dollars are widely expected to go toward scaling and replicating programs backed by strong evidence, while at least some of the remaining dollars would go toward testing innovative new approaches and rigorously evaluating them. The hope is, if you put these two approaches together, the state will be spending $91 million on evidence-based models and substantially grow the supply of shovel-ready effective programs in Colorado between now and 2026. 

Tell us a little about House Bill 22 – 1349, the Postsecondary Student Success Data System.

This bill put $3 million toward a student success and outcome data tracking system, overseen by the Colorado Department of Higher Education to help learners understand which college programs are helping people get a good return on their investment. This bill builds on other productive work in this space already underway in Colorado to help students and families better understand the data on these programs — especially how many learners graduate, and their job prospects once they leave. As many education programs are vocational in nature, you can see that student success and workforce training programs clearly have some overlap. 

We worked with our advocacy partners to get the bill substantially redrafted prior to introduction to make it less of a ​“dashboard,” and more of a full-fledged system that integrates learner data across the higher education and workforce domains. Importantly, we also helped make sure the system would have clear access to workforce earnings data, and include not just colleges, but also any training provider receiving federal Workforce Innovation and Opportunity Act (WIOA) funding that opts into it.

Your organization has been a proponent of student success and workforce training programs such as ASAP, Nevada REA, and Per Scholas. Why are these specific programs the ones you look at when considering scaling up evidence-based programs, and what do they have in common? 

All the organizations you mentioned are backed by multiple randomized controlled trials (or RCTs), which show that they significantly increased graduation rates and/​or significantly increased employment wages. 

There is a growing list of programs that have been proven to be effective by these rigorous evaluations, and they do have some features in common. Notably, most of them are intensive. It’s hard to affect the trajectory of a learner’s life post high school with a cheap, light-touch intervention. There are some exceptions, but most of these programs are intensive. Most of them are also industry-aligned, meaning they’re aligned with the needs of high-wage, high-growth industries where talent is very much in demand, such as IT, health care, or the skilled trades. It is also important that these programs meet learners where they are. These programs should be structured in a way that is accessible to learners in poverty, learners near poverty, learners of color, and learners with barriers to employment. 

What options do programs have if they don’t yet have much of an evidence base?

We also work closely with program providers in the state that are much earlier in their evidence-building journey — we hope that some may ultimately be evaluated in an RCT, but many won’t. We understand that and we definitely don’t want programs rushing into doing an RCT before they’re ready. With these program providers, we strategize about how to measure the outcomes and impacts of their programs as rigorously as is possible for them.

What have you noticed about federal-level advocacy versus state-level advocacy for evidence in legislation?

It’s worth underscoring the value of having an entity that is doing this sort of advocacy at the state level, which most states don’t have. Some organizations are doing fantastic work at the national level, and we partner very closely with many of them. It is also true that very few organizations at the state and local levels are doing this sort of intensive, outcomes-driven, evidence-focused advocacy on economic mobility. 

From some of the legislation that has already been passed, there’s a real ROI to funding a state-based organization. It turns out you can do state-level advocacy on a smaller budget, smaller footprint, and because of the nature of federalism, it can often be easier to get stuff done. Part of our theory of change is that winning at the state and local levels will help build momentum for national and federal change, and that is what we’re seeing. 

What questions do you encourage lawmakers to ask as they bring evidence into lawmaking?

Many federal job-training programs use a standard formula to disperse funds that does not take into consideration effectiveness or evaluation of these programs. The method they use to fund WIOA, SNAP E&T, or some of the many other federal job training programs out there, should have a stronger focus on evidence — not just about whether they work, but who do they work for? Do these programs work for Black and brown learners? Do they work for learners in poverty? Do they work for learners who have other barriers to employment? Are they narrowing equity gaps rather than widening them? These are critical things we should be measuring. 

I think, because of some of the bills we’re talking about here, we’re starting to make real progress on that in Colorado.