Proving That Skills Training Programs Can Help Working Families

Tell us a little about how and why you got started with identifying and evaluating programs to advance economic mobility.

The work started at the Ford Foundation, where I was supporting workforce organizations preparing people with low incomes for good jobs in particular sectors of the local economy. I saw programs that I felt were making a meaningful difference in people’s lives, but, at the time, all the formal research said that job training programs did not work. I left Ford to go work at Public/​Private Ventures, a nonprofit that tested innovations for effectiveness, to launch and develop its labor market department. We ran a number of demonstrations that looked at how much money people made before and after they participated in a program. I would share this information with policymakers, but without a randomized controlled trial, they did not believe the programs had an impact, because all rigorous evaluations had shown job training had little or no benefit for participants.

What happened was that the Mott Foundation held a board meeting that included a panel about policy directions. One of the panelists said to the board, ​“Why are you investing in job training when all the research shows it doesn’t work?” My colleague Jack Litzenberg at the Mott Foundation and I decided to launch the Sectoral Employment Impact Study. We looked for the most promising programs nationally that were ready to go through a rigorous evaluation, and we selected Project QUEST, Per Scholas, JVS Boston, and the Wisconsin Regional Training Partnership. Each of them went through a separate study, with treatment and control groups, and, lo and behold, each of them showed large, statistically significant earnings impacts. They started to build the foundation of evidence that good workforce development programs can make a difference in people’s lives.

What are Economic Mobility Corporation’s preferred methodologies for assessing whether a given program works?

We work with organizations on analyzing their performance and helping them figure out how to improve it. It’s a range of outcomes analysis and ethnographic work all the way to full-blown randomized controlled trial evaluations. We enjoy the nitty-gritty of program operations and conversations with staff who have a thirst to understand what’s working, what’s not, and what they can do to improve their performance. As specialists in randomized controlled trial evaluations, we only do them when we feel an organization has a good chance of succeeding. 

In recent years, you have evaluated several workforce development programs — including through Arnold Ventures-funded randomized controlled trials — that have been shown to meaningfully improve earnings for low-income workers. When you look for programs to evaluate, what are the indicators that suggest a program will show promise?

There are a few criteria we use. A program has to focus on people with low incomes and be in operation for several years. We select organizations that have a well-implemented strategy that includes a substantial investment of resources and time into the development of participants’ skills and credentials. It’s also important that a high proportion of the people they enroll in their programs not only graduate but get good jobs in the occupations they were trained for. We also do focus groups with students to understand what kinds of careers they’ve had and what kind of difference the targeted occupations would make for them. Finally, we want to get a sense of the quality, stability, and capacity of the program, so we interview staff and observe training. 

How did those indicators appear in some of the programs that have recently been shown to be particularly effective — for example, Year Up, Per Scholas, Project QUEST, and English for Advancement?

They all met those criteria and had strategies that we believed were going to make a substantial difference in people’s lives. 

Project QUEST was training people to be licensed vocational and registered nurses. It would take one to four years for people to complete their degrees. San Antonio is a very low-wage economy driven by retail, hospitality, and tourism, so nursing occupations promised to pay a much better wage. We knew they had strong performance in terms of graduation and placement rates. 

Per Scholas was training people for information technology, a sector that has strong demand, pays good wages, and does not depend much on people having educational degrees — it’s credentials that matter. The program makes a 14-week, full-time investment in its students and has focused strategies, substantial investment, and high graduation and placement rates. 

Year Up has a really interesting model. Of the programs we’ve evaluated, it’s the only one that has shown an impact with young adults. It has a six-month, full-time training program in which the students get paid a stipend. It was placing students in Fortune 500 companies, opportunities they would have been unlikely to land on their own. 

JVS Boston English for Advancement program is preparing immigrants to get good jobs in the Boston-area economy. To our knowledge, it’s the first adult education program for English language learners that has shown statistically significant employment and earnings impacts. JVS’ relationships with employers open doors for people that I don’t think would be possible otherwise. That evaluation is hot off the presses and very exciting.

Given the populations served by these programs, what role do you see them playing in helping to advance racial equity?

One of the most serious issues that this country will have to contend with for coming generations is inequality of opportunity. These programs are providing a pathway to opportunity for participants who have all the talent in the world. The vast majority of the participants in these programs are African American or Latinx — 80 to 90 percent are nonwhite. Many of them have tried college, and it has not worked out for them, for all kinds of reasons. Having other options, especially these high-performing ones that have been shown to make a difference, is essential. We just need a lot more of them.

With government budgets strained right now, what would you like to see in the way of smart, strategic investments that will help with greater economic mobility?

There’s a lot of money the government spends that could be spent better. I don’t think federal, state, and local governments are focused enough on supporting the programs that have proven to make a difference, and they’re not aware of the research. For example, there’s a tremendous amount of money spent on adult education in this country. Many of the people going to these programs are really learning English so they can get a better job, but the programs don’t focus on helping them do that. 

I’d like to see the federal government in particular invest, as part of its workforce strategy coming out of this pandemic, in proven workforce development programs. We have been working with New Jersey Representative Mikie Sherrill on legislation that would create a fund dedicated exclusively to organizations that have shown impact. It would invest $100 million a year, over seven to nine years, in organizations like Project QUEST, Year Up, Per Scholas, and JVS Boston.

What role can philanthropy play in these efforts?

Invest more in identifying programs that can really make a difference in people’s lives. Use rigorous evaluations to test whether they work. When they do work, invest to bring them to scale.