Funding strong, causal evidence on what works: Q&A with Jennifer Doleac

Could you tell us a little bit about the new RFP and why it is a priority for the criminal justice program at AV?

Our priorities at AV are to help build the evidence base on what works, and then push for policy change based on that evidence. In the criminal justice field, there are many open questions about how to improve community safety and how to make our criminal justice system fairer and more effective. This new RFP will be a cornerstone of our approach going forward. In particular, we are doubling down on funding strong, causal evidence on what works. Our goal is to generate new, innovative solutions that we can bring to policymakers as quickly as possible.

AV has issued many RFPs in the past related to components of the criminal justice system or to specific policy solutions. What is the motivation behind issuing an open RFP covering any and all aspects of the criminal justice system?

There are at least three reasons for this shift.

First, it is increasingly obvious how interconnected our safety and criminal justice challenges are. For instance, there may be some problems regarding prisons that are best solved by intervening in courts or with law enforcement or even outside the criminal justice system – and vice versa. This RFP moves beyond our traditional policy siloes to ensure that we’re always thinking outside the box to identify the most efficient, effective paths forward.

Second, we want to make sure that we’re never doing the research equivalent of just focusing on where it is easiest to look. Expertise and good intentions may make particular interventions seem promising. But we need to stay humble about the possibility that the most effective solutions aren’t yet on our radar. By allowing the academic and practitioner communities to work together to evaluate existing innovations and pilot new ones with limited restrictions from us at AV, we are seeking to maximize our chances of finding policies that work.

Lastly, while our criminal justice program is focused on evidence-based solutions that can make a difference today, we never know what crises or opportunities may present themselves tomorrow. We want to be ready with policy ideas that meet the moment. By steadily building a broad evidence base in the background, we can ensure that we are always equipped with new, evidence-based solutions when they are needed.

The RFP has a specific focus on causal research projects. What are the advantages of causal research projects and how can they advance the twin goals of making communities safer and making the criminal justice system more equitable and effective?

AV’s priority across our portfolios, including but not limited to criminal justice, is supporting evidence-based policy. But not all evidence is created equal. Research based on causal research designs – such as difference-in-differences, regression discontinuities, instrumental variables, and randomization – is best suited to answering the question that is always front of mind for us: Is this intervention effective? 

These research designs aim to cleanly divide people or places into treatment and control groups – a real-world approximation of a lab experiment. As we all know, correlation doesn’t equal causation. Causal methods allow us to isolate the treatment effect of an intervention from other underlying differences between groups – differences like motivation or income. Those other differences might make it seem like the intervention worked if we were less careful. 

There is of course a place for strong qualitative and descriptive research in the policy process, but those types of research are best suited for describing problems and generating hypotheses about how to solve them. Causal research tests those hypotheses. In AV’s view, that is the type of evidence our criminal justice policy conversation most desperately needs at this moment. And so that is where we are investing our time, energy, and money.

The reality is that most good ideas won’t work in practice – these are difficult problems to solve! Finding the ideas that do work is like finding needles in a haystack. Strong causal research designs work like magnets to draw out those needles, enabling us to identify and scale effective solutions as quickly as possible. 

What does success look like for you in terms of this RFP? Are there any specific short and long-term outcomes you are hoping to see come out of this process?

For me, success looks like lots of strong proposals from across the research community. Our primary goal with this RFP is to make it as easy as possible for scholars to tell us what they’re working on so that we can in turn help them do that work. Over time, I hope that our emphasis on causal research designs draws more scholars who would not otherwise have worked on such projects into this research space. We need more people, from a broad range of backgrounds, at the causal research table. 

And of course, we hope that all this research generates new solutions that we can turn into policy change. Our ultimate goal is making people’s lives better and communities safer. That’s why we’re doing all of this. 

Before coming to AV, you had a background in criminal justice research in academic settings. In what way did those experiences motivate and help inform the development of this RFP?

I was an economics professor for over a decade and spent a lot of time raising money to support my research. Because of this, I know that the research funding process often seems opaque and intimidating to academic researchers. I’m determined to change that. I want to limit the role that established networks play in the funding process – you should not need to know me or someone on my team in order to get your research funded. That will make science better. I also know funding applications are often a laborious process. Academic researchers are so busy, and their writing time is very precious. My goal with this RFP is to minimize the up-front cost of telling us about your work. A letter of interest (LOI) is just 3 pages! That’s enough for us to tell whether your project might be a fit. If we want more information, we’ll invite a full proposal. I promise that we won’t waste your time.

The RFP states that priority will be given to ​“researchers who have not previously received funding from Arnold Ventures as the primary or principal investigator, or are early-career/junior researchers (those who received their PhD in the past 6 years).” Why is this important to both AV and the criminal justice field?

In general, the research funding process is usually very opaque. Because networks typically play such a large role, a subset of scholars has become very good at raising research dollars as they figure out the rules of the game. For their part, foundations get to know and trust particular researchers, and they’re likely to fund them again and again. Meanwhile, a grant is especially valuable for scholars who do not receive much funding and for junior scholars who are just starting out. They are often the ones with fresh ideas that we want to make sure we’re hearing. To counter the built-in advantages that senior, well-funded scholars have in the funding process, we’ll prioritize proposals from principal investigators (PIs) who are new to AV or to the field. The proposals still need to be high-quality, but they’ll go to the top of our queue.

How will letters of interest and proposals be evaluated? Do you have any advice for people interested in submitting an LOI?

Since we’re expecting a much larger volume of LOIs than we’ve received in the past, we’re focused on streamlining our review process. LOIs will be sent to external reviewers, primarily to provide feedback on the quality of the research design and the scientific contribution. Internally, we’ll also consider whether it fits with our eligibility criteria and aligns with the research priority areas listed in the RFP. While research on any topic is welcome and will be considered, priority in our review process will go to proposals in these areas. Based on all of this, we’ll invite a subset of applicants to submit full proposals. We’ll include the feedback we received from the external reviewers, and full proposals will go back out to those same reviewers, so make sure you address any questions they had!

My main advice is to use your space in the 3‑page LOI to focus on what you plan to do, rather than why the question is interesting. Our team is familiar with the criminal justice research and policy space, so there is no need to include a lengthy literature review or convince us that criminal justice policy is important. Provide a brief motivation, and then dive into your main research questions, data, and research designs. Make sure it is clear why you believe your design will isolate the causal effect of the intervention you’re studying – that is our priority.

You’re also welcome to reach out to folks on our team with any questions. Keep an eye out for us at conferences too. We’re there to meet you and will be eager to hear about your research.