By Debbie Mayer

Across the country, justice system agencies and partners have embraced data-driven tools that predict an individual’s likelihood of committing a crime. To successfully implement these risk assessments—and ensure that these tools do not increase racial disparities within the justice system—agencies must establish a host of new processes to administer the tools, capture the results, use the data to inform decision making, and monitor quality assurance. Through RDA’s work with probation departments and other justice system agencies and partners, we have observed a number of attempts to implement risk assessment tools, with varying degrees of effectiveness. This blog provides an overview of risk assessments, highlights concerns about their use, and offers recommendations for implementation.

First developed in the 1970s, risk assessments have evolved over time and become more specialized. There are an ever-growing number tools intended for different decision points in the criminal justice system; agencies use risk assessment tools to inform decisions around pretrial detention, juvenile detention, sentencing, parole release, supervision intensity, and program eligibility. Assessments, which are essentially questionnaires, vary in length and content, but almost all include factors (i.e., questions) around gender, age, and criminal history, which have been found to be the most predictive factors of future criminal involvement. Assessments output a raw score which then can assign an individual into a risk category such as low-risk, moderate-risk, or high-risk.

Deemed an evidence-based practice, risk assessments are intended to provide unbiased, objective predictions that are more accurate than individual judgment and support a better, more structured allocation of resources within the criminal justice system. Research has shown that sanctions and services are only effective when well-targeted, and can even be counterproductive when poorly targeted. For example, putting low-risk individuals in highly structured residential programs can lead to negative peer effects and weaken protective factors such as jobs or family relationships. Additionally, risk assessments have been—and continue to be—critical components of criminal justice reforms to decrease juvenile detention and eliminate money bail.

While risk assessments hold great promise to improve the criminal justice system, a number of concerns remain regarding their use. Though they are intended to reduce bias in the justice system and do not adjust scores based on race, many factors within the risk assessments, such as criminal history and neighborhood, are highly correlated with race and may therefore be more likely to score black people as high risk. Additionally, because many risk assessments are proprietary, there is a lack of transparency about what information they use and how scores are calculated.

Based on our work with criminal justice agencies, there are a number of considerations that should be taken into account to maximize the effectiveness of risk assessments. These considerations include:

• Creating a stakeholder working group knowledgeable about the local context to assist with the design of the tool.
• Making sure that tools are validated on the local population.
• Establishing clear policies about how scores should guide decision making and communicating these policies to staff responsible for administering the tools and using the scores for decision-making.
• Ensuring staff are adequately trained to administer the tools and providing refresher trainings as needed.
• Developing data systems that capture scores and allow monitoring of how scores are used. If staff can override the tool, make sure the reason is clearly documented.
• Constantly analyzing score results, particularly focusing on racial disparities, effects on subgroups, and overrides. If the tool is found to increase racial disparities, agencies should modify or eliminate the factors responsible for the disparities.

Overall, justice systems agencies showing the desire to use data in their decision-making processes is a step in the right direction. However, risk assessment tools should be used responsibly in order to avoid exacerbating existing inequities.