A study examining computer algorithms that rate a defendant’s risk of future crime found they falsely labeled Black defendants as future criminals at nearly twice the rate of white defendants.
At the same time, white defendants were wrongly identified as low risk more often than Black defendants.
Judges, prosecutors, prison officials, and parole officers increasingly are using risk assessments to make decisions at every stage in the criminal justice process, from pretrial release to sentencing to parole. Dozens of different risk assessment tools are in use nationwide, but few independent studies have evaluated their accuracy or investigated whether they are racially biased.
ProPublica examined risk scores assigned to more than 7000 people arrested in Broward County, Florida, in 2013 and 2014. The scores were produced by a for-profit company, Northpointe, whose software is one of the most widely used risk assessments in the country.
Looking at how many defendants were charged with new crimes over the next two years, which is the benchmark used by Northpointe, the study found that the risk scores were “remarkably unreliable in forecasting violent crime.” Only 20 percent of those predicted to commit a violent crime actually were charged with a subsequent violent offense.
Overall, the analysis showed that “the algorithm was somewhat more accurate than a coin flip.” Of those labeled likely to re-offend, 61 percent were arrested for any subsequent crimes (including misdemeanors) within two years.
ProPublica also reported finding significant racial disparities. A statistical analysis revealed that, after controlling for other factors, Black defendants were 77 percent more likely to be identified as at higher risk of committing a future violent crime and 45 percent more likely to be predicted to commit any kind of future crime.
Northpointe disagreed with the study’s results, but it does not publicly disclose the calculations used to produce risk scores.
ProPublica’s analysis showed that higher Northpointe risk scores are slightly correlated with longer pretrial incarceration in Broward Country. Even when risk assessment results are used in determining a defendant’s bond amount or sentence, most defendants have no opportunity to challenge the scores in court because the underlying data usually is not disclosed to defendants or their attorneys.
The study bears out former United States Attorney General Eric Holder’s concerns about the potential for bias in risk assessments. As ProPublica reports, he called on the United States Sentencing Commission to study the use of risk scores. “Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice,” he said in 2014, adding, “they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.”