ProPublica’s investigation into COMPAS, the Correctional Offender Management Profiling for Alternative Sanctions risk assessment tool, revealed a system fraught with racial bias and questionable accuracy. The nonprofit news organization’s reporting brought national attention to the fact that a widely used algorithm, designed to predict the likelihood of a defendant becoming a future criminal, was fundamentally flawed in its application. This examination of the proprietary software exposed how statistical predictions, when embedded within the criminal justice system, can codify historical inequities under a veneer of technological objectivity.
Understanding the COMPAS Algorithm
COMPAS, developed by the Northpointe company (now part of Equivant), is a suite of risk assessment instruments used by courts across the United States. It generates scores intended to help judges make decisions regarding bail, sentencing, and parole. The algorithm analyzes a defendant’s static data, such as age and criminal history, alongside dynamic factors like education and employment, to assign a risk tier. However, the core issue lies not necessarily in the mechanics, but in the transparency and validation of these metrics, which are treated as trade secrets.
The ProPublica Investigation and Findings
In 2016, ProPublica obtained and analyzed COMPAS data for more than 7,000 arrestees in Broward County, Florida. Their data science team discovered that the algorithm was twice as likely to incorrectly flag Black defendants as future criminals, compared to white defendants. Conversely, white defendants were much more likely to be mislabeled as low-risk when they subsequently committed another crime. This disparity indicated a systemic bias where the tool’s accuracy was not equal across racial lines.
Key Disparities Highlighted
False positive rates for Black defendants were significantly higher than for white defendants.
The algorithm struggled to accurately predict recidivism for white individuals, often understating their risk.
Black defendants were consistently scored as higher risk, even when their actual outcomes were similar to white counterparts.
Legal and Ethical Implications
The revelations sparked a fierce debate about the role of AI in the justice system. Legal scholars argued that using a biased algorithm violates the Due Process Clause of the Fourteenth Amendment, as it can lead to unjust sentencing based on race rather than individual culpability. Ethicists pointed out the danger of "automation bias," where humans defer to algorithmic decisions even when they are flawed, thereby outsourcing moral judgment to unaccountable code.
Industry Response and Secrecy
Equivant, the company behind COMPAS, defended its methodology, stating that the tool was not designed to assess race and that its general accuracy rates were in line with industry standards. They emphasized that the specific validation data and weightings are proprietary intellectual property. This stance of secrecy, however, directly contradicts the scientific principle of peer review and makes it impossible for independent researchers to verify the claims or improve the model.
Impact on Criminal Justice Reform
The ProPublica report acted as a catalyst for change, prompting several states and municipalities to reevaluate their reliance on algorithmic tools. Some jurisdictions moved to ban the use of such risk assessments, while others called for greater transparency and oversight. The investigation underscored the need for legislation that ensures audits for bias and mandates that vendors provide clear evidence of their tools' validity and fairness before deployment in courtrooms.
The Broader Context of Algorithmic Bias
COMPAS is merely one example of how algorithms can perpetuate systemic inequality when deployed without rigorous scrutiny. From hiring tools to facial recognition, similar patterns of bias have been found in other AI applications. The ProPublica investigation serves as a critical lesson that technology is not neutral. Without diverse data sets, transparent methodologies, and continuous monitoring, automated decision-making risks amplifying the very injustices society seeks to rectify.
