Can Risk Assessment Tools Produce Fair Outcomes in an Unfair Society?

Pretrial detention depends on weighing an unconvicted defendant’s right to freedom against the risk that defendant poses to the community. The prediction of future criminal activ- ity lies at the heart of the justification of pretrial detention; however, both intuition and data suggest that such a predic- tion is difficult and prone to subjective bias. Using data from the 2018 Virginia Pretrial Data Project, we attempt to predict new violent criminal arrest using a variety of machine learning methods, and we compare our results with predic- tions from a far simpler tool, the Public Safety Assessment (PSA) from Arnold Ventures. We further explore alternative frameworks for algorithmic fairness when using these models as risk assessment tools and demonstrate inherent tradeoffs in competing definitions of fairness. We show that while the PSA does pass certain thresholds for algorithmic fairness, the implications of the competing definitions may not be evident to decision-makers. Our study emphasizes the current limitations of using risk assessment tools for pretrial decision-making and highlights the need for a multidisci- plinary approach that considers alternatives to detention to protect public safety and respect the rights of defendants. The results of this study contribute to the ongoing discourse on the role of data-driven tools in the criminal justice system and underscore the need for comprehensive approaches to address systemic inequalities