Disagreement in Criminal Justice Risk Models: What Should We Audit?
```html
The use of risk assessment models in the criminal justice system has become widespread, promising more objective and data-driven decisions in areas such as bail, sentencing, and parole. Yet, as these models grow in influence, auditing their fairness, accuracy, and robustness remains critical to prevent unintended harm—particularly disparate impact on protected groups. One important but often overlooked auditing focus is disagreement: the cases where different risk assessment models or different runs of the same model produce conflicting risk predictions.
In this article, we explore why measuring disagreement is a high-signal risk indicator in criminal justice risk models. We introduce key disagreement metrics like the disagreement rate and predictive entropy, and explain how auditing these can surface edge cases, highlight distribution shifts, and reveal data gaps. We'll also discuss the challenges posed by objective mismatch and loss function tradeoffs that can exacerbate disagreement. By the end, you'll understand why disagreement audits are an essential part of responsible deployment and ongoing monitoring of risk assessment models—especially to protect vulnerable and protected groups from disparate impact.
Why Audit Disagreement in Criminal Justice Risk Models?
Traditional audits of risk assessment selective classification best practices https://smoothdecorator.com/feature-conditional-disagreement-how-do-i-build-those-slices/ models often focus on overall accuracy or fairness metrics such as false positive rates and statistical parity. While these are important, they can mask a critical area of model uncertainty and potential harm: the cases on which models disagree.
When one algorithm predicts high risk and another predicts low risk, or the same model's predictions vary substantially across training data splits or retraining cycles, it signals unstable and uncertain risk estimates. Such disagreement is often a proxy for:
Edge cases where data is scarce or ambiguous Distribution shifts between training and deployment contexts Data gaps, especially within protected subgroups Objective mismatch between what the model optimizes and what the decision context demands
Disagreement acts as a kind of “early warning system” that can complement traditional performance metrics to help identify where models are least reliable, where audit bias might hide, and where real-world harms might be concentrated.
Key Metrics for Measuring Disagreement Disagreement Rate
The disagreement rate measures the frequency with which two or more models (or model instances) disagree on the risk category assigned to cases. For example, if Model A classifies 20% of defendants as high risk while Model B classifies a different 20%, but on only 15% overlap, then the disagreement rate could be as high as 5% of cases classified differently.
How to compute: Given risk labels (e.g., "low", "medium", "high") from two models across the same dataset, the disagreement rate is the fraction of cases where labels differ. Interpretation: A high disagreement rate indicates instability in decision-making and highlights cases requiring further scrutiny or additional information. Auditing protected groups: Measuring disagreement rates within and between subgroups (e.g., race, gender) may reveal disproportionate prediction uncertainty, which could imply disparate impact. Predictive Entropy
Predictive entropy quantifies the uncertainty in a model's risk prediction at a probabilistic level. Instead of just assigning a discrete risk label, many models output probability distributions over risk categories—for example, a vector like [0.1, 0.3, 0.6] representing low, medium, and high risk probabilities respectively.
Entropy, originating from information theory, measures the impurity or unpredictability in such a distribution:
DefinitionH(p) = - Σ p_i log(p_i), where p_i is the probability of the i-th risk category
A high predictive entropy indicates the model is uncertain—probabilities are spread more evenly across categories instead of concentrated in one, suggesting the case is an edge scenario or ambiguous. A low entropy means the model is confident and peaked on one risk level.
https://seo.edu.rs/blog/counterfactual-augmentation-for-disputed-inputs-how-does-it-work-11189 https://seo.edu.rs/blog/counterfactual-augmentation-for-disputed-inputs-how-does-it-work-11189 Utility in audit: Cases with high predictive entropy can be flagged for human review, further data collection, or enhanced scrutiny. Distribution shifts and subgroups: Analyzing changes in entropy distributions over time or across groups can shed light on shifting data patterns and coverage gaps. Disagreement as a High-Signal Risk Indicator
In practical criminal justice settings, disagreement is often concentrated in cases where risk features are ambiguous or data quality is poor. These are precisely the scenarios where decisions are most consequential and least certain.
By auditing disagreement, we can:
Spot edge cases: Defendants with unusual combinations of attributes, rare criminal history patterns, or incomplete information often generate disagreement. These cases may require special protocols or safeguards. Monitor distribution shift: Over time, if a model trained on historic data faces new population distributions, disagreement rates and entropy can spike—signaling the need for retraining or updating. Uncover data gaps: Protected groups (e.g., racial minorities) may have limited representation or missing features in data. Elevated disagreement within these subgroups suggests modeling risk and disparate impact.
Ignoring disagreement risks masking biases behind aggregate accuracy numbers. For example, a model might achieve high test accuracy overall but generate wildly inconsistent predictions on Black defendants—leading to unfair treatment and accountability breakdowns.
Edge Cases, Distribution Shift, and Subgroup Coverage Edge Cases
Edge cases have sparse representation in the training data, causing models to extrapolate with less confidence. Different algorithms or hyperparameters may produce conflicting risk scores here.
Auditing disagreement helps pinpoint these cases, which are often the most vulnerable to misclassification and unjust outcomes.
Distribution Shift
When the deployment environment evolves from the original training environment (e.g., changes in crime patterns, policing, or social policy), predictive features' statistical properties can shift, resulting in more disagreement and uncertainty.
Continuous monitoring of predictive entropy and disagreement rates helps detect these shifts early and guides retraining or model tune-up decisions.
Data Gaps and Subgroup Coverage
Certain protected groups may systematically have less or noisier data. For example, if socioeconomic features critical to prediction are missing or underrecorded for a subgroup, the model may disagree internally or across versions on their risk labels.
A subgroup-specific disagreement audit can expose these gaps and spotlight fairness risks that aggregate metrics hide. Fixing these requires targeted data collection or model adjustments.
Objective Mismatch and Loss Function Tradeoffs
Risk assessment models optimize explicit loss functions during training, such as cross-entropy or mean squared error, reflecting some proxy notion of "risk prediction quality." However, these optimization objectives may diverge from the justice system's real-world goals, such as minimizing disparate impact or balancing public safety with rehabilitation.
This objective mismatch causes different models to weight types of errors unevenly, which in turn increases disagreement.
For instance:
One model may minimize false negatives aggressively to avoid releasing high-risk defendants, producing more false positives and higher risk labels overall. Another model may prioritize calibration or subgroup fairness, resulting in opposite tradeoffs and different risk assignments.
Auditing disagreement reveals where loss tradeoffs produce divergent decisions, raising flags for policy review and possible alternative loss formulations better aligned with justice principles.
Practical Steps for Disagreement Auditing in Criminal Justice Run multiple model variants: Compare agreements across different architectures, feature sets, random seeds, or training subsets to measure disagreement rate. Collect probabilistic outputs: Use predictive entropy to identify low confidence predictions for targeted review. Stratify disagreement metrics: Evaluate overall and subgroup disagreement rates and entropies to reveal disparate patterns. Monitor longitudinally: Track time trends in disagreement and entropy to detect distribution shifts early. Combine with traditional fairness audits: Use disagreement audits alongside error rates, equalized odds, and calibration metrics for a full picture. Engage domain experts: Review high-disagreement cases with legal and social workers for contextual insights before automated decision impacts. Things Accuracy Hides: Why Disagreement Matters
As an ML practitioner who has shipped risk-scored decision systems in high-stakes domains, my constant concern is: what happens on the worst day in production? Models are rarely perfect, and pure accuracy numbers can make us complacent.
Here’s a quick list of “Things Accuracy Hides” worth considering when auditing risk assessment models for criminal justice:
Conflicting decisions: Different models may yield opposed outcomes for the same defendant. Uncertainty spikes: Sudden increases in prediction entropy signal out-of-distribution inputs or broken assumptions. Subgroup-specific blind spots: Overall accuracy swamps poor performance and disagreement on protected groups. Calibration failures: Poorly calibrated risk probabilities misguide threshold-based decisions. Tradeoffs baked into loss functions: Hidden preferences mean “errors” aren’t equal across outcomes and groups.
Auditing and mitigating disagreement uncovers these hidden risks and promotes fairer, more trustworthy criminal justice risk assessments.
Conclusion
Disagreement metrics like disagreement rate and predictive entropy offer valuable, underused lenses to audit criminal justice risk models. They highlight prediction uncertainties concentrated in edge cases, distribution shifts, and among protected groups, informing efforts to reduce disparate impact.
Combined with fairness audits, subgroup analysis, and objective alignment efforts, disagreement-based audits empower stakeholders to make more informed, cautious, and equitable decisions in high-stakes justice contexts.
Remember: compliance with test-set accuracy alone is not enough. Understanding and managing disagreement will be key to ethical, effective deployment of automated risk assessment in the criminal justice system.
```