British Police Built a Sprawling Crime-Prediction Machine. Some Results Couldn’t Be Trusted

How many times have we been told that data is neutral, that algorithms are objective, and that predictive analytics will bring fairness and precision to policing? In Bristol, England, the reality was far messier. By 2016, the Avon and Somerset Police and Bristol City Council had created a sprawling predictive analytics system known as the Think Family Database, housing information on nearly half a million residents. What began as a tool to improve child welfare and crime prevention soon became a cautionary tale about the risks of opaque, unregulated data collection and algorithmic decision-making.

The Hidden Machine That Scored Everyone

At the heart of the system was a series of risk-scoring models designed to flag individuals deemed more likely to commit crimes, become victims, or be in need of intervention. These models incorporated data on everything from mental health and education records to parenting courses and free school meals. One police data scientist described the process as "dumping all that data in a big bucket and stir[ring] it with a data-science spatula."

The result was a league table of the region’s most dangerous individuals, based on a mix of known and suspected risk factors. However, the system’s logic and the data used to build it were rarely made public. Residents were not informed they were being scored, nor were they given the chance to challenge or understand their risk ratings.

Flawed Models and Public Distrust

Despite initial optimism, the system’s effectiveness was called into question. At least two of the models were quietly abandoned after internal reviews found they lacked predictive accuracy. Documents obtained by WIRED reveal that some models performed poorly, with genuinely poor predictive performance noted in over 36,000 model evaluations. These failures, combined with the lack of transparency, raised serious concerns about the integrity of the system.

  • The CSE (Child Sexual Exploitation) model, which used data from multiple agencies, was criticized for relying on proxies for poverty, such as school absences and mental health issues.
  • Independent reviewers warned that the lack of public understanding could undermine trust in law enforcement.
  • Residents like John Pegram, who discovered he was on the Offender Management App, found themselves in legal battles just to know whether they had been scored and how.

A Push Toward Widespread AI Policing

As the UK moves toward predictive policing as a national priority, the lessons from Bristol are being ignored. Andy Marsh, former chief constable of Avon and Somerset, now leads the College of Policing and has called for AI to be “injected like heroin” into the force. His team is evaluating around 100 AI tools, aiming to deploy them widely across the country.

This push raises new questions: If Bristol’s experience shows that predictive models can be biased, untrustworthy, and opaque, what safeguards are in place for the next wave of AI-driven policing? The answer, for now, remains unclear.

The Think Family Database may have been a noble attempt to bring clarity to complex social issues, but its execution highlights a dangerous gap between technological promise and the realities of implementation. As the UK moves forward with predictive analytics, it must ask whether the tools being developed are truly serving the public interest—or simply reinforcing existing inequalities under the guise of data-driven precision. The answer will determine whether the next generation of policing is a leap forward or a step into the dark.