The use of AI by police could erode public trust and lead to miscarriages of justice unless the government establishes an independent body to set national standards, a new report has found.

The report, from law reform organisation Justice, warns that England and Wales risk becoming an AI “wild west” if 43 police forces continue to experiment without transparency, technical standards and legal clarity.

The deployment of poorly designed AI can embed inaccuracy or discrimination in the process of investigation, leading to unjust outcomes. For example, AI facial recognition technologies have repeatedly been found to be less accurate for darker-skinned people and women, risking wrongful arrests.

The report, which draws on evidence from countries around the world, urges the government to create a central body that can set national rules for the application of AI and help police forces innovate responsibly.

It also calls for national legislation in areas like biometrics, rather than the current practice of individual forces writing their own rules for live facial recognition, which mean people are treated differently depending on where they live.

The speed of the technological revolution and the government’s push to use AI in public services leaves just a small window for urgent reform, the report concludes.

‍

Expert perspectives

Alex Murray OBE, the National Police Chiefs’ Council AI lead and National Crime Agency Director, who is backing the call for a new central body, said: “This is a pivotal moment for policing. If we do not act decisively and collaboratively, policing risks falling behind, leaving gaps that undermine public trust, effectiveness, and justice.

“Without considered standards, legal clarity, and robust governance, we face fragmented approaches and inconsistent safeguards – all of which will reduce trust.

“The decisions we make now will define how policing uses AI for at least a decade. We must act, because the cost of inaction is too high.”

‍

Ellen Lefley, Senior Lawyer at JUSTICE, says: “The relentless pace of change, financial pressures to policing, plus the lack of any clear standards creates an AI wild west.

“Responsible AI in policing will not happen by accident. We must ensure public trust and fairness so that, wherever you live in England and Wales, police AI is subject to the same technical standards, and we have consistent and clear laws on when police can legally use it, and when they can’t.

“A body with the power to set basic technical rules and give the public a say could ensure new tools respect rights, support responsible innovation and build public trust.

“Meanwhile, we have identified several areas in which legal clarity is urgently needed, with biometrics such as facial recognition top of the list.”  

‍

International examples of poorly designed or deployed police AI

The report recognises that AI can offer real opportunities for improving policing, analysing ever expanding digital evidence and providing valuable assistance with tasks such as translation. But it argues that any new AI policing tool must be accurate, accountable, and uphold the police’s legal duties, and that this will not happen if technical standards are set by the commercial interests of private AI companies.

The research gives various examples of how poorly designed or deployed police AI use risks the erosion of public trust and major miscarriages of justice. For example:  

• An algorithm produced racist risk assessments where black people were falsely more likely to be labelled as high risk and white people falsely more likely to be labelled as low risk.  
• Supposedly world-leading transcription AI has hallucinated racial identities for speakers.  
• AI tools to draft witness statements from body cam audio have hallucinated a police officer who was not at the scene. Issues like these have led to some US prosecutors refusing to work with witness statements drafted using AI.  
• One tool to alert police of gunshot fire claimed 97% accuracy but an independent review found its real accuracy figure to be 14%.  
• Decades of evidenced racial bias in “machine vision” technologies, like facial recognition, which are less accurate for darker-skinned individuals and women.  
• In the Netherlands, AI learned to stereotype by nationality when flagging fraudsters, and led to huge breaches of people’s rights without any access to fair challenge, driving many families into severe financial hardship.
• AI for summarisation and drafting did not bring efficiencies, while there is evidence of automation causing deskilling.

‍

Key recommendations  

• The government should set up an independent central body to establish mandatory technical and governance standards, help forces test against those standards and ensure they are embedded in the tools they buy from the private sector.  
• Because public participation is a key part of securing transparency, trust, and incorporating vital community perspectives into the design and use of AI, this body should incorporate a citizens’ panel or assembly.  
• To allow for independence and agility in a fast-moving landscape, this new body should set these standards in statutory codes of practice, like the Forensic Science Regulator does for forensic science in the criminal justice system.  
• The Home Office should legislate for biometric data and technologies, to clarify a confusing and disaggregated legal landscape. This should not be limited to policing but include other public sector use of these technologies, and private sector use as well.  
• The independent body would work collaboratively with police forces, regulators, lawyers, the public and academics to continue identify where legal clarity is needed in the future and recommend these to the Home Office.

‍

Read the full report

Click here to read the full report.