Artificial intelligence has permeated most aspects of society, and the legal system is no exception. Today, every step of criminal justice, from the police investigation to the trial, can — and often does — involve AI. That change comes with many implications, some of them desirable and some less so.
Last month, three SFI Faculty — Resident Professors Cristopher Moore and Melanie Mitchell, and External Professor Melanie Moses, a distinguished professor of computer science from the University of New Mexico — addressed these implications during panel discussions at the National Judicial Summit on the Foundation and Future of the Judiciary, held in Santa Fe. Around 300 judges attended the summit, with two-thirds of the participants coming from New Mexico. The additional hundred judges represented 22 different states and eight tribal nations.
Two guests of SFI also participated in the panel discussions: Berkeley law professor Andrea Roth and Maryland public defender Marc Canellas. Both are experts on AI-generated evidence who also attended a 2025 SFI working group on AI and Justice supported by the Robert Wood Johnson Foundation through SFI’s Emergent Engineering project.
The researchers’ goal was to give judges a realistic view of how AI can expedite their work while also warning them of pitfalls they might encounter. “If judges and juries are dazzled by the technology, it’s going to be hard for them to think critically about the evidence,” says Moore. “I want people to understand that AI has strengths, and it has weaknesses.”
For example, AI has made facial-recognition technology much more effective. But when using this technology, police need to balance the need to catch criminals with the expectation of privacy. “Do we want cameras on every street corner that are constantly watching us?” Moore asks. “That’s a policy question, and I think a very important one.” He also stressed that police and judges should be aware that facial recognition is far from infallible. He thinks it should be treated as an anonymous tip, as opposed to being used as grounds for arrest.
Facial recognition and other AI tools may also perpetuate biases against certain demographic groups because the content they generate reflects their training data, in which vulnerable groups are often underrepresented. Humans also perpetuate biases, however, so whether AI is more prone to this than we are is still an academic question worth studying, Moses says.
Similarly, AI expedites legal writing, but comes with the risk of hallucinations. Some lawyers and judges have been embarrassed when their briefs were found to contain fictional cases or misinterpretations of real cases. Reading AI-generated text carefully is a first step toward catching these inaccuracies, but “as these tools get better, the mistakes become more subtle and harder to spot,” says Moses.
Meanwhile, the accessibility of the justice system has been remade by AI. Laypeople can now ask chatbots whether their case is worth bringing before a judge, and as a result, the number of lawsuits filed without a lawyer has jumped substantially. For example, federal courts saw a 69% increase in cases related to the Fair Housing Act filed without a lawyer in the first nine months of 2025 compared to 2024. Some of these cases are worthwhile, but other times AI, with its tendency toward sycophancy, erroneously tells users they have a good case. Moses says judges will have to adjust to how AI is shifting the nature of cases that end up in their courtrooms.
Judges who attended the sessions had a wide range of mindsets regarding AI, from enthusiastic to highly skeptical, mirroring the general population, Moore says. His hope is that the panel discussions demystified the technology and gave judges a feel for how the field of AI is evolving so that they can make informed decisions about which tools can be used responsibly in which situations.
In particular, one message Moore was sure to convey is that AI can help deliver justice, but only if the tools are designed to be transparent. Some use proprietary systems that prevent independent groups from testing them for accuracy and fairness, and that makes it hard or impossible for the defense to contest the evidence they produce. “We have a constitutional right to confront our accusers, and you can’t confront a black box,” says Moore.