Artificial intelligence can improve access to justice, but could come into conflict with important legal values and even cause harm, new research finds.
Henry Zwartz
h.zwartz@unsw.edu.au
Imagine finding yourself in court but rather than a human judge considering your case, a computer will be considering your evidence.
In Estonia, the automation of small contract disputes is being actively pursued. The Estonian Ministry of Justice says it will seek to clear a backlog of cases using 100 so-called ‘AI judges’, the intention being to give human judges more time to deal with the more complex disputes.
The project could adjudicate small claim disputes under 7000 euros. In concept, the two parties would upload documents and other relevant information, and the AI system will issue a decision that can be appealed to a human judge.
While this implementation of AI has a direct impact on the parties in the case, AI is increasingly seeping into court process around the world in often quite mundane tasks.
A joint research project by the Australian Institute for Judicial Administration (AIJA), UNSW Law & Justice, UNSW Allens Hub for Technology, Law and Innovation and the Law Society of NSW’s Future of Law and Innovation in the Profession (FLIP Stream), has identified some of the key issues arising from the increasing presence of AI in court systems around the globe.
The project’s report, “AI Decision-Making and the Courts: A Guide for Judges, Tribunal Members and Court Administrators”, identified examples of the use of AI in Australia and overseas, from computer- based dispute resolution software to the use of computer code based directly on rules driven logic, or ‘AI judges’ to help clear a backlog of cases.
Professor Lyria Bennett Moses is the Director of the UNSW Allens Hub, as well as the Associate Dean of Research at UNSW Law & Justice.
Professor Bennett Moses said that despite hesitancy, AI was a growing part of court processes.
“Artificial intelligence, as a concept and as practice, is becoming increasingly popular in courts and tribunals internationally. There can be both immense benefits as well as concerns about compatibility with fundamental values,” she said.
“AI in courts extends from administrative matters, such as automated e-filing, to the use of data-driven inferences about particular defendants in the context of sentencing.
“Judges, tribunal members and court administrators need to understand the technologies sufficiently well to be in a position to ask the right questions about the use of AI systems.”
Image: Shutterstock
Some of the concerns around AI’s compatibility with legal values have been identified in the United States following the use of what is known as the Correctional Offender Management Profiling for Alternative Sanctions tool, or COMPAS, the report says.
The tool is intended to augment judicial process by conducting a risk assessment on the likelihood that an offender will break the law again. As the research report notes, COMPAS integrates 137 responses to a questionnaire.
Questions range from the clearly relevant consideration, ‘how many times has this person been arrested before as an adult or juvenile?’, to the more opaque ‘do you feel discouraged at times?’.
The code and processes underlying COMPAS is secret, and so not known to the prosecution, defence or judge.
The findings of the COMPAS tool have very real consequences, and will inform the judge on whether the alleged offender can be granted bail, or whether the accused should be eligible for parole.
In a 2013 case, Paul Zilly was convicted of stealing a lawnmower. The prosecution together with Mr Zilly’s lawyers agreed to a plea deal of one year in a county jail and a subsequent supervision order.
Read more: The growing data privacy debate: is AI being used to undermine your privacy?
But on the basis of a high risk of reoffending COMPAS score, the judge rejected the plea deal and sentenced Mr Zilly to two years in jail.
In 2016, non-profit investigative journalist site ProPublica looked through around 10,000 criminal defendants in Florida. It found that African American defendants were more likely to be given a false positive flag of high risk on the COMPAS software than white defendants who were more likely to be given a false positive low-risk score, despite not in fact being low risk.
Professor Bennett Moses questioned whether similar tools should ever be acceptable in an Australian context.
“Everyone has a right to be treated impartially,” she said. “The use of some tools is in conflict with important legal values.
“[T]here are tools, frequently deployed in the United States, that ‘score’ defendants on how likely they are going to re-offend. This is not based on an individual psychological profile, but rather on analysis of data. If people ‘like’ you have reoffended in the past, then you are going to be rated as likely to re-offend,” she said.
“The variables used in this analysis include matters such as whether parents are separated (and, if so, one’s age when that occurred) – the kinds of things that might statistically correlate with offending behaviour but are outside one’s own control.
“The tool is also biased (on some fairness metrics) against certain racial groups. It is important to ask whether the use of such tools would be appropriate in an Australian court,” she said.
Even though the Estonion project could save court resources and improve efficiency, the report raised several concerns around the implementation of AI in courtrooms, including how the secret nature of many AI systems and code meant that judges, in addition to parties, could be unaware of the way in which decisions were generated.
'Everyone has a right to be treated impartially. The use of some tools is in conflict with important legal values.'
Another concern was that if AI models were created and used on predominantly English-speaking non-minority datasets, the software could have greater difficulty in interpreting accents or working with people from non-English speaking backgrounds.
Also, an over reliance on AI systems in court processes could take an important human element out of justice or, as the report said, remove some of the “moral authority” and discretion used in applying the law. In some cases, judges have overridden their own decision based on the recommendations of AI leading to significant differences in outcome, particularly in the US with the use of COMPAS.
Image: iStock
While the American experience of AI in the courtroom has raised questions domestically and internationally, the report also identified positive experiences where AI has aided access to justice.
Professor Bennett Moses said language barriers was just one key area where AI could be of enormous value.
One practical and non-controversial example of a benefit is the use of natural language processing in converting audio of what is spoken by judges, witnesses in court and counsel, to text, she said.
This can make access to court transcripts faster and easier, particularly for those with hearing impairments. In China, some trials are captured ‘in real time’ in Mandarin and English text.
"I’ve always believed that interesting legal questions lie on the technological frontier, whether that relates to AI or other new contexts to which the law is called to respond.
"My main advice is to tread carefully, to seek to understand how things work before drawing conclusions on what the law should do about it. But we need people to ask the right questions, and help society answer them,” Professor Bennett Moses said.