CHICAGO (CN) — This past winter, an AI system known as GPT-4 passed the Uniform Bar Exam - as the name suggests, a standardized version of the infamously difficult qualifying test for would-be lawyers. It often takes flesh-and-blood attorneys several attempts to pass the exam, and in a way, GPT-4 was no different.
A multidisciplinary team of researchers and attorneys, led by law professor Daniel Katz from the Illinois Institute of Technology's Chicago-Kent College of Law, put multiple iterations of artificial intelligence programs through the bar before GPT-4 took its turn.
In a study the team published earlier this month, they noted that in one of the first attempts, a program known as Text-ADA-001 scored only a dismal 8% on the test's multiple choice section. Other programs floundered when answering essay questions, or failed to grasp different domains of legal practice – civil procedure was especially challenging.
But GPT-4 passed with flying colors in every field and test segment, even outperforming the average human student. Other AIs will be sure to follow. So what does it mean that sci-fi imaginings of machines interpreting our laws have been replaced by a reality where they already have?
Even Katz isn't entirely sure.
"We're at the beginning of something here. The dawn of a new technology," Katz said in an interview. "No one is sure what it's going to mean."
Katz did predict that within a few years, perhaps even by the end of this year, large corporations like Microsoft and Google may start experimenting with offering AI legal services. Nothing that could replace an attorney in a courtroom - yet - but work that is typically performed by law researchers and paralegals, like compiling data or answering clients' questions about the law.
Several attorneys Courthouse News spoke with, including Katz, see this possibility as a net positive. As Katz pointed out, the American legal system's highly privatized nature means that for most people, attorneys outside of public defenders are an unaffordable luxury.
"Most people in this country today can't afford a lawyer," Katz said.
He and others said they hoped AI could extend some basic legal services to those who have historically gone without - immigrants, low-income families and those who file their suits pro se, without the benefit of counsel.
"I'm generally very optimistic about it," said Matthew Shepard, a public defender and board member of the Chicago chapter of the National Lawyers Guild, which often provides pro bono counsel to arrestees and activists. "I think it will be helpful in assisting attorneys with filing motions and going through large quantities of discovery"
"Sounds like it could help out a lot of pro se litigants," agreed Zane Thompson, a workers' compensation attorney with the Chicago law firm Ganan & Shapiro.
But others are wary – not only of the AI itself, but over who will control it. The price tag attached to building and maintaining an AI system can range into six-digit territory or higher every month, and their complexity means that only wealthy corporations have the resources necessary to utilize them en masse.
Tech culture author Wendy Liu and University of Illinois at Chicago computer science professor Abolfazl Asudeh, who has extensively researched fairness in data, both said that large corporations controlling such powerful technology could have negative consequences. One might be the expansion of ongoing digital enclosure - the process by which ever-more goods and services are provided by a shrinking number of massive conglomerates, making society at large increasingly dependent on and subservient to them.
"Who can create these large-language models? Other than big tech companies such as Microsoft, Google, and Amazon, who else has the capacity?" Asudeh said. "So now this technology is great and everyone wants to use it, but only the big corporations have it."