The AI revolution: Is your next lawyer a machine?

Could an AI chatbot be the key to justice for the thousands of Kiwis who can’t afford legal advice? Nikki Macdonald investigates its potential, and tests its response to some basic legal questions.

The world’s first robot lawyer bombed before it even hit the courtroom.

American consumer advocacy business DoNotPay promised that, on February 22, its AI lawyer would whisper in the ear of a traffic court defendant, telling them what to say to defend the speeding charge.

But CEO Joshua Browder pulled the plug before the case got anywhere near court, after reportedly being threatened with jail time if he followed through.

READ MORE:
* Can I trust ChatGPT for medical advice?
* Will AI steal our jobs, or just rob us of our self-esteem at work?
* If AI can write a novel, can it autocomplete my chores?
* ChatGPT: a force for good or evil?

So there’s no precedent yet for courtroom defence by AI. But there are hopes the technology could make the answers to legal problems cheaper and easier to find.

Question: My landlord keeps turning up at the house with no notice. Is this legal?

ChatGPT: In New Zealand, the Residential Tenancies Act 1986 sets out the rights and responsibilities of landlords and tenants. Under the act, landlords are required to give tenants reasonable notice before entering the property. This notice period is typically 24 hours, unless there is an emergency.

If your landlord is entering your property without giving you proper notice, they may be acting in breach of the act. You can raise this issue with the Tenancy Tribunal, which is a government-run organisation that resolves disputes between landlords and tenants.

Verdict: 7/10 Key message, relevant act and Tenancy Tribunal complaint avenue are correct, but minimum notice period for inspections is 48 hours. (It’s 24 hours for maintenance.)

Can an AI chatbot solve your legal problems?

Way back in about 2015, tech guy Matthew Bartlett was helping edit the legal manual at Community Law, which offers free legal advice. Convinced there should be an easier way for ordinary Kiwis to get legal answers as and when they are needed, he wondered if chatbots could be the answer.

With funding from the Borrin Foundation, he and then Community Law Wellington manager Geoffrey Roberts started charity CitizenAI. They developed three chatbots, for rental law, for employment law, and for prison law.

“It was access to justice, pure and simple,” says Roberts. “And a new way to do that.”

More than 60,000 Kiwis used the bots, and the bots did pretty well at answering simple legal questions. They sometimes got it wrong, or made inappropriate comments, but it was still better than a Google search.

But the money dried up and the bots are no longer online.

Matthew Bartlett says OpenAI’s question-and-answer tech, ChatGPT, is on a whole new level. He asked it to write a legal-sounding letter to solve his faulty dishwasher dispute. The result was impressive.

“It feels like there’s huge potential … None of the other things I’ve played with really felt like, this might really shake things up in a strange way that’s impossible to predict. And that’s what this one feels like.”

His dream, in, say, five years, would be to have a Work and Income bot – bottle a benefit advocate and stick it on a smartphone.

Roberts reckons the new technology could run a live criminal defence. If AI can learn to win complex strategy game Go, why couldn’t it also game out courtroom arguments?

But there are at least three barriers to that ever becoming reality – accuracy, money and hype.

Question: My neighbour’s apple tree hangs over my path. Can I take the apples on my side?

ChatGPT: In most countries, including New Zealand, if a tree grows on someone’s property and its branches extend over a boundary into a neighbour’s property, the neighbour has the right to remove the branches and the fruit that is growing on them.

Verdict: 0/10 Wrong. You can cut back encroaching branches, but the fruit belongs to the tree owner. When we rephrased “take” to “pick”, it cited an Australian act as the relevant law.

Convincing, but wrong

Matt Bartlett is a law and tech researcher at New York’s Columbia University, and a former Kiwi employment lawyer. (Yes, there really are two, unrelated Matthew Bartletts in this story, sorry. Lawyer Bartlett is Matt, tech Bartlett is Matthew.)

He ran some employment law problems through ChatGPT, with mixed results. It gave good answers about sexual harassment at work, including practical tips such as documenting what happened. But when he asked it to summarise a specific case, it wrote four paragraphs that were impressive but wrong.

“It’s really convincing. The problem is, it’s convincing when it’s right, and it’s also convincing when it’s factually wrong … And at the moment, it’s hard to tell, unless you’re a lawyer or have background.

”This thing does not understand the law. I think it would be really easy for someone who was just typing in legal questions, getting some pretty smart-sounding stuff back, to be tricked by this kind of facade, or simulated legal knowledge.”

Matt Bartlett says no-one should be rushing to get legal advice from chatbots now, especially on important issues. But he is optimistic they might eventually become reliable enough to make the law more accessible.

“That’s my great hope. Maybe the end of the decade, maybe before then, we will have a tool, where some person can just type in a legal question and get a good answer, and they don’t need to know anything about the law.”

But AI courtroom lawyers? He’s not so sure. But not because he thinks AI is incapable of dealing with the complexities of a specific case.

Although what lawyers do is hard and complicated, “it’s also a little bit algorithmic”, he says. They extract a principle from statutory law and immense archives of case law, and apply it to a new set of facts.

“What is ChatGPT, but this amazing engine of billions of sources? And it takes all that and applies it to a new thing, which is the question you ask it.

“I think we would have to be incredibly arrogant in the legal industry to think that we were special, or that what we did was so cognitively advanced that it’s beyond the auspices of AI.”

Question: My boss pinched my bum. What should I do?

ChatGPT (Edited for length) : If your boss has touched you inappropriately, it is considered sexual harassment and is against the law in New Zealand.

The Human Rights Act 1993 and the Employment Relations Act 2000 prohibit sexual harassment in the workplace. If you have been subjected to sexual harassment, you have several options:

• Report the incident to your employer.
• Make a complaint to the Human Rights Commission.
• Seek legal advice.

It is important to take action as soon as possible to address the issue and protect your rights. Sexual harassment is a serious matter and should not be tolerated in any workplace.

Verdict: 6/10. Generally helpful, but you can get better info at employment.govt.nz

Who would pay?

When Nick Whitehouse set up legal AI company McCarthyFinch, his goal was to use AI to make the law cheaper, faster and less intimidating, to reach some of the 700,000-odd Kiwis and Australians denied access to justice every year.

To understand the gaps, he started a free legal advice website. Within three months, with no advertising, he had 700 clients ranging from people going through court to people arguing with their neighbours or being defamed.

“It was eye-opening to me. This was work that would cost too much for normal people to go get done at a law firm. A lot of it ended up at Community Law and Citizens Advice.”

But Whitehouse quickly worked out that wasn’t a sustainable business, so he switched instead to reducing corporate legal bills by using AI to streamline contract drafting, or exiting a lease because of Covid, or understanding the contract implications of Russian sanctions.

McCarthyFinch was bought by Onit in 2020 and Whitehouse is now the managing director of its AI centre of excellence. But he still hopes the technology could improve access to legal information for more routine problems.

“Putting something like ChatGPT into a Citizens Advice Bureau or a Community Law I think would be very powerful. And I don’t think it would be cost-prohibitive to do that in the next decade.”

But Community Law chief executive Sue Moroney says while its trial showed chatbots worked, they were too expensive for a cash-strapped NGO. As well as the hosting cost, there was the expense of monitoring.

“If you’re not watching it, and keeping it updated all the time, that can go very wrong.”

“Frequently Asked Questions” are a much cheaper way of getting information across, Moroney says. However, chatbots might work for well-funded government departments, she says.

Is AI coming for lawyers’ jobs?

Lawyer Gene Turner left his partnership at Buddle Findlay six years ago to set up LawHawk, a company that automates legal documents.

When you talk to lawyers about getting a machine to do anything they do, they’re usually deeply sceptical, he says.

“They’ll think of the hardest thing they’ve ever done, and it still can’t do that, so it’s useless.”

Turner doesn’t think AI will suddenly put lawyers out of business. Like other commentators, he thinks that, as accuracy improves, the public will use question-and-answer AI, such as ChatGPT, for some legal issues. And that could help bridge the gulf in access to justice.

“I think it’s going to need to be treated with some care, but if you believe some of the statistics, 80% of people that have legal problems don’t go to a lawyer. So it’s already probably going to fix a problem, which is, are you going to be better off relying on the AI to give you a pretty good steer in the right direction, or do you only get safety if you have to go to a lawyer?

“I think there’s a middle ground, where a bunch of people will probably be able to sort stuff out without needing to go to lawyers, because most stuff is not life or death.”

There will still be “huge scope” for lawyers in more complex situations, where you need to understand how people behave and what they’re trying to achieve, Turner says. But lawyers will use AI to do the boring stuff – admin, or providing the bones of an unfamiliar contract type.

And because lawyers usually use time-based charging, which provides no incentive to improve efficiency, that business model will have to change, Turner says.

“I think it will transform the whole business of law. There’s no doubt about that. I think law firms will have to move to subscription pricing.”

Hype and hyperbole?

Tesla boss Elon Musk has been promising fully self-driving cars since 2014. But he’s yet to deliver.

That’s what Whitehouse calls the “trough of disillusionment” that follows the kind of hype surrounding ChatGPT. Because as with self-driving cars, small errors can have huge implications.

Businesses often ask him, can his technology rewrite their contracts for them? Sure, but it might make mistakes and miss glaring things. And if you then have to reread and correct the whole contract, you’ve probably wasted more time than you’ve saved.

Instead, you have to understand what the AI can reliably do, and leave the rest for the humans, Whitehouse says.

“You’re trying to get the time savings and the improvement and the help, without doing damage, effectively. And you’re always fighting against those over-inflated expectations. Because everybody is trying to be really flashy. At the end of the day, you just want to be able to work with something that you trust.”