How to pick the right AI tool for criminal practice?

Within the legal industry, there have been growing conversations on Artificial Intelligence and the impact it can have on lawyer productivity and law firm profitability.

But, as we are at a critical moment in the evolution of the legal industry, it’s important for firms to truly understand AI, its limitations and its potential, before deciding on whether to implement it. In areas of law that deal with particularly sensitive information, such as criminal law, security comes first.

Particularly following the extensive conversation on the subject sparked by the cases of Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank , regulatory compliance is essential for criminal law firms.

When evaluating new technology, law firm managers should first assess its suitability, with those compliance needs in mind.

LEAP’s legal AI, for instance, is contained within a secure case management system, where clients’ information is securely stored rather than made available in an open-source format, unlike the more mainstream GPTs used in the UK. That matters, particularly in criminal law, because of the nature of the information we deal with, which is often sensitive or confidential.

Picking the right AI tool and the dangers of open-source AI

In a recent study, LEAP put the theory to the test by comparing LawY's accuracy with two of the most widely used GPTs. The results were astounding: LawY was found to be superior in accuracy, relevance, and, most importantly, security.

The biggest concern around the use of open-source tools and mainstream large language models (LLMs) linked to accuracy is “AI hallucinations”.

AI hallucinations are, in short, confident but false or misleading responses generated by artificial intelligence, particularly LLMs, which are designed to predict the next likely word in a sequence rather than checking facts.

If law firms were to use mainstream AI for document creation or review, they could end up making costly mistakes that would surface later in a case. That doesn’t just pose a risk to the fee earner’s case success, but it can also have wider implications for the law firm and fee earner, as was detailed extensively in the case of Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank – those implications included action by regulators, possible criminal offences and financial penalties to name but a few, not to mention the huge and often irreversible reputational damage.

On the other hand, when law firms adopt and enforce the exclusive use of purpose-built AI tools - designed to identify the right area of law and jurisdiction and programmed specifically not to hallucinate - that problem is eradicated.

Firm-wide education is crucial for a successful implementation

For an AI tool to successfully impact productivity and firm-wide profitability, it must be used appropriately. This is why education during the early stages of adoption is key.

As someone who has discovered the power of AI relatively recently, moving from a law firm to a legal tech company, I see it this way: The computer isn’t going to do our job for us, as it’s always going to need human input – either the human checks the AI product, or the human uses the AI product to check existing their own knowledge. We don’t want to be replaced by AI; we want to work with it.

But that’s easier said than done, given the sophistication of some of the products that can be created with AI. If, for example, a lawyer prompts AI to generate a legal document, what they are likely to receive is something that looks pretty much like the finished product, from its contents to its formatting. Often, when we draft a document from scratch, it is only in the very final stages of the process that it starts to look ‘finished’. But this is where the mindset shift needs to happen. Though it may look finished, it doesn’t mean it is, and it doesn’t remove the need to review the document.

When training legal teams to work with AI tools, law firms must reiterate the importance of treating them as support, not as tools to replace human judgment. Any document produced by AI should be treated as a first draft.

Many in the legal tech industry ask their users to consider AI as “an eager paralegal”. If you prompt AI to create a document for you, it will try to please you with the right information and will probably also get most of it in the right place, but you still need to check the draft. This is not only because you are the Lawyer and will likely have regulatory obligations you must comply with, but because you have human experience that a machine cannot replicate intuitively in the same way that a human can (at least not without further prompting and refinement - based on user input).

Think of the nuance of an advice letter to a client facing a lengthy prison sentence that they have not yet been told about. AI might correctly state the likely length of a sentence – the client, however, might not appreciate being informed in that way, and our human judgement may realise that that piece of information ought not to go directly into an advice or outcome letter without a prior conversation. We may also think of scenarios such as writing to clients at their home addresses as regards to sensitive offences, such as rape or sexual assault. Technically accurate, but won't do us any favours with a client.

Why Matter AI is an easy win for criminal lawyers

As part of my role at LEAP, I routinely test new AI features and functionalities to understand how they can make criminal lawyers' lives easier. For me, the standout feature is Matter AI, a tool that lets lawyers quickly ask a question about a file's contents and have the answer extracted at speed.

Like many lawyers working in Crime, I’ve been in court many times when a judge has asked a question about a case for which the answer isn’t immediately at my fingertips. Think of those situations where you are covering a case that isn’t yours, or where you have been asked to step into a hearing by a colleague who is delayed at the police station. Maybe the case has been dealt with, but a question about the client's bail position arises – with Matter AI, there is no more searching around whilst under pressure, as you are able to pop a question in and get an answer at speed. Being able to obtain information such as the client's bail history and the conditions they are subject to, including whether they have a history of breaching their bail or non-compliance with conditions, and to do so quickly, is phenomenally helpful.

Even on your own matters, Matter AI helps you double-check what has been completed and stay on top of your case preparation. Think of the ease of asking ‘Is there a rep order on this file and what date was it granted’ or ‘ Has the client paid any money on account?’ and getting a reply, instead of searching through the correspondence and documents on the file manually. It is also easy to check things like ‘When did we last have contact with the client?’ for the purposes of assessing inactivity and monitoring client care and service standards. In this way, Matter AI helps lawyers with case work and with staying on top of the business side of the job – like making sure we are actually getting paid for the work we undertake.

Is legal AI becoming “mandatory”?

I understand why people are cautious about the word ‘mandatory’, and no, there is no hard rule mandating that lawyers use AI on their cases - but I do think there’s an obligation that lawyers ought to feel to consider new ways of working to best help their clients.

As lawyers, it is well established that we owe numerous duties to our clients. It is also very apparent that being the subject of a criminal investigation or a prosecution costs our clients, emotionally, if not financially. If, as lawyers, we are taking a long time to get a document out to a client, or simply served on the other side because of being busy with multiple cases, or we aren’t able to respond to client enquiries immediately because we are engaged with other things, that has a toll on the client as well - they’re worrying whether the case is being progressed and whether or not they have instructed a lawyer who cares about them and their case.

If you know there’s a tool out there that’s going to skyrocket your efficiency, why, in a high-pressure, unpredictable and high-volume practice area such as crime, wouldn’t you use seek to use it, and how can we justify not doing so? As managers, knowing that there is a tool that could improve not just your team's productivity but also their work-life balance, is it really right to avoid adopting it?

With the publication of Part 2 of Leveson's Review of the Criminal Courts and the suggestions within that the Court, and perhaps more importantly, the Police and CPS ought to be using AI responsibly to help with case preparation, how can defence lawyers and/or private prosecutors really afford to ignore AI, and when does this become an equality of arms issue?

There’ll always be a small pocket of resistance, people who like their paper and pens. I get it – sometimes I, too, revert to those methods even now – but it’s not about abandoning old habits entirely. It’s about finding new ways of working to incorporate into our existing practice. It’s scary when we hear horror stories about compliance issues, data protection, and other pitfalls that often make the press. But if you get the right package that sorts that out for you and understand how easy it is to avoid those issues in the first place, then AI adoption really does become an obvious choice, if not quite yet mandatory.

Start small, then build

Naturally, as the legal industry is still in the early stages of legal AI adoption, law firms will have reservations about adopting new tools. My advice, as a lawyer first, is twofold.

To law firm managers, I say: research tools that would make your legal teams’ work easier and your firm more profitable, and test them on a small scale in a safe environment.

To lawyers, I say: Take your time researching the best AI tool for your area of law to ensure compliance, and once you have it, start small and build from there.

If you’re not confident, don’t start by using AI for drafting the most sensitive documents on your file.

Start with:

• Asking questions about what’s already in the file

• Generating chronologies

• Checking compliance

• Comparing documents

• Using AI to conduct file reviews

• Generating client success stories or articles for marketing purposes

And then, once you trust your process, use it for drafting, and remember it’s a first draft.

I am certain that AI won’t replace criminal lawyers, as the nature of our profession is incredibly human, multi-layered and nuanced. But law firms that adopt the right tools and use them effectively will lead the charge in the modern legal industry.

About the author

A seasoned Criminal and Regulatory Law specialist, Catherine Gaynor brings over a decade of experience to her role as Head of Criminal Law at LEAP Legal Software. Formerly a Partner at JMW Solicitors LLP, she has advised on high-profile criminal cases, VHCCs, and professional misconduct matters. At LEAP, she uses her deep understanding of the criminal justice sector to drive digital transformation and support firms in modernising their workflows.