Putting an LLM in front of lawyers

What we were actually solving

A great deal of what a firm produces is a short list of documents written over and over. The same shape every time, entirely different content every time. Turnaround on the first one we took on ran about two days, from the file arriving to the document going out.

That is a drafting problem, and it is not the same as a chatbot problem. The difference decided everything that followed.

A general assistant is judged on whether the answer was useful. A drafting assistant is judged on whether the attorney whose name goes on the document is willing to sign it. That is a much higher bar and a far less forgiving one, and very little of what gets written about building AI chat tools survives contact with it.

The model never sees the client's name

Client files are about as sensitive as documents get. Names, personal histories, financial and medical detail, the facts of somebody's case.

So none of it goes to the model.

Everything on its way in passes through a service that finds the identifying details and swaps them for placeholders. The AI drafts against the placeholders. On the way back out, the same service puts the real names in where the placeholders were.

The attorney reads a document about their client. The model only ever saw a document about nobody in particular.

That still leaves the question of where the confidentiality boundary belongs. In what we built, the application decided who could see which documents. I have come to think it should sit one level lower, in the data layer itself, so that no mistake in the application can widen who sees what. I published a working version of that approach for anyone who wants the detail.

Teaching it to stop and ask

Ask an AI to draft and it will draft. Give it half the facts and you get back something confident, complete and wrong, which is worse than getting nothing, because now somebody has to find the mistake before they can fix it.

So we taught it to stop.

Before it writes anything it checks whether it actually has what that kind of document requires. If something is missing it does not guess, does not leave a placeholder, does not draft and flag. It asks one question and waits for the answer.

That single behaviour did more for trust than any improvement in the quality of the writing. Lawyers do not need a tool that is right most of the time. They need one that knows which time this is.

The week it kept quoting its own instructions

This one cost me a week.

The assistant learned house style from approved past work. Before drafting, it would work out what a strong document of that type looks like, and carry that forward into the writing.

And then it started printing that at the top of the letter.

Not using it. Printing it. A tidy little summary of how to write a good document, sitting above the actual document, in some share of the drafts.

I tried everything you would try. Moved it elsewhere in the instructions. Rewrote it as flowing prose instead of a list. Restructured it completely. Told the assistant, politely and then a good deal less politely, never to reproduce it. It kept coming back, once with an apology attached to the front of it.

What worked was giving up on telling it anything. Rather than hand the assistant a block of guidance and ask it not to repeat the guidance, we broke that block apart and folded each piece into the instruction it belonged to. There was no longer a passage sitting there to be copied.

Getting lawyers to use it

A drafting tool nobody opens changes nothing, and legal teams have good reasons to be sceptical of software that produces confident prose.

So the rollout was not an announcement. It went out department by department across a week, each group shown its own work rather than a general tour, because the ten minutes that convince a marketing team are not the ten minutes that convince an attorney. Every session named the failure modes out loud. People trust a tool more, not less, when you tell them where it breaks.

Those numbers exist only because turnaround was measured before any of it shipped. Without a baseline the whole thing would have been defended on impressions, and a project defended on impressions loses the first budget review it walks into.

What I would do differently

1. Agree what happens when a safeguard fails before launch, not after. It is a short conversation early and an awkward one late.
2. Check who can see what using two different accounts on day one. One person testing their own access will find nothing wrong, right up until it matters.
3. Assume anything you show the model can end up in the output, and design so that outcome is harmless instead of trying to instruct your way around it.
4. Put the automated checks in earlier. The parts of a system like this that are predictable enough to test are also the parts everything else rests on. I got to that later than I should have.