0:05 – Dave York

Welcome to the fifth edition of TCDI Talks. I’m Dave York with TCDI, joined by Caragh Landry at TCDI and David Gaskey, Altumatim. Welcome guys.

0:15 – David Gaskey

Thank You

0:16 – Caragh Landry

We’re here at LitForward, which is TCDI’s think tank where we’re discussing all things AI. So, we thought we would do it together today.

0:27 – David Gaskey

Yeah, something different.

0:28 – Dave York

So, today’s topic, everybody’s favorite: prompting. So, David, at a high level, what is prompting? What’s considered prompting? What’s not considered prompting? Because I think a lot of things get lumped under the auspice of prompt.

0:45 – David Gaskey

Sure. So, I think at the basic level, prompting is an instruction to an LLM to perform something. You can think of it like computer code. I mean, computers have certain capabilities. Good code will give the computer instructions to do better things.

Similarly with prompts, a good prompt will instruct the LLM to do something better. So, when you’re prompting an LLM, you are actually giving it instructions and telling it what to do. Computers only do what we tell them to do. So, if you don’t put a good prompt in, don’t expect a good result out.

1:26 – Dave York

What are some good recommendations? I mean, we work in…we work in an industry that is very verbose in how we write things. I imagine that probably doesn’t work well when you’re typing up a prompt.

1:40 – David Gaskey

It doesn’t. Actually, our experience has been looking at…seeing some prompts from different people, different legal professionals, whether they’re attorneys, paralegals, you know, service providers. And most people tend to…they’re in their comfort zone.

Like, if you write requests for production as part of your normal practice. The way you write that is typically designed so that the attorneys on the other side cannot escape giving you what you want them to give you. If you use that type of language like “including but not limited to,” and throw in all those phrases that lawyers love. That confuses the computer. It confuses the LLM, I should say. So, you definitely don’t want to just go with that. You have to think more like a programmer to some extent.

2:33 – Caragh

So, I wasn’t that bad with the “included but limited to,” but I definitely had paragraphs for my prompts. And David’s feedback has always been to make it briefer. Make it more concise. Get rid of all the fluff. So, that’s been great advice I think.

2:49 – David Gaskey

Yes, yeah, definitely. You want to get rid of the fluff. You want to be clear. You want to be concise. You need to be consistent as well.

I’ve seen where people will…they’ll use one term on line one, and then you get to line four of their prompt, and it’s a different word. Well, you might think that’s the same thing, but you’re dealing with a machine. You really need to guide it carefully if you want good results. Some LLMs will actually pick up on, “ok, I think you mean the same thing,” and it’ll go with that. But there’s no guarantee.

3:19 – Caragh Landry

So, that can actually explain…we’re working on a project right now where we’re doing issue coding, and we were getting amazing results. Like 90 recall, 90 precision. Like high 90s recall, low 90s precision which, as we all know, is amazing.

But there was one or two issue codes where it was not. It wasn’t doing as well. It was maybe in the 70s for precision. Which is still much better than human review, but based on what you just said, this was a foreign language review. There were many words. Many…there’s many foreign language words for some of the topics, and I think we did use multiple. Or maybe in the length, maybe in the documents themselves, they were referring to it in different ways.

4:05 – David Gaskey

Yeah, and what was interesting about that is that there were two issue codes where the precision was lower than the rest. One was a very short description of the issue. The other one was very long.

So, you…you really kind of have to hit that sweet spot in between. The other thing about that is those criteria that we were using, that’s not the actual prompt. So, the way we use input from a customer, is that gets fed into what I like to call our prompting engine.

So, we dynamically generate prompts based on the input. Like for example, if you’re using Emmet in Altumatim, you ask a question. The question is not the prompt. It’s a feed to the system to then generate a prompt based on your question. So, there’s a lot of things that happen in the background. To actually execute, [and] get to the result that you’re looking for.

5:03 – Dave York

Oh, I’m sorry. Go ahead.

5:04 – David Gaskey

Go ahead.

5:05 – Dave York

Oh, I was gonna say…I imagine that’s very similar to a documents database, and that people are used to having pretty field labels and information they plug in. But if you look at the raw text of what the database is actually doing to execute that search…looks vastly different. Probably very similar.

5:26 – David Gaskey

I think about that like you as a user, you’re going in and selecting fields. Well, there’s code that the machine is executing to say, oh, you want stuff with this field and then it does its job. Yeah, so very similar to that.

I think a lot of people think of asking a question like in a ChatGPT interface. Yes, that is prompting in some respect, but when it comes to more complex things, you need more than just a question to get consistent good results.

So, I mean, there’s so many techniques that can be used to accomplish good prompt engineering. The better prompting you do, again, the better results you get. The techniques that are talked about a lot…like for example, chain of thought prompting, where you actually lead the LLM through a series of steps. That has been shown to enhance the reasoning capability of the LLM. It can actually follow and get to better results doing that.

Then people have expanded from chain of thought to tree of thoughts, plural, where you actually create a series of branches, and the machine that the LLM can choose which branch it’s going to go down. Even beyond that is what’s called plan-based property where you present the LLM with a problem and tell it to create a plan and then go solve it. So, there’s a lot that happens behind the scenes that the typical user has no idea is really going on. If something is sophisticated and good results.

6:59 – Caragh Landry

So, from a user perspective, what does that mean? Like, as a front-end person knowing that all of that is going on. Do I still just need to be concise and to the point? And then all of that happens behind the scenes, and then I do validation and then I build off of that?

7:14 – David Gaskey

Yes, yeah. You still want to follow basic rules of…yeah, clear, concise, consistency. The three C’s. We could call them three C’s of Good Prompting for that…for that user input.

7:28 – Dave York

Trademark them.

7:29 – David Gaskey

Yeah. But it’s like doing a search in Google in a sense. Like you have no idea what Google’s algorithm is. It’s not just doing a keyword search. It’s doing all kinds of stuff in the back end. But putting in the wrong words, or putting in words that don’t really describe what you’re looking for, you’re never gonna find, you know, the results.

So, from a user perspective, yeah, I think keep it simple. By having users, at least that’s the way we’ve designed our system, let the users do what they’re comfortable doing, and then we guide them into the clear, concise, consistent framework. But the real magic is happening with all that dynamic prompting on the back end.

8:15 – Caragh Landry

Cool. I think that answers the basics of “what is prompt engineering?” Hopefully that was a lot…hopefully that clears up a lot. I like clear, concise, consistent. I can live with that.

8:25 – David York

I believe the session was clear, concise, and consistent.

8:29 – David Gaskey

I don’t know if it was.

8:31 – Caragh Landry

We’ll see.

8:32 – Dave York

I know it was.

8:33 – Caragh Landry

All right, well, thank you for watching, and we will see you in the next TCDI Talks.

8:38 – Dave York

Bye now.

8:39 – Caragh Landry

Bye.

8:39 – David Gaskey

See you.