Okay, so here we are.
We're going to write a better chat instead of returning bananas.
It's actually going to call OpenAI or whichever model you like and return an answer from the LLM.
But it's the same as far as Gradio is concerned.
It's the same callback.
It's calling.
It doesn't know that we're calling an LLM.
Okay.
So first up, there's just a little line here that is not necessary.
If you're using OpenAI.
It doesn't make any difference.
But it means that if you're using Gemini or some of the other models then it will definitely work.
And it is.
Do you remember I mentioned that that history variable that I showed you, it looked very similar to
OpenAI's format, but it also had some extra fields in there.
It had metadata.
And that's OpenAI doesn't care if you add in extra fields, but some of the models do.
So I'm just doing this one line here, which just replaces history with a new version of history that
only has role and content in it and clears out metadata or anything else.
And this makes no difference for OpenAI.
But if you're using Gemini, it means it will work rather than give you an error that it doesn't like
metadata in there.
So that's just a small point.
Look out for that.
I have it all over the place just so that it'll always work on Gemini too.
All right.
And now we should know this.
Well, we first need to to package everything together into the messages messages that we're going to
want to send to OpenAI.
And it's very simple.
I press tab.
This curse has done it for me.
Uh, basically we just want this list of dictionaries.
We want to start with role system content and the system message, because Gradio is just passing in
the contents of the UI.
So Gradio doesn't know that we have a system message.
So we need to put that in to start with and then put in all of the history that Gradio sent in, and
then end with the current user's prompt as role user content.
This message, you see, we take all the history and then we shove in one more message, which is the
user's message at the end.
And so this is now got all of the messages in it.
Great.
And now we can say response is OpenAI create model is model messages.
Is messages perfect.
And then finally we return response dot choices zero dot message dot content the chat completions API.
There we have it.
Let's run.
This is our new callback.
It looks great.
New update available.
I'll get that later.
Thank you.
Cursor.
Um, and now we will do um, chat interface that that same chat interface again and we'll see whether
now we get the same user interface.
Just just as easy as before.
Gradio doesn't know that anything has changed, except we've got this, this new callback.
And when we type in messages, we're hoping to see something better than bananas as the answer.
Here we go.
Are you ready for this?
I run it, up comes the chat and I'm gonna say hi there.
Drumroll.
Hello.
How can I assist you today?
That of course, coming back from GPT.
Hi there.
My name is editor.
Hello, editor.
Nice to meet you.
How can I help you today?
What's my name?
Your name is Ed.
How can I assist you further?
Ed?
All right, so I imagine you get it.
You're with me.
But just a really press this point home.
We asked Gradio to make a chat interface for us.
We gave it this callback function.
And so as a result, every time that the user presses something in here, like what's my name?
Uh, that is considered the message.
Everything so far is considered the history.
Gradio takes those two bits of information and it automatically calls the callback chat, passing in
the message passing in the history.
And this is in OpenAI format or very close to it.
We scrub the format to make it just right, only needed for Gemini and a couple of others for grok as
well, I think.
And then messages, uh, we construct exactly the right messages, including shoving the system prompt
at the top, we call OpenAI.
We return the response from OpenAI.
So instead of bananas, what comes back and appears in the screen is actually the response from the
model.
I know you've got it already, but I like to talk.
All right, all right, let's move on.
Let's do more.
Okay.
So I'm I'm now going to update that chat callback to do streaming as well.
And so this time it's exactly the same as before.
But I've got OpenAI ActionScript and I'm passing in stream equals true.
And then I can just simply put here for chunk in stream.
Remember that we're just going to iterate over what comes back.
And then we're going to get the next chunk of what comes back, accumulate it in this variable response
and yield response each time so that gradio it will see that this isn't a function anymore.
It's a generator, a special kind of function.
It's a function that you can iterate over.
And as a result, Gradio will automatically not just wait until it's finished and show the results,
but iterate over it and surely, surely slowly show the response as it comes.
Surely.
Uh, all right.
Uh, then let's just bring this up and see what we get.
Here it is.
Uh, so now I can say hi there again.
Let's try.
Hi there.
And you can see that it came streaming.
And let's make it something, uh, explain the pros and cons of a genetic, uh, architecture, uh,
genetic AI, uh, for commercial purposes.
Let's do that and see what we get.
Is this something that we're having going off on a on a big old diatribe?
There you go.
And you can see it streaming away.
And it's even gradio nicely handles, uh, markdown coming back out of the box, uh, without needing
to use markdowns or anything this time.
Looks great.
Uh, isn't that easy?
Isn't it?
Wasn't it easy to build like an interactive chatbot?