And I expand my screen so you can see everything at one time and enjoy.
Let me zoom out a little bit so you get to see it all together.
Uh, here we are.
Enjoy these great results.
And of course, the action for you now is that the challenge is only beginning.
This is your opportunity now to beat me.
Crush me.
Come in and iterate.
Iterate.
This is the results of a matter of hours of me working on this.
You can do better.
The trick.
There's there's no.
There's no magic trick.
The trick is work.
The trick is iterating.
Measure, iterate, measure and see if you can't move the needle.
There's lots of things to try here.
There's some of the techniques that I had mentioned.
There's obviously doing more query expansion.
There's trying out different chunking strategies, changing the prompts, just simple stuff.
Changing the prompts makes so much difference.
Hierarchical rag.
Try and fix this problem with holistic.
I'm not sure.
Look into what's happening with numerical category.
That might be an easy win there.
And with holistic that's the one to fix with hierarchical rag, makes some summary documents and then
do a query over them and shove them in the prompt.
And then if you really want a bigger challenge, then the bigger challenge will be to do a gigantic
rag.
Let's just dwell for a second on that.
So a gigantic rag for those that want to take on this challenge.
And I have actually built a little version of it myself, but I'm not I'm not going to share it yet.
Make make this a challenge for you, uh, for people, uh, who are fairly new to Agentic AI.
The simplest thing you can do is just take the tools that we worked on in week two.
We already built tools, so you know how to add a tool to an LLM, have a tool that just allows it.
The last LLM call, uh, equip it with a tool so that it can search all of the files for a keyword if
it wishes.
And then you could just write a very simple, uh, utility that scans all the documents for a particular
string.
And if it finds it, then then return that document in its entirety to the LLM.
That would be a simple tool that would add a gigantic genetic rag just like that.
And I tried that and it didn't make a massive difference.
It did improve things, but not massively.
But that's a nice thing for you to try it yourself and see.
But for the proper agentic people who've taken my Agentic course, you can take this further in two
ways.
First of all, you can change the entire flow so that instead of doing a rag retrieval and then calling
the LLM, it calls the LLM, and it gives it a tool that allows it to come in and do a vector lookup.
And you could use MCP for that if you wanted very easily.
And you could not only give it a vector lookup, but also a file lookup, you could give it a number
of different ways that it can query the documents so that it can answer different questions, including
questions from any of these categories.
It would be pretty easy to set up tools to allow it to do any kind of search vector or graph or otherwise,
across all of that data.
But then there's more for the real agentic coders there.
Think about the other thing you could do.
Can you imagine the other thing you could do that would be like, like completely like earth shattering
for these metrics is you could add a tool or not.
Not a tool.
You could add a step which is calling the evaluator.
You could have it once it generates its output, you could evaluate it.
And if it's not scoring top marks for accuracy, completeness and relevance, you can kick it back and
say, hey, you scored poorly on the on the category of relevance.
Try again.
You didn't score well on completeness.
Try again.
And you could just keep going, keep iterating over it until it scores well and you could have it in
a loop.
So it just keeps going until it finally cracks it.
And if you do this, presumably you could build a system that might take a long time to run, but should
be scoring five out of five in all categories.
And then yeah, then then you should take a screenshot and put it on LinkedIn and put it everywhere
and it will go viral.
And that would be incredible.
That would be the the ultimate agentic solution.
And I would love to see it.
And I would love someone to be sharing their like four point nines across the board when you get there.
Of course, with the true genetic solution, you won't have retrieval stats because you're not really
doing retrieval in the same way.
It's a much more interactive style of retrieval.
So that's a great, exciting challenge for you.
A really fun assignment.
I can't wait to see people doing it.
If you haven't taken the genetic course, then don't worry about this super advanced genetic part,
but just play with all of the other aspects.
Play with hierarchical rag.
Um, play with with different models as well different LMS.
Try varying it to see what kinds of results you get.
And of course different encoders too.
There are lots of great open source encoders to try.
Okay.
And with that take great pride in our good stats here and I will see you back for the wrap up.
Well this is of course the end of the week.
It's the end of week five.
And so in addition to that little Agentic challenge or just general rag challenge to to beat me and
to get amazing scores, there is a bigger end of week assignment for you.
And that is now to go and apply your learning to a different field, to building a different knowledge
worker.
And it could be a knowledge worker for your business, because this is such a prototypical business
use case of generative AI.
You could also make a private knowledge worker to help you, something which is optimized on your own
personal data.
For example, you could put all of your files that you have on your Google Drive or on your local drive,
organize them and write something that brings them into a vector data store so that you can ask any
questions that pertains to documents you've written, information you've gathered over the years.
So you have your your own personal knowledge base at your fingertips.
You could do it all with open source models.
It'd be like a cool twist to it so that it doesn't cost a penny.
It's all running on your computer.
It also means everything is kept private.
If you're using open source models, nothing is going to the cloud.
Uh, and uh, and then make it so that it's a conversational AI that you can talk to with expert knowledge
of everything across your documents, even more than you remember.
That would be fantastic.
And then you can also take that even further.
And if you're, for example, someone that uses Google Workspace, Google has APIs so that you could
be accessing your own emails.
You can do it in a read only mode to avoid any any unexpected behavior.
And you can read your own emails and your Google Docs.
And you could do the same with Microsoft Office.
If you're a Microsoft user, and you could have it so that that's something which you're able to to
look over and and respond to your emails based on a full knowledge of everything that you've already
sent and you'd be able to build them.
Imagine visualizing the vector data store that is, the vector data store of the sum total of all of
your information.
Wouldn't that be cool?
That would be another.
That would be a great LinkedIn post showing that the vector store of you, and your full map of all
of the information that pertains to you.
Uh, so this is a great challenge.
Uh, I can't wait to see what you make of this.
I know a bunch of people have been have been working on this really interesting challenge.
the.
So the mandatory challenge is beating my my rag numbers.
This one is the stretch challenge.
Make a personal knowledge worker for you.
Make it vast.
Make it it's knowledge.
Be across everything you've done and then show it off on LinkedIn and let me come in and amplify your
success for you.
I can't wait to see that.
And with that, that brings us to the conclusion of week five.
What a week it's been.
You are 62.5% on your journey to being an LMN engineer, proficient engineer of all things large language
model.
And I think at this point, I hope that it now really feels like your 62.5% that this week was a big
week.
We really I feel like we crossed a line into the more advanced territory at this point.
The first couple of bullets here are like, yeah, yeah, yeah, you can generate text and code.
You get it, you can, you can pick the right LM, but now you can build advanced Rag solutions with,
with really, uh, like cutting edge techniques that are used in industry today.
But most importantly, I feel like you get the joke about how important it is to evaluate your performance
and to use that to put a sort of scientific framework around something that is more of an art form.
And that's a wrap on rag.
But as I said at the beginning, don't think, okay, then it's a wrap on on on on LM engineering.
I'll see you in another course.
Uh, no.
The next few weeks are really terrific.
They're really great.
It's the capstone project for the next few weeks.
It starts with a week.
That is.
That is a bit of a traditional data science week, but in a in a very juicy way, culminating in running
with LMS frontier models and fine tuning LMS.
We're going to cover the whole spectrum of machine learning, deep learning LMS as we try and solve
a problem, and that's going to dive into week seven and week eight.
It's going to be really, really great.
Get some rest.
Uh, complete the assignments, get some rest, and then I will see you for the beginning of week six
coming next.