1
00:00:00,160 --> 00:00:05,320
And I expand my screen so you can see everything at one time and enjoy.

2
00:00:05,360 --> 00:00:08,280
Let me zoom out a little bit so you get to see it all together.

3
00:00:08,560 --> 00:00:09,960
Uh, here we are.

4
00:00:10,280 --> 00:00:12,640
Enjoy these great results.

5
00:00:12,840 --> 00:00:18,320
And of course, the action for you now is that the challenge is only beginning.

6
00:00:18,480 --> 00:00:20,680
This is your opportunity now to beat me.

7
00:00:20,720 --> 00:00:21,680
Crush me.

8
00:00:21,840 --> 00:00:22,880
Come in and iterate.

9
00:00:22,880 --> 00:00:23,240
Iterate.

10
00:00:23,240 --> 00:00:26,760
This is the results of a matter of hours of me working on this.

11
00:00:26,800 --> 00:00:28,160
You can do better.

12
00:00:28,200 --> 00:00:28,960
The trick.

13
00:00:29,000 --> 00:00:29,720
There's there's no.

14
00:00:29,760 --> 00:00:30,840
There's no magic trick.

15
00:00:31,080 --> 00:00:32,960
The trick is work.

16
00:00:33,000 --> 00:00:34,440
The trick is iterating.

17
00:00:34,440 --> 00:00:38,400
Measure, iterate, measure and see if you can't move the needle.

18
00:00:38,400 --> 00:00:40,480
There's lots of things to try here.

19
00:00:40,520 --> 00:00:42,200
There's some of the techniques that I had mentioned.

20
00:00:42,240 --> 00:00:44,640
There's obviously doing more query expansion.

21
00:00:44,680 --> 00:00:49,320
There's trying out different chunking strategies, changing the prompts, just simple stuff.

22
00:00:49,320 --> 00:00:51,320
Changing the prompts makes so much difference.

23
00:00:51,480 --> 00:00:52,920
Hierarchical rag.

24
00:00:52,960 --> 00:00:55,320
Try and fix this problem with holistic.

25
00:00:55,360 --> 00:00:55,840
I'm not sure.

26
00:00:56,000 --> 00:00:57,720
Look into what's happening with numerical category.

27
00:00:57,720 --> 00:00:59,280
That might be an easy win there.

28
00:00:59,320 --> 00:01:05,020
And with holistic that's the one to fix with hierarchical rag, makes some summary documents and then

29
00:01:05,060 --> 00:01:07,420
do a query over them and shove them in the prompt.

30
00:01:07,980 --> 00:01:13,980
And then if you really want a bigger challenge, then the bigger challenge will be to do a gigantic

31
00:01:13,980 --> 00:01:14,460
rag.

32
00:01:14,700 --> 00:01:16,780
Let's just dwell for a second on that.

33
00:01:16,900 --> 00:01:19,980
So a gigantic rag for those that want to take on this challenge.

34
00:01:19,980 --> 00:01:24,060
And I have actually built a little version of it myself, but I'm not I'm not going to share it yet.

35
00:01:24,100 --> 00:01:29,700
Make make this a challenge for you, uh, for people, uh, who are fairly new to Agentic AI.

36
00:01:29,780 --> 00:01:34,700
The simplest thing you can do is just take the tools that we worked on in week two.

37
00:01:34,860 --> 00:01:41,820
We already built tools, so you know how to add a tool to an LLM, have a tool that just allows it.

38
00:01:41,860 --> 00:01:48,860
The last LLM call, uh, equip it with a tool so that it can search all of the files for a keyword if

39
00:01:48,860 --> 00:01:49,620
it wishes.

40
00:01:49,620 --> 00:01:54,900
And then you could just write a very simple, uh, utility that scans all the documents for a particular

41
00:01:54,900 --> 00:01:55,420
string.

42
00:01:55,460 --> 00:02:00,580
And if it finds it, then then return that document in its entirety to the LLM.

43
00:02:00,780 --> 00:02:05,160
That would be a simple tool that would add a gigantic genetic rag just like that.

44
00:02:05,520 --> 00:02:07,880
And I tried that and it didn't make a massive difference.

45
00:02:07,880 --> 00:02:09,960
It did improve things, but not massively.

46
00:02:10,160 --> 00:02:13,320
But that's a nice thing for you to try it yourself and see.

47
00:02:13,640 --> 00:02:19,640
But for the proper agentic people who've taken my Agentic course, you can take this further in two

48
00:02:19,680 --> 00:02:20,240
ways.

49
00:02:20,360 --> 00:02:25,600
First of all, you can change the entire flow so that instead of doing a rag retrieval and then calling

50
00:02:25,600 --> 00:02:31,080
the LLM, it calls the LLM, and it gives it a tool that allows it to come in and do a vector lookup.

51
00:02:31,080 --> 00:02:34,760
And you could use MCP for that if you wanted very easily.

52
00:02:34,760 --> 00:02:39,200
And you could not only give it a vector lookup, but also a file lookup, you could give it a number

53
00:02:39,200 --> 00:02:44,600
of different ways that it can query the documents so that it can answer different questions, including

54
00:02:44,600 --> 00:02:46,560
questions from any of these categories.

55
00:02:46,560 --> 00:02:53,160
It would be pretty easy to set up tools to allow it to do any kind of search vector or graph or otherwise,

56
00:02:53,200 --> 00:02:54,640
across all of that data.

57
00:02:55,480 --> 00:02:59,280
But then there's more for the real agentic coders there.

58
00:02:59,480 --> 00:03:01,480
Think about the other thing you could do.

59
00:03:02,040 --> 00:03:09,180
Can you imagine the other thing you could do that would be like, like completely like earth shattering

60
00:03:09,180 --> 00:03:13,220
for these metrics is you could add a tool or not.

61
00:03:13,220 --> 00:03:13,620
Not a tool.

62
00:03:13,620 --> 00:03:18,140
You could add a step which is calling the evaluator.

63
00:03:18,140 --> 00:03:22,220
You could have it once it generates its output, you could evaluate it.

64
00:03:22,220 --> 00:03:27,900
And if it's not scoring top marks for accuracy, completeness and relevance, you can kick it back and

65
00:03:27,900 --> 00:03:31,740
say, hey, you scored poorly on the on the category of relevance.

66
00:03:31,740 --> 00:03:32,620
Try again.

67
00:03:32,660 --> 00:03:34,300
You didn't score well on completeness.

68
00:03:34,300 --> 00:03:35,060
Try again.

69
00:03:35,260 --> 00:03:40,500
And you could just keep going, keep iterating over it until it scores well and you could have it in

70
00:03:40,500 --> 00:03:40,740
a loop.

71
00:03:40,740 --> 00:03:43,620
So it just keeps going until it finally cracks it.

72
00:03:43,620 --> 00:03:49,140
And if you do this, presumably you could build a system that might take a long time to run, but should

73
00:03:49,140 --> 00:03:52,060
be scoring five out of five in all categories.

74
00:03:52,060 --> 00:03:56,660
And then yeah, then then you should take a screenshot and put it on LinkedIn and put it everywhere

75
00:03:56,660 --> 00:03:57,500
and it will go viral.

76
00:03:57,500 --> 00:03:58,860
And that would be incredible.

77
00:03:58,860 --> 00:04:02,380
That would be the the ultimate agentic solution.

78
00:04:02,380 --> 00:04:03,620
And I would love to see it.

79
00:04:03,620 --> 00:04:09,080
And I would love someone to be sharing their like four point nines across the board when you get there.

80
00:04:09,120 --> 00:04:12,760
Of course, with the true genetic solution, you won't have retrieval stats because you're not really

81
00:04:12,760 --> 00:04:14,120
doing retrieval in the same way.

82
00:04:14,240 --> 00:04:16,760
It's a much more interactive style of retrieval.

83
00:04:16,760 --> 00:04:19,480
So that's a great, exciting challenge for you.

84
00:04:19,640 --> 00:04:20,960
A really fun assignment.

85
00:04:21,000 --> 00:04:22,480
I can't wait to see people doing it.

86
00:04:22,520 --> 00:04:27,040
If you haven't taken the genetic course, then don't worry about this super advanced genetic part,

87
00:04:27,040 --> 00:04:29,720
but just play with all of the other aspects.

88
00:04:29,720 --> 00:04:31,200
Play with hierarchical rag.

89
00:04:31,280 --> 00:04:34,640
Um, play with with different models as well different LMS.

90
00:04:34,680 --> 00:04:37,440
Try varying it to see what kinds of results you get.

91
00:04:37,440 --> 00:04:39,280
And of course different encoders too.

92
00:04:39,320 --> 00:04:41,960
There are lots of great open source encoders to try.

93
00:04:42,200 --> 00:04:42,720
Okay.

94
00:04:42,960 --> 00:04:48,760
And with that take great pride in our good stats here and I will see you back for the wrap up.

95
00:04:48,880 --> 00:04:50,680
Well this is of course the end of the week.

96
00:04:50,680 --> 00:04:51,960
It's the end of week five.

97
00:04:51,960 --> 00:04:57,680
And so in addition to that little Agentic challenge or just general rag challenge to to beat me and

98
00:04:57,680 --> 00:05:03,160
to get amazing scores, there is a bigger end of week assignment for you.

99
00:05:03,320 --> 00:05:10,110
And that is now to go and apply your learning to a different field, to building a different knowledge

100
00:05:10,110 --> 00:05:10,750
worker.

101
00:05:10,750 --> 00:05:15,830
And it could be a knowledge worker for your business, because this is such a prototypical business

102
00:05:15,830 --> 00:05:17,270
use case of generative AI.

103
00:05:17,470 --> 00:05:24,590
You could also make a private knowledge worker to help you, something which is optimized on your own

104
00:05:24,630 --> 00:05:25,830
personal data.

105
00:05:25,950 --> 00:05:31,310
For example, you could put all of your files that you have on your Google Drive or on your local drive,

106
00:05:31,390 --> 00:05:36,710
organize them and write something that brings them into a vector data store so that you can ask any

107
00:05:36,710 --> 00:05:41,830
questions that pertains to documents you've written, information you've gathered over the years.

108
00:05:41,830 --> 00:05:45,390
So you have your your own personal knowledge base at your fingertips.

109
00:05:45,390 --> 00:05:46,870
You could do it all with open source models.

110
00:05:46,870 --> 00:05:49,870
It'd be like a cool twist to it so that it doesn't cost a penny.

111
00:05:49,910 --> 00:05:51,190
It's all running on your computer.

112
00:05:51,190 --> 00:05:52,910
It also means everything is kept private.

113
00:05:52,910 --> 00:05:55,870
If you're using open source models, nothing is going to the cloud.

114
00:05:55,990 --> 00:06:02,590
Uh, and uh, and then make it so that it's a conversational AI that you can talk to with expert knowledge

115
00:06:02,590 --> 00:06:05,630
of everything across your documents, even more than you remember.

116
00:06:05,870 --> 00:06:07,350
That would be fantastic.

117
00:06:07,630 --> 00:06:10,490
And then you can also take that even further.

118
00:06:10,490 --> 00:06:16,210
And if you're, for example, someone that uses Google Workspace, Google has APIs so that you could

119
00:06:16,210 --> 00:06:18,570
be accessing your own emails.

120
00:06:18,570 --> 00:06:23,610
You can do it in a read only mode to avoid any any unexpected behavior.

121
00:06:23,610 --> 00:06:26,370
And you can read your own emails and your Google Docs.

122
00:06:26,890 --> 00:06:28,450
And you could do the same with Microsoft Office.

123
00:06:28,490 --> 00:06:34,530
If you're a Microsoft user, and you could have it so that that's something which you're able to to

124
00:06:34,570 --> 00:06:41,850
look over and and respond to your emails based on a full knowledge of everything that you've already

125
00:06:41,890 --> 00:06:44,130
sent and you'd be able to build them.

126
00:06:44,170 --> 00:06:49,690
Imagine visualizing the vector data store that is, the vector data store of the sum total of all of

127
00:06:49,690 --> 00:06:50,810
your information.

128
00:06:50,850 --> 00:06:51,850
Wouldn't that be cool?

129
00:06:51,850 --> 00:06:52,530
That would be another.

130
00:06:52,530 --> 00:06:58,090
That would be a great LinkedIn post showing that the vector store of you, and your full map of all

131
00:06:58,090 --> 00:06:59,970
of the information that pertains to you.

132
00:07:00,210 --> 00:07:02,770
Uh, so this is a great challenge.

133
00:07:02,890 --> 00:07:05,450
Uh, I can't wait to see what you make of this.

134
00:07:05,490 --> 00:07:10,050
I know a bunch of people have been have been working on this really interesting challenge.

135
00:07:10,270 --> 00:07:10,950
the.

136
00:07:10,990 --> 00:07:14,910
So the mandatory challenge is beating my my rag numbers.

137
00:07:14,910 --> 00:07:16,950
This one is the stretch challenge.

138
00:07:16,990 --> 00:07:20,030
Make a personal knowledge worker for you.

139
00:07:20,190 --> 00:07:21,390
Make it vast.

140
00:07:21,430 --> 00:07:22,550
Make it it's knowledge.

141
00:07:22,590 --> 00:07:28,870
Be across everything you've done and then show it off on LinkedIn and let me come in and amplify your

142
00:07:28,870 --> 00:07:30,110
success for you.

143
00:07:30,390 --> 00:07:31,510
I can't wait to see that.

144
00:07:31,550 --> 00:07:36,030
And with that, that brings us to the conclusion of week five.

145
00:07:36,110 --> 00:07:37,510
What a week it's been.

146
00:07:37,510 --> 00:07:46,230
You are 62.5% on your journey to being an LMN engineer, proficient engineer of all things large language

147
00:07:46,230 --> 00:07:46,710
model.

148
00:07:46,870 --> 00:07:53,950
And I think at this point, I hope that it now really feels like your 62.5% that this week was a big

149
00:07:53,950 --> 00:07:54,390
week.

150
00:07:54,390 --> 00:08:00,230
We really I feel like we crossed a line into the more advanced territory at this point.

151
00:08:00,270 --> 00:08:03,470
The first couple of bullets here are like, yeah, yeah, yeah, you can generate text and code.

152
00:08:03,470 --> 00:08:09,470
You get it, you can, you can pick the right LM, but now you can build advanced Rag solutions with,

153
00:08:09,470 --> 00:08:14,730
with really, uh, like cutting edge techniques that are used in industry today.

154
00:08:14,730 --> 00:08:21,250
But most importantly, I feel like you get the joke about how important it is to evaluate your performance

155
00:08:21,250 --> 00:08:27,170
and to use that to put a sort of scientific framework around something that is more of an art form.

156
00:08:27,370 --> 00:08:29,770
And that's a wrap on rag.

157
00:08:30,410 --> 00:08:35,450
But as I said at the beginning, don't think, okay, then it's a wrap on on on on LM engineering.

158
00:08:35,490 --> 00:08:37,050
I'll see you in another course.

159
00:08:37,210 --> 00:08:38,490
Uh, no.

160
00:08:38,530 --> 00:08:40,770
The next few weeks are really terrific.

161
00:08:40,810 --> 00:08:42,130
They're really great.

162
00:08:42,170 --> 00:08:44,570
It's the capstone project for the next few weeks.

163
00:08:44,570 --> 00:08:46,330
It starts with a week.

164
00:08:46,530 --> 00:08:47,130
That is.

165
00:08:47,170 --> 00:08:52,370
That is a bit of a traditional data science week, but in a in a very juicy way, culminating in running

166
00:08:52,370 --> 00:08:55,730
with LMS frontier models and fine tuning LMS.

167
00:08:55,770 --> 00:09:01,010
We're going to cover the whole spectrum of machine learning, deep learning LMS as we try and solve

168
00:09:01,010 --> 00:09:04,250
a problem, and that's going to dive into week seven and week eight.

169
00:09:04,290 --> 00:09:06,050
It's going to be really, really great.

170
00:09:06,330 --> 00:09:07,450
Get some rest.

171
00:09:07,610 --> 00:09:12,810
Uh, complete the assignments, get some rest, and then I will see you for the beginning of week six

172
00:09:12,850 --> 00:09:13,610
coming next.