1 00:00:00,080 --> 00:00:05,120 And so here we are back in cursor and in the number five folder for Autogen. 2 00:00:05,120 --> 00:00:08,560 And going to lab for Autogen Core Distributed. 3 00:00:08,560 --> 00:00:14,560 I'm only giving a teaser of what this is about because, uh, I'm to be honest, I'm I'm not even sure 4 00:00:14,560 --> 00:00:16,960 how relevant this is to, to many, many of you. 5 00:00:17,000 --> 00:00:19,400 So I, you know, I'm going to give a flavor for it. 6 00:00:19,400 --> 00:00:22,800 If you do want me to go into more detail, if you want more content, then please do. 7 00:00:23,040 --> 00:00:25,240 Do, uh, send me a note, let me know. 8 00:00:25,280 --> 00:00:29,600 I can I can always go deeper into this, but I don't want to hold, uh, others up. 9 00:00:29,840 --> 00:00:35,960 Um, and, um, I think that particularly as Microsoft is saying, that it's, it's still experimental. 10 00:00:35,960 --> 00:00:37,920 It's better to get a flavor for it. 11 00:00:37,960 --> 00:00:38,440 Okay. 12 00:00:38,600 --> 00:00:44,120 So I've done this and there's actually two different ways that I want to show you this working. 13 00:00:44,360 --> 00:00:47,320 Uh, one of them is what I'm calling all in one worker. 14 00:00:47,360 --> 00:00:49,560 And one of them is with multiple workers. 15 00:00:49,720 --> 00:00:53,400 And so I've got this flag here that you can set from true to false. 16 00:00:53,400 --> 00:00:56,360 And we're going to go through it once with it true, and then once with it at false. 17 00:00:56,360 --> 00:00:57,960 And you'll see what I mean. 18 00:00:57,960 --> 00:01:01,680 So we do some imports and we load the dot env. 19 00:01:02,040 --> 00:01:08,970 And we start with our message, uh, class as as before, which I'm going to clear all outputs and restart. 20 00:01:09,010 --> 00:01:09,690 Do this again. 21 00:01:09,690 --> 00:01:10,130 Sorry. 22 00:01:10,570 --> 00:01:10,730 Uh. 23 00:01:10,730 --> 00:01:11,530 Run this. 24 00:01:11,810 --> 00:01:14,970 I start as before, with our message class defining this thing. 25 00:01:14,970 --> 00:01:20,810 It's the sort of analogy to the state with with Landgraf, although it's to to describe how we interact 26 00:01:20,810 --> 00:01:22,970 between our agents. 27 00:01:23,250 --> 00:01:23,770 Okay. 28 00:01:24,090 --> 00:01:28,650 And now I mentioned that the distributed runtime consists of these two things. 29 00:01:28,650 --> 00:01:30,130 One of them is the host. 30 00:01:30,130 --> 00:01:41,170 And this is how you create the host Autogen runtimes, gRPC, the gRPC worker agent runtime host. 31 00:01:41,170 --> 00:01:49,330 So this is something which uses gRPC, the remote procedure called technique to be able to to send messages. 32 00:01:49,330 --> 00:01:55,130 And this host will run on my local host on port 551. 33 00:01:55,130 --> 00:01:57,210 And I will start it with this. 34 00:01:57,210 --> 00:02:00,090 So this is now going to be a running host. 35 00:02:00,090 --> 00:02:08,580 And gRPC of course, is a cross-language, uh, approach for sending, uh, A function calls between 36 00:02:08,580 --> 00:02:11,660 different languages, and it's super powerful. 37 00:02:11,660 --> 00:02:14,860 It's used in many different, different places. 38 00:02:14,860 --> 00:02:16,060 You can think of it. 39 00:02:16,100 --> 00:02:23,220 It's like it's like making rest HTTP calls, except you're able to call directly from one function to 40 00:02:23,260 --> 00:02:30,780 another, and gRPC is used all over the place where, uh, where interactive messaging needs to be implemented. 41 00:02:30,780 --> 00:02:32,900 That crosses process boundaries. 42 00:02:33,420 --> 00:02:34,660 So that is gRPC. 43 00:02:34,700 --> 00:02:37,820 We've started a host and it is running. 44 00:02:38,060 --> 00:02:42,380 And what we're now going to do is, first of all, reintroduce an old friend. 45 00:02:42,420 --> 00:02:48,420 We're going to bring back the Autogen Sirpa tool using our Sirpa search API. 46 00:02:48,860 --> 00:02:53,540 And the reason we're the way we're doing it is through we're we're doing it through Lang Chain. 47 00:02:53,540 --> 00:03:00,060 So we're introducing a few things from, from before, uh, so we uh, we create the Google Serp API 48 00:03:00,060 --> 00:03:00,620 wrapper. 49 00:03:00,620 --> 00:03:04,540 We create a Lang chain tool for internet search. 50 00:03:04,740 --> 00:03:11,980 Uh, this is the same tool that we used last week, and now we wrap that in a long chain tool wrapper 51 00:03:12,180 --> 00:03:14,660 so that it becomes an autogen tool. 52 00:03:14,940 --> 00:03:21,060 And there it is, an autogen tool for searching the internet using the Sirpa API. 53 00:03:21,420 --> 00:03:25,980 So originally I was going to have this play rock paper, scissors in a distributed way, but I realized 54 00:03:25,980 --> 00:03:31,100 that it's a bit frivolous and we should be trying to at least have some commercial footing here. 55 00:03:31,100 --> 00:03:35,260 So I was going to make it do the the stock price comparison. 56 00:03:35,260 --> 00:03:38,500 And I thought, you know, we've already done some, some of them and we've got a whole week coming 57 00:03:38,500 --> 00:03:38,860 on that. 58 00:03:38,860 --> 00:03:43,460 So instead I've gone with this trying to we're trying to make a business decision. 59 00:03:43,660 --> 00:03:50,100 And let's say that business decision is whether we should use Autogen in a new AI agent project. 60 00:03:50,300 --> 00:03:58,860 And we want to have a two different agents, one agent research the pros of autogen by using web searches 61 00:03:58,860 --> 00:04:03,420 and the other agent research the cons, the negatives, the drawbacks. 62 00:04:03,700 --> 00:04:09,460 Uh, and so the two agents will go off and do their analysis, do their searching, and then they will 63 00:04:09,460 --> 00:04:15,030 come together and we will have a judge agent that must make a decision whether to use Autogen for a 64 00:04:15,030 --> 00:04:15,710 project. 65 00:04:15,950 --> 00:04:23,070 Uh, and, uh, it must be based purely on research from its team, from its agent team. 66 00:04:23,230 --> 00:04:25,150 Respond with the decision. 67 00:04:25,190 --> 00:04:26,270 Brief rationale. 68 00:04:26,510 --> 00:04:29,550 So this is sort of by analogy with the rock paper, scissors. 69 00:04:29,550 --> 00:04:33,510 We've got instruction one for player one, instruction two for player two. 70 00:04:33,510 --> 00:04:36,190 And the judge that will make the call. 71 00:04:36,470 --> 00:04:36,990 All right. 72 00:04:36,990 --> 00:04:38,670 And so now we have our agents. 73 00:04:38,670 --> 00:04:41,310 And this is again by analogy with last time. 74 00:04:41,310 --> 00:04:42,710 It's really very similar. 75 00:04:42,910 --> 00:04:48,190 Player one I've kept calling it player one because I you know, this really is a copy paste job basically. 76 00:04:48,390 --> 00:04:54,470 Uh, except I'm using GPT four or mini for both because I thought it wasn't fair to use, uh, a llama, 77 00:04:54,470 --> 00:04:56,990 we didn't get the same the same quality. 78 00:04:57,310 --> 00:05:06,230 Uh, so we've got GPT four or mini, um, and, uh, so we are this is the the the the player one routed 79 00:05:06,230 --> 00:05:16,640 agent, and we are passing in, uh, here we're using the Autogen gen super tool so that that is what 80 00:05:16,640 --> 00:05:17,320 we're doing. 81 00:05:17,320 --> 00:05:21,400 I realize even looking at this, that we don't actually need a player one and player two class. 82 00:05:21,400 --> 00:05:24,080 This could be done in a simpler way with just one. 83 00:05:24,360 --> 00:05:27,280 Uh, so uh, that's an obvious point. 84 00:05:27,280 --> 00:05:33,800 But anyways, for whatever reason, we we've got two, two agents here, two different types of agent, 85 00:05:34,000 --> 00:05:38,760 uh, that uh, but we're going to prompt it for whether it should find the pros or cons, but you could 86 00:05:38,760 --> 00:05:42,560 imagine you could switch in a different model here if you wanted, if you wanted to have deep seek, 87 00:05:42,600 --> 00:05:44,200 do one of the research. 88 00:05:44,200 --> 00:05:47,200 So I'll keep it as two separate agents in case you choose to do that. 89 00:05:47,640 --> 00:05:48,480 Um, okay. 90 00:05:48,600 --> 00:05:50,400 So other than that, this is exactly the same. 91 00:05:50,440 --> 00:05:56,320 Other than supplying the search tool, reflecting on tool use, everything here is the same. 92 00:05:56,600 --> 00:06:01,520 Uh, it delegates to its underlying LM and the judge is the same. 93 00:06:01,560 --> 00:06:08,920 It also has a delegate, an underlying LM, and in its message handler, the the method that's decorated 94 00:06:08,920 --> 00:06:14,600 message handler, it collects the two messages that we set up above. 95 00:06:15,000 --> 00:06:19,690 It puts them, it finds the two agents player one and player two. 96 00:06:19,970 --> 00:06:26,090 It uses this lookup and then it calls send message and this code is identical. 97 00:06:26,170 --> 00:06:31,410 This is all exactly the same as the code we just used yesterday with the runtime. 98 00:06:31,410 --> 00:06:32,690 That was that was local. 99 00:06:32,890 --> 00:06:37,450 And so the reason I do this, I want to show you that without changing anything I didn't. 100 00:06:37,490 --> 00:06:39,290 There's nothing here about it being distributed. 101 00:06:39,290 --> 00:06:41,170 It doesn't know that it's distributed. 102 00:06:41,330 --> 00:06:43,650 This this could still be doing rock, paper, scissors. 103 00:06:43,650 --> 00:06:44,890 It would be the same thing. 104 00:06:44,890 --> 00:06:47,330 We're just calling self dot send message. 105 00:06:47,330 --> 00:06:52,410 And what we don't realize is this is running remotely and it's running on a, on a runtime that's running 106 00:06:52,410 --> 00:06:53,610 on a port here. 107 00:06:53,930 --> 00:07:01,490 And uh, it's going to be the, the Autogen core is going to be handling calling the right function 108 00:07:01,490 --> 00:07:03,450 in the right, uh, agent. 109 00:07:03,450 --> 00:07:10,170 So these calls that appear to just be simply, uh, I'm calling send message, uh, right here, I call 110 00:07:10,210 --> 00:07:10,810 send message. 111 00:07:10,810 --> 00:07:15,130 And that is going to result in this handle my message getting called. 112 00:07:15,370 --> 00:07:20,900 And before that was just directly like some if statements in Python that we're just making that call. 113 00:07:21,100 --> 00:07:28,660 Now this is going to be happening using gRPC remotely orchestrated by this distributed runtime. 114 00:07:28,860 --> 00:07:32,340 But that is completely unknown to us as far as we're concerned. 115 00:07:32,340 --> 00:07:34,140 We're just doing exactly the same thing. 116 00:07:34,380 --> 00:07:40,060 And that is the power of Autogen core distributed that we don't have to worry about the fact that these 117 00:07:40,060 --> 00:07:44,660 are different processes running, and they could be written in different computer programming languages. 118 00:07:44,660 --> 00:07:50,780 And all of the stitching together of messages is happening for us, just based on looking up player 119 00:07:50,820 --> 00:07:52,260 one and player two. 120 00:07:52,580 --> 00:07:54,220 Um, everything is happening. 121 00:07:54,220 --> 00:07:55,940 So enough prattle. 122 00:07:56,100 --> 00:07:57,060 Let's run that. 123 00:07:57,380 --> 00:07:57,700 Okay. 124 00:07:57,740 --> 00:08:01,780 And this is where this is where the meat happens. 125 00:08:01,860 --> 00:08:04,420 So I've got two different implementations that I want to show you. 126 00:08:04,420 --> 00:08:07,500 And we're starting with all in one worker. 127 00:08:07,620 --> 00:08:08,900 And here's how it works. 128 00:08:09,020 --> 00:08:17,340 We we, uh, first of all, say that we want to create a new worker agent runtime, and we point it 129 00:08:17,340 --> 00:08:18,580 at our host. 130 00:08:18,620 --> 00:08:19,980 This is the host. 131 00:08:19,980 --> 00:08:26,180 So this will be a new runtime connecting to that host and we will start that worker. 132 00:08:26,420 --> 00:08:31,620 We are then going to register, uh, three agents with that worker. 133 00:08:31,940 --> 00:08:34,060 Agent one a player one, player two. 134 00:08:34,100 --> 00:08:35,020 And the judge. 135 00:08:35,180 --> 00:08:35,620 Here they are. 136 00:08:35,620 --> 00:08:36,100 Player one. 137 00:08:36,100 --> 00:08:43,340 Player two, and the judge all being registered with this worker with this gRPC worker agent runtime 138 00:08:43,500 --> 00:08:44,540 at that host. 139 00:08:45,060 --> 00:08:45,900 And there we go. 140 00:08:45,940 --> 00:08:48,700 There is a player one, player two. 141 00:08:48,820 --> 00:08:50,380 And the, the the judge. 142 00:08:50,380 --> 00:08:53,020 These are the factories that will create them. 143 00:08:53,300 --> 00:09:00,340 And we are now um, now we, uh, collect the agent ID of the judge. 144 00:09:00,700 --> 00:09:01,580 So I'm going to run this. 145 00:09:01,580 --> 00:09:06,420 And because this is set to true, only this code here is going to run and it's done. 146 00:09:06,900 --> 00:09:09,100 And now this is the same as before. 147 00:09:09,100 --> 00:09:15,780 It's just the same thing we're going to send go the go message to the agent ID that is that I've just 148 00:09:15,820 --> 00:09:19,420 that I set here to the judge, uh, and let's see what happens. 149 00:09:21,220 --> 00:09:23,900 So it's thinking stuff is happening. 150 00:09:23,900 --> 00:09:26,750 I'm still thinking. 151 00:09:27,710 --> 00:09:28,990 Messages are flying. 152 00:09:30,150 --> 00:09:35,310 It's what we hope is happening is that OpenAI, GPT four, and Minnie is coming up with pros and cons 153 00:09:35,310 --> 00:09:37,190 through two separate agent calls. 154 00:09:37,190 --> 00:09:40,590 And then a judge has put it all together, and this is what we get back. 155 00:09:40,630 --> 00:09:41,750 Pros of Autogen. 156 00:09:41,790 --> 00:09:45,310 Here are some advantages of using Autogen in your AI agent projects. 157 00:09:45,430 --> 00:09:47,510 The cons are right here. 158 00:09:47,870 --> 00:09:54,830 Uh, limited AI capabilities, less customizable, less structured potential bugs and performance issues 159 00:09:54,830 --> 00:09:56,230 with lower end models. 160 00:09:56,270 --> 00:10:02,630 Okay, but based on this, basically there's also some, uh, some strengths that appears, uh, memory, 161 00:10:02,630 --> 00:10:07,710 coherent context, ease of development, scalability, which is what we're experiencing and versatile 162 00:10:07,710 --> 00:10:08,670 applications. 163 00:10:08,870 --> 00:10:12,870 And the decision is recommend using Autogen. 164 00:10:12,870 --> 00:10:17,270 So the uh based purely on this research, that is the decision. 165 00:10:17,510 --> 00:10:23,390 And with that we will then stop our workers and we stop our host. 166 00:10:23,390 --> 00:10:27,830 And we have just experienced distributed autogen core.