1 00:00:00,440 --> 00:00:00,800 Well. 2 00:00:00,800 --> 00:00:08,680 I am so excited to welcome you to week two of our time together in this journey into Agents Land. 3 00:00:08,840 --> 00:00:18,360 And as well, you know, week two is all about open AI agents SDK, formerly known as swarm, released 4 00:00:18,360 --> 00:00:25,040 very recently, the newest of our arrivals into the agents Framework land and one that I cannot wait 5 00:00:25,040 --> 00:00:26,360 to tell you all about. 6 00:00:27,000 --> 00:00:34,200 But first, isn't there always a but first but first, before we can do open AI agents SDK, I have 7 00:00:34,200 --> 00:00:40,840 a sidebar, a topic that I have to cover with you, a very important topic, and it's to talk to you 8 00:00:40,840 --> 00:00:44,840 about asynchronous Python async IO. 9 00:00:45,160 --> 00:00:50,320 It's something which is common across all of the agent frameworks. 10 00:00:50,320 --> 00:00:57,080 So all of them make use of asynchronous Python, and it's something which you can get by without really 11 00:00:57,080 --> 00:00:58,120 understanding. 12 00:00:58,120 --> 00:01:02,540 There's a couple of rules that you can you can learn and then you just follow those rules always. 13 00:01:02,540 --> 00:01:04,660 And it will just kind of work and that's okay. 14 00:01:05,060 --> 00:01:12,860 But I'm here to tell you that that's unsatisfactory and that it's much better to just take half an hour 15 00:01:12,860 --> 00:01:19,140 to get get to the bottom of this, to really understand it, to thrash it out until you're like, okay, 16 00:01:19,180 --> 00:01:24,420 I get asynchronous Python, I understand it, and if you do this, if you take the half an hour, you 17 00:01:24,420 --> 00:01:25,500 will thank me. 18 00:01:25,660 --> 00:01:30,340 It will again and again you will come across this and you'll you will be completely comfortable with 19 00:01:30,340 --> 00:01:30,540 it. 20 00:01:30,540 --> 00:01:31,780 And it's only half an hour. 21 00:01:31,780 --> 00:01:36,420 And so I've put a guide and the guides that will take you through this, so that you can be left with 22 00:01:36,420 --> 00:01:41,260 no questions in your mind at all about what it means to write asynchronous Python. 23 00:01:41,540 --> 00:01:45,900 And I'm also now going to cover it just very briefly at a high level. 24 00:01:45,900 --> 00:01:48,100 Let's talk async IO. 25 00:01:48,940 --> 00:01:52,900 Well, look, first of all there is like a short version, a simple version. 26 00:01:52,900 --> 00:01:56,260 This is what I mean when I say you can get by without really understanding it. 27 00:01:56,260 --> 00:02:05,840 So async IO is a way of writing Python code, which is a kind of lightweight version of multithreading. 28 00:02:05,840 --> 00:02:11,520 So many people from a software engineering background will be familiar with the idea of multithreading. 29 00:02:11,520 --> 00:02:17,480 When you write code that can run concurrently, you can have multiple threads which are each executing 30 00:02:17,480 --> 00:02:23,480 code together, and there's often a lot of baggage that comes with that, some sort of framework stuff 31 00:02:23,480 --> 00:02:24,560 that comes with it. 32 00:02:24,840 --> 00:02:32,240 Well, async IO, which was introduced I think, in Python 3.5, is this very lightweight way of doing 33 00:02:32,240 --> 00:02:36,760 it, which doesn't actually involve threads at an operating system level. 34 00:02:36,760 --> 00:02:42,720 And it also doesn't involve what's called multiprocessing, which is when you spawn multiple Python 35 00:02:42,760 --> 00:02:44,720 processes and they all run together. 36 00:02:44,760 --> 00:02:48,280 This is another way of doing it that is super lightweight. 37 00:02:48,280 --> 00:02:53,680 And because it's super lightweight, it means that you can have thousands or tens of thousands of these 38 00:02:53,680 --> 00:02:57,340 things all running without consuming much resources at all. 39 00:02:57,340 --> 00:03:03,780 So it's a simple, easy peasy, lightweight way of writing code that can run concurrently. 40 00:03:03,780 --> 00:03:10,660 And particularly it's good when you have code that makes use of of input output, like waiting on networks. 41 00:03:10,820 --> 00:03:13,060 It allows other things to be running. 42 00:03:13,060 --> 00:03:19,340 Whilst one bit of code is waiting on network, and when you're running LM requests, you're mostly. 43 00:03:19,340 --> 00:03:25,140 If you're using paid APIs like OpenAI, most of the time is being spent waiting for stuff to feed back 44 00:03:25,140 --> 00:03:27,300 from a model running on the cloud. 45 00:03:27,300 --> 00:03:31,940 So there's a lot of waiting on networks, a lot of IO bound waiting. 46 00:03:32,220 --> 00:03:36,340 And as a result, asynchronous code is great to use. 47 00:03:36,340 --> 00:03:41,700 And when you're thinking of multi-agent frameworks, when potentially you have lots of these all hitting 48 00:03:41,700 --> 00:03:45,300 different APIs, it makes so much sense to be using it. 49 00:03:45,300 --> 00:03:50,540 And that is why all of the frameworks we'll look at use async IO. 50 00:03:51,020 --> 00:03:51,540 Okay. 51 00:03:51,780 --> 00:03:54,900 So with that preamble this is the short version. 52 00:03:55,040 --> 00:03:56,920 The short version is async. 53 00:03:56,920 --> 00:03:58,440 IO is actually two things. 54 00:03:58,440 --> 00:04:06,200 It's a a package called async IO that you can import, and it's also some language constructs baked 55 00:04:06,200 --> 00:04:07,360 into Python. 56 00:04:07,680 --> 00:04:12,960 And those language constructs include two keywords that you see right here. 57 00:04:13,240 --> 00:04:16,440 And one of those keywords is the word async. 58 00:04:16,840 --> 00:04:24,280 And anytime that you have a function which is potentially going to be able to run in a way that allows 59 00:04:24,320 --> 00:04:32,520 other things to happen concurrently, you simply put the word async before it like this async def do 60 00:04:32,520 --> 00:04:37,600 some processing, so def do some processing becomes async def do some processing. 61 00:04:37,640 --> 00:04:42,200 That's how you say this is a function which can run asynchronously. 62 00:04:42,920 --> 00:04:47,880 And when you call that function you don't just call do some processing like you're used to. 63 00:04:48,080 --> 00:04:50,480 You have to put the word await before it. 64 00:04:50,480 --> 00:04:56,820 So you say await, do some processing and if do some processing returns a string, then you can't just 65 00:04:56,820 --> 00:04:59,380 say result equals do some processing. 66 00:04:59,620 --> 00:05:02,140 You have to say result equals await. 67 00:05:02,340 --> 00:05:03,460 Do some processing. 68 00:05:03,900 --> 00:05:06,180 But otherwise it's just the same. 69 00:05:06,340 --> 00:05:07,700 So that's the short version. 70 00:05:07,700 --> 00:05:12,140 And if you wanted to get by without really understanding it, you could just follow those rules and 71 00:05:12,140 --> 00:05:14,220 keep going and it might be enough. 72 00:05:14,380 --> 00:05:16,860 But now let me tell you the bigger story. 73 00:05:17,860 --> 00:05:18,380 Okay. 74 00:05:18,420 --> 00:05:23,500 Now we're going to go a little bit deeper and talk about the real story with async IO. 75 00:05:23,740 --> 00:05:29,740 So look as I say, it's a lightweight alternative to multithreading or multiprocessing. 76 00:05:29,940 --> 00:05:33,140 Something like multithreading is implemented at the OS level. 77 00:05:33,140 --> 00:05:39,340 And it supports concurrent execution of different programs where the CPU, the CPU level that's being 78 00:05:39,340 --> 00:05:44,500 managed to sort of switch between these two programs and treat them as if they're running at the same 79 00:05:44,500 --> 00:05:45,020 time. 80 00:05:45,500 --> 00:05:48,660 This is different in the case of async IO. 81 00:05:48,700 --> 00:05:50,780 So there are some some special keywords. 82 00:05:50,780 --> 00:05:52,360 There's this keyword async. 83 00:05:52,680 --> 00:05:59,560 If you define a function as async def and then the function instead of just def the function, then 84 00:05:59,560 --> 00:06:02,080 this thing is no longer actually called a function. 85 00:06:02,080 --> 00:06:04,280 It's known as a coroutine. 86 00:06:04,560 --> 00:06:06,360 Now most people still use the word function. 87 00:06:06,720 --> 00:06:11,360 You'll hear me saying function, but strictly speaking, anytime you see async def, you should say, 88 00:06:11,400 --> 00:06:13,560 okay, that's not a function, it is a coroutine. 89 00:06:14,080 --> 00:06:18,320 And that means it's something special that that Python can pause and resume. 90 00:06:19,360 --> 00:06:25,320 When you call a coroutine, it doesn't run unlike any other function that you call that does run. 91 00:06:25,320 --> 00:06:34,360 When you call a coroutine, it just returns a coroutine object, but nothing is executed to actually 92 00:06:34,360 --> 00:06:36,160 run that coroutine object. 93 00:06:36,160 --> 00:06:40,000 There are a few ways to do it, but the most common one, and the one that we will use all the time 94 00:06:40,360 --> 00:06:42,160 is to await it. 95 00:06:42,160 --> 00:06:48,320 So you say await, and then the coroutine object and that schedules it for execution. 96 00:06:48,800 --> 00:06:51,310 What does it mean to say it's scheduled for execution. 97 00:06:51,310 --> 00:06:57,390 Well, there is this piece of code written in the async IO library, which does what's known as an event 98 00:06:57,390 --> 00:07:04,310 loop, which means it's a kind of while loop that iterates and it takes this coroutine and it starts 99 00:07:04,310 --> 00:07:08,230 executing it, and it can only execute one coroutine at a time. 100 00:07:08,230 --> 00:07:09,710 It's not like multithreading. 101 00:07:09,710 --> 00:07:18,270 It's going to be executing this coroutine, except if this coroutine gets to a point when it's stopped 102 00:07:18,270 --> 00:07:21,670 and it's waiting for something like it's waiting for IO. 103 00:07:21,830 --> 00:07:22,590 Perhaps. 104 00:07:22,630 --> 00:07:26,710 Perhaps it's made a call to OpenAI and it's waiting for OpenAI to respond. 105 00:07:27,230 --> 00:07:35,630 At that point, the event loop can put that on pause and start executing a different one of the coroutines 106 00:07:35,630 --> 00:07:38,550 that's in its sort of list of coroutines waiting to be run. 107 00:07:38,990 --> 00:07:44,630 And if that hits a point when it's waiting on on IO, then the event loop can run another one or it 108 00:07:44,630 --> 00:07:46,110 can continue the first one. 109 00:07:46,670 --> 00:07:50,050 So it's a sort of manual way of handling multithreading. 110 00:07:50,250 --> 00:07:55,450 It's a manual approach to implementing multithreading, but only really working when something is blocked. 111 00:07:55,450 --> 00:07:56,450 Waiting on I o. 112 00:07:56,970 --> 00:08:02,090 And because of this, because it's sort of written at a code level and it's super simple and very easy 113 00:08:02,090 --> 00:08:05,090 to understand, it's also lightweight. 114 00:08:05,090 --> 00:08:08,650 You can have thousands or tens of thousands of these things. 115 00:08:08,850 --> 00:08:13,050 And, and so it's and it's really easy and quick to get this running. 116 00:08:13,210 --> 00:08:15,930 And that is why it's so popular. 117 00:08:15,930 --> 00:08:21,130 And that is why it's used so ubiquitously across a genetic frameworks. 118 00:08:21,610 --> 00:08:26,490 So I hope obviously I've only given you a bit of detail here, but I hope that gives you some perspective 119 00:08:26,490 --> 00:08:27,610 on what's going on. 120 00:08:27,810 --> 00:08:34,090 And when we look back now at at this, you can see that that whilst a simple interpretation is just 121 00:08:34,090 --> 00:08:38,650 to say, okay, whenever I use async I've got to use await. 122 00:08:39,130 --> 00:08:45,610 The deeper interpretation is to understand that when I say async def do some processing that's no longer 123 00:08:45,610 --> 00:08:49,110 a function, it's now a coroutine do some processing. 124 00:08:49,110 --> 00:08:53,630 We need to call await, do some processing in order to schedule that coroutine. 125 00:08:53,630 --> 00:09:00,150 And that is now blocking until that completes and the result will go into that variable result. 126 00:09:01,070 --> 00:09:05,270 I hope that made some sense, but if not, as I say, there's a whole guide. 127 00:09:05,270 --> 00:09:07,190 You should go and look at the guide now. 128 00:09:08,030 --> 00:09:12,470 And so just to beat this one to death, I'll give you a couple more examples. 129 00:09:12,470 --> 00:09:14,670 And then hopefully you've really got this. 130 00:09:15,030 --> 00:09:16,110 So look at this again. 131 00:09:16,110 --> 00:09:20,590 This is back to the async def do some processing. 132 00:09:20,790 --> 00:09:21,950 It's going to do some work. 133 00:09:21,950 --> 00:09:24,190 And then it's going to return the string done. 134 00:09:24,750 --> 00:09:31,390 If I just say like my coroutine equals do some processing, you might, you know, if you didn't know 135 00:09:31,390 --> 00:09:34,190 already about this, you might think that that will run do some processing. 136 00:09:34,190 --> 00:09:35,590 But of course it will not. 137 00:09:35,630 --> 00:09:41,870 It will return a coroutine object and that's what will go into the variable, my coroutine. 138 00:09:42,390 --> 00:09:45,590 In order to actually run it you have to call await. 139 00:09:45,590 --> 00:09:49,010 So you'd say my result equals await my coroutine. 140 00:09:49,010 --> 00:09:50,330 It's now going to run. 141 00:09:50,530 --> 00:09:51,490 And the result? 142 00:09:51,490 --> 00:09:55,170 The string done is going to go into my result. 143 00:09:55,690 --> 00:09:57,170 So that is the point. 144 00:09:57,170 --> 00:09:57,970 That's how it works. 145 00:09:57,970 --> 00:09:59,450 And of course you don't need to do that. 146 00:09:59,690 --> 00:10:00,330 It's simpler than that. 147 00:10:00,330 --> 00:10:02,970 You can just say my result equals await. 148 00:10:03,010 --> 00:10:05,210 Do some processing, which is what we always do. 149 00:10:06,130 --> 00:10:10,850 And then just to hopefully sort of connect the dots to make this really click for you. 150 00:10:10,850 --> 00:10:14,490 There are other constructs you don't just have to call await. 151 00:10:14,610 --> 00:10:19,130 And then the name of of a coroutine, because if that's all you could do, you might think, you might 152 00:10:19,130 --> 00:10:20,650 say to me, okay, hang on. 153 00:10:20,650 --> 00:10:24,610 But in that case there's no real there's nothing concurrent happening here. 154 00:10:24,650 --> 00:10:28,690 Every time I call await and then a coroutine, it will block until that finishes and then it will go 155 00:10:28,690 --> 00:10:29,610 on to the next. 156 00:10:29,810 --> 00:10:36,170 Well, there are a few other constructs in the async IO package that allow us to be more to do more 157 00:10:36,170 --> 00:10:40,290 interesting things, and this is one of them, and it's one that we will use this week in our code. 158 00:10:40,490 --> 00:10:47,110 And it's where you can say results, plural Results equals await. 159 00:10:47,110 --> 00:10:50,750 And then call this this function Asyncio dot gather. 160 00:10:51,150 --> 00:10:54,430 And you can then pass in multiple coroutines. 161 00:10:54,790 --> 00:10:56,630 And I'm sure you can imagine what happens. 162 00:10:56,630 --> 00:11:00,350 But the event loop is going to schedule all three. 163 00:11:00,510 --> 00:11:05,230 And as soon as one of them is blocking, the others will start running, or one of the others will run 164 00:11:05,230 --> 00:11:07,430 until it's blocking, and then the third one will run. 165 00:11:07,430 --> 00:11:13,390 And so all three of those coroutines will, will, will run, will execute. 166 00:11:13,390 --> 00:11:20,550 And the results as a list will go into the variable results, a list of each of the three results from 167 00:11:20,550 --> 00:11:23,030 each of those three coroutines. 168 00:11:23,710 --> 00:11:26,590 So in some ways it's kind of fake multithreading. 169 00:11:26,870 --> 00:11:29,550 It's sort of brute force multithreading. 170 00:11:29,550 --> 00:11:35,470 It's multithreading not at the operating system level, but multithreading implemented almost manually 171 00:11:35,470 --> 00:11:41,390 with this event loop, and just handling things like IO blocking and being able to schedule the next 172 00:11:41,390 --> 00:11:41,750 one. 173 00:11:42,190 --> 00:11:43,470 That's how to think about it.