1 00:00:00,200 --> 00:00:07,600 The other big topic is Google Colab, and that is the other thing that we're going to be using today 2 00:00:07,640 --> 00:00:10,200 as we we it's all about new stuff today. 3 00:00:10,200 --> 00:00:13,880 So Google Colab, you may have already used it. 4 00:00:13,880 --> 00:00:16,800 A lot of people have used it already and they're like, why don't we start with Google Colab? 5 00:00:16,800 --> 00:00:19,720 There's a lot of a lot to love about Google Colab. 6 00:00:19,720 --> 00:00:21,120 First of all, what is it? 7 00:00:21,120 --> 00:00:25,560 So it is something which allows you to to run a notebook. 8 00:00:25,560 --> 00:00:29,960 Notebook is this kind of interface we've been playing with in cursor when you have like, like cells 9 00:00:29,960 --> 00:00:35,240 of code that you run a cell at a time, there's very experimental way of interacting with code. 10 00:00:35,240 --> 00:00:42,160 So Google has built something that lets you run a notebook like that, except you're running it in a 11 00:00:42,160 --> 00:00:48,400 browser, you're running it in a browser window, and it's actually running on one of Google's computers. 12 00:00:48,560 --> 00:00:54,920 So you can like bring up a like a Chrome page, point it at a Google URL, and then you effectively 13 00:00:54,920 --> 00:01:01,080 have like like a boxes there, and you're remoting in to a Google box and you're able to run the code. 14 00:01:01,080 --> 00:01:05,080 And when you run the cell, it's as if it's running in your browser, but of course it's not. 15 00:01:05,120 --> 00:01:10,120 The code is executing on one of Google's machines out there in the cloud, and you're seeing the results 16 00:01:10,120 --> 00:01:10,880 immediately. 17 00:01:10,960 --> 00:01:16,760 So it lets you remote in to a to a remote box running on Google's cloud. 18 00:01:16,960 --> 00:01:19,640 And there are a few things about this that are really great. 19 00:01:20,000 --> 00:01:26,600 One of them is that it allows you to collaborate in this super seamless way with other people on the 20 00:01:26,600 --> 00:01:27,840 same code. 21 00:01:27,880 --> 00:01:30,160 Like in the same notebook construct. 22 00:01:30,800 --> 00:01:36,720 Google is famous, of course, for Google Docs for really bringing the kind of collaborative office 23 00:01:36,720 --> 00:01:39,400 idea, uh, to, to the mainstream. 24 00:01:39,520 --> 00:01:44,600 And so everyone's very familiar with this idea that you can share a Google doc and share a sheet and 25 00:01:44,600 --> 00:01:49,160 be able to to work together with someone else in it and really collaborate. 26 00:01:49,160 --> 00:01:56,280 And Google Colab is bringing that same mentality to a notebook so that you can be coding with somebody 27 00:01:56,280 --> 00:01:56,720 else. 28 00:01:56,760 --> 00:02:00,920 It's not quite you don't quite get to see the changes in quite the same way, but it's almost as good 29 00:02:00,920 --> 00:02:01,520 as that. 30 00:02:01,520 --> 00:02:04,040 And you can both be adding code and running things. 31 00:02:04,080 --> 00:02:10,880 And so it's such a great way to work together with people and, you know, it's like a it's a different 32 00:02:10,880 --> 00:02:16,160 mindset to something like a using a repo where you where you check stuff in and merge it and so on. 33 00:02:16,160 --> 00:02:23,240 It's much more like live working, but it suits the notebook mindset, that experimental mindset, it 34 00:02:23,240 --> 00:02:24,480 works so well with that. 35 00:02:24,480 --> 00:02:27,760 So very collaborative as as the name suggests. 36 00:02:28,000 --> 00:02:31,320 Uh, and it's also integrated with, with the other Google services. 37 00:02:31,320 --> 00:02:36,480 So, so later this this week we're going to to pull Google Drive in so you can have your Google Drive 38 00:02:36,520 --> 00:02:39,960 docs available there too, which is very convenient. 39 00:02:39,960 --> 00:02:43,000 So for many reasons it is convenient. 40 00:02:43,360 --> 00:02:46,600 But above all else the real the. 41 00:02:46,640 --> 00:02:47,960 So what is that? 42 00:02:47,960 --> 00:02:52,080 It gives you access to high end GPUs. 43 00:02:52,360 --> 00:02:54,000 That's that's the big deal. 44 00:02:54,000 --> 00:02:59,120 And I probably need to mention, just in case you don't, uh, you haven't got the connection between 45 00:02:59,120 --> 00:03:00,720 what we're doing and GPUs. 46 00:03:00,760 --> 00:03:08,800 I mean, obviously everyone knows about Nvidia stock and that GPUs are at the heart of, of modern AI. 47 00:03:08,840 --> 00:03:15,200 But but if you're wondering why, when we talk about these models with these huge number of parameters, 48 00:03:15,240 --> 00:03:21,120 these tons and tons, these 8 billion numbers, when you when you're predicting the most likely next 49 00:03:21,120 --> 00:03:27,440 token, the way you do that is by taking all of these numbers and multiplying them repeatedly by the 50 00:03:27,480 --> 00:03:34,080 sort of input sets of tokens in some very clever way, which is known as the transformer architecture, 51 00:03:34,080 --> 00:03:38,640 which is the way that these things will hook together and you're doing lots of multiplications and adding 52 00:03:38,880 --> 00:03:43,520 multiplications and adding is basically matrix math, linear algebra. 53 00:03:43,680 --> 00:03:49,280 It's lots and lots of matrix calculations, and they can all happen very efficiently in parallel with 54 00:03:49,280 --> 00:03:50,000 one another. 55 00:03:50,040 --> 00:03:51,280 They're all sort of independent. 56 00:03:51,280 --> 00:03:54,360 So they can all run together in this very, very efficient way. 57 00:03:54,360 --> 00:03:55,960 But they all have to run. 58 00:03:56,080 --> 00:04:01,240 And it turns out when, when all this is becoming a big thing, in the early 2000, people were saying 59 00:04:01,240 --> 00:04:07,560 what we really need is like specialized custom hardware that is that is efficient, that is designed 60 00:04:07,560 --> 00:04:12,440 for matrix calculations in in fast, real time. 61 00:04:12,440 --> 00:04:16,080 And you can imagine all of these scientists sitting around there thinking about this. 62 00:04:16,120 --> 00:04:21,720 And while they were doing it, the other people in the room were busy playing Doom or Quake or whatever 63 00:04:21,760 --> 00:04:28,680 was big in those days and, uh, using using PCs with which were like hammering away at drawing this 64 00:04:28,680 --> 00:04:29,720 3D graphics. 65 00:04:29,880 --> 00:04:37,560 And the way that you do these kinds of games is by doing a lot of quick polygon calculations, which 66 00:04:37,600 --> 00:04:40,920 involves lots of efficient matrix maths. 67 00:04:41,080 --> 00:04:45,120 So the graphics cards, everyone was trying to buy a PC with a bigger graphics card. 68 00:04:45,160 --> 00:04:51,920 Those graphics cards were basically custom hardware designed to run matrix calculations, multiplies 69 00:04:51,920 --> 00:04:59,040 and adds in parallel like huge numbers of them at the same time so that it could draw, uh, screens 70 00:04:59,040 --> 00:05:00,680 like 60 times a second or whatever. 71 00:05:00,880 --> 00:05:08,280 Uh, and so it turned out that this kind of hardware was absolutely perfect for the kinds of AI that 72 00:05:08,280 --> 00:05:11,000 we were doing at the time and that we are still doing to this day. 73 00:05:11,200 --> 00:05:13,570 Uh, and luckily for Nvidia. 74 00:05:13,810 --> 00:05:16,930 And so that is why GPUs you probably do all that already. 75 00:05:16,970 --> 00:05:17,730 So I'm sorry. 76 00:05:17,970 --> 00:05:22,610 That's why GPUs are so essential to to modern day AI. 77 00:05:23,010 --> 00:05:24,570 And they're quite expensive. 78 00:05:24,570 --> 00:05:29,930 And what you really need more than anything is you need them to have a lot of memory, which typically 79 00:05:29,970 --> 00:05:32,650 you need, like it's different to the memory of your computer. 80 00:05:32,690 --> 00:05:39,850 The GPU also has Ram, as games players all know, and you need to have enough to fit all of the model 81 00:05:39,890 --> 00:05:40,090 on. 82 00:05:40,090 --> 00:05:42,450 And if you're training, you need a lot more than that too. 83 00:05:42,690 --> 00:05:47,170 And so you need a lot of memory on your GPU for it to be able to run efficiently. 84 00:05:47,170 --> 00:05:48,250 And that's a problem. 85 00:05:48,250 --> 00:05:52,250 And as, uh, that's not a problem, that's expensive and is the constraint. 86 00:05:52,410 --> 00:05:53,970 And Apple people will know. 87 00:05:53,970 --> 00:05:59,490 Well, like I might happen to be a mac person, uh, that, uh, one of the great things about Apple 88 00:05:59,490 --> 00:06:05,770 Silicon is that you have what's called a unified memory, which means that the memory that your CPU 89 00:06:05,810 --> 00:06:08,530 uses is the same as the memory the GPU uses. 90 00:06:08,530 --> 00:06:11,170 So you can you can share it together, which is really amazing. 91 00:06:11,170 --> 00:06:14,650 But for PCs, you typically have to get a graphics card with big enough memory. 92 00:06:14,650 --> 00:06:16,090 And they're expensive. 93 00:06:16,130 --> 00:06:22,570 They are many thousands of dollars and boxes which have big Nvidia chips on on them. 94 00:06:22,570 --> 00:06:24,530 Graphics cards are very expensive. 95 00:06:24,570 --> 00:06:28,370 You can buy good ones for like $6,000 or something. 96 00:06:28,410 --> 00:06:31,850 They're expensive, but there's an alternative. 97 00:06:31,890 --> 00:06:37,850 There is an alternative and it's called Google Colab, where you rent GPUs in the cloud instead. 98 00:06:37,850 --> 00:06:43,010 And so here are the details you get when you connect to Colab. 99 00:06:43,010 --> 00:06:50,370 You choose a runtime and runtime is it's name of like the, the, uh, like the instance that you're 100 00:06:50,370 --> 00:06:51,690 connecting to on the cloud. 101 00:06:51,690 --> 00:06:57,610 And it includes like a kernel, like we're used to a Python process that's, that's running there. 102 00:06:57,770 --> 00:07:03,130 And you can get some that are CPU based, meaning that there's no graphics card at all, there's no 103 00:07:03,130 --> 00:07:08,050 GPU, so you can just use it for everyday tasks, a bit like you can on your own computer. 104 00:07:08,290 --> 00:07:11,810 You can have ones with a lower spec GPU. 105 00:07:11,930 --> 00:07:13,170 It's still pretty mighty. 106 00:07:13,210 --> 00:07:14,970 It's still a powerful GPU. 107 00:07:15,250 --> 00:07:19,210 The typical one is the Nvidia Tesla T4. 108 00:07:19,370 --> 00:07:21,730 The T4 is the typical one. 109 00:07:21,730 --> 00:07:26,250 It has 15GB of GPU Ram, which is a lot. 110 00:07:26,530 --> 00:07:28,250 And here's the thing. 111 00:07:28,490 --> 00:07:29,690 It's free. 112 00:07:29,930 --> 00:07:31,810 It's it's completely free. 113 00:07:31,930 --> 00:07:38,170 You can have a Google Colab with a T4 GPU, and you can have it for free at any point. 114 00:07:38,170 --> 00:07:40,090 And we're just gonna do it in just a second. 115 00:07:40,090 --> 00:07:40,730 Can you believe it? 116 00:07:40,730 --> 00:07:45,650 And that's what you'll be using for this week, for week three and for week seven on this course when 117 00:07:45,650 --> 00:07:47,690 we are running on GPUs in the cloud. 118 00:07:47,690 --> 00:07:48,810 So that's amazing. 119 00:07:49,010 --> 00:08:00,010 And then if you want to splash out you can pay to have a higher spec one, including an A100 for 40GB. 120 00:08:00,010 --> 00:08:04,290 But when I say splash out, we're talking about like a few bucks an hour. 121 00:08:04,530 --> 00:08:08,450 We're talking about like, you know, a Big Mac an hour kind of level. 122 00:08:08,570 --> 00:08:12,130 I believe we'll do the maths later to see exactly how much it costs. 123 00:08:12,130 --> 00:08:15,650 But, uh, it's it's, uh, I was going to say it's cheap. 124 00:08:15,650 --> 00:08:16,810 And you might disagree with me there. 125 00:08:16,810 --> 00:08:22,170 You might say a few dollars an hour is not cheap at all, but compare it with the cost of buying an 126 00:08:22,170 --> 00:08:26,170 A100 on a PC that you're going to have for yourself. 127 00:08:26,170 --> 00:08:28,370 That that is an expensive proposition. 128 00:08:28,370 --> 00:08:33,090 And when you think of it in that light, to be able to run it for $2 an hour only when you need it, 129 00:08:33,090 --> 00:08:35,290 and then you don't use it anymore, you don't keep it running. 130 00:08:35,290 --> 00:08:38,410 You just you only pay for the minutes that you have it alive. 131 00:08:38,410 --> 00:08:39,170 That's it. 132 00:08:39,490 --> 00:08:42,410 It seems like it's actually a really great deal. 133 00:08:42,410 --> 00:08:48,850 And I feel like we are lucky to have this ability to rent super high powered horsepower whenever we 134 00:08:48,850 --> 00:08:49,530 want it. 135 00:08:49,570 --> 00:08:52,050 And so, yeah, I think this is fabulous. 136 00:08:52,050 --> 00:08:56,050 And that's why I always when people say to me, I want to buy a box with blah, blah, blah, blah, 137 00:08:56,250 --> 00:09:02,850 I say, really, really, because you know that that you can have as many a100s as you want for a fraction 138 00:09:02,850 --> 00:09:05,850 of that, uh, just, you know, every weekend. 139 00:09:06,090 --> 00:09:12,650 So anyways, the, the other the final sales pitch for it and then I'll shut up the, the other point 140 00:09:12,650 --> 00:09:16,730 to make is that GPUs do go obsolete pretty quickly. 141 00:09:16,730 --> 00:09:20,050 The GPUs that are hot today might not be hot in a year's time. 142 00:09:20,050 --> 00:09:26,530 And if you buy one, you're kind of stuck with it, but with with Google Colab, you just get the whatever 143 00:09:26,530 --> 00:09:27,570 they have on offer. 144 00:09:27,850 --> 00:09:30,290 They have a range of different boxes that you can get. 145 00:09:30,290 --> 00:09:32,690 And so it's also a nice way to always stay current. 146 00:09:32,730 --> 00:09:37,770 I guess it's like leasing a car, you know, it's like it's the buy versus versus lease kind of conundrum 147 00:09:37,770 --> 00:09:38,730 that people think about. 148 00:09:38,730 --> 00:09:39,410 Anyways. 149 00:09:39,570 --> 00:09:41,050 That is Google Colab. 150 00:09:41,090 --> 00:09:41,330 All right. 151 00:09:41,330 --> 00:09:45,090 So I just took like ten minutes of your time just talking about this thing. 152 00:09:45,090 --> 00:09:46,330 We need to go and see it. 153 00:09:46,330 --> 00:09:47,450 Let's go and see it. 154 00:09:47,450 --> 00:09:54,010 So look there's people constantly ask me how to find the link to the Google Colab that we use in these 155 00:09:54,010 --> 00:09:54,490 courses. 156 00:09:54,490 --> 00:10:02,090 And I've put the links everywhere, but but so one place they are is that if you go into the repo week 157 00:10:02,130 --> 00:10:08,570 three, day one, there is a notebook there in in cursor with a link to the Colab. 158 00:10:08,690 --> 00:10:11,850 But also if you look in the readme, there's a link to the Colab. 159 00:10:11,850 --> 00:10:14,810 If you look in the class resources, there's a link to the Colab. 160 00:10:14,850 --> 00:10:19,610 Like I don't know, you have to just not look anywhere, look up at the ceiling and you won't see it, 161 00:10:19,610 --> 00:10:23,130 but it's everywhere else, so you got no excuses. 162 00:10:23,130 --> 00:10:26,330 You can find the link to the colab and I will see you right there.