The other big topic is Google Colab, and that is the other thing that we're going to be using today
as we we it's all about new stuff today.
So Google Colab, you may have already used it.
A lot of people have used it already and they're like, why don't we start with Google Colab?
There's a lot of a lot to love about Google Colab.
First of all, what is it?
So it is something which allows you to to run a notebook.
Notebook is this kind of interface we've been playing with in cursor when you have like, like cells
of code that you run a cell at a time, there's very experimental way of interacting with code.
So Google has built something that lets you run a notebook like that, except you're running it in a
browser, you're running it in a browser window, and it's actually running on one of Google's computers.
So you can like bring up a like a Chrome page, point it at a Google URL, and then you effectively
have like like a boxes there, and you're remoting in to a Google box and you're able to run the code.
And when you run the cell, it's as if it's running in your browser, but of course it's not.
The code is executing on one of Google's machines out there in the cloud, and you're seeing the results
immediately.
So it lets you remote in to a to a remote box running on Google's cloud.
And there are a few things about this that are really great.
One of them is that it allows you to collaborate in this super seamless way with other people on the
same code.
Like in the same notebook construct.
Google is famous, of course, for Google Docs for really bringing the kind of collaborative office
idea, uh, to, to the mainstream.
And so everyone's very familiar with this idea that you can share a Google doc and share a sheet and
be able to to work together with someone else in it and really collaborate.
And Google Colab is bringing that same mentality to a notebook so that you can be coding with somebody
else.
It's not quite you don't quite get to see the changes in quite the same way, but it's almost as good
as that.
And you can both be adding code and running things.
And so it's such a great way to work together with people and, you know, it's like a it's a different
mindset to something like a using a repo where you where you check stuff in and merge it and so on.
It's much more like live working, but it suits the notebook mindset, that experimental mindset, it
works so well with that.
So very collaborative as as the name suggests.
Uh, and it's also integrated with, with the other Google services.
So, so later this this week we're going to to pull Google Drive in so you can have your Google Drive
docs available there too, which is very convenient.
So for many reasons it is convenient.
But above all else the real the.
So what is that?
It gives you access to high end GPUs.
That's that's the big deal.
And I probably need to mention, just in case you don't, uh, you haven't got the connection between
what we're doing and GPUs.
I mean, obviously everyone knows about Nvidia stock and that GPUs are at the heart of, of modern AI.
But but if you're wondering why, when we talk about these models with these huge number of parameters,
these tons and tons, these 8 billion numbers, when you when you're predicting the most likely next
token, the way you do that is by taking all of these numbers and multiplying them repeatedly by the
sort of input sets of tokens in some very clever way, which is known as the transformer architecture,
which is the way that these things will hook together and you're doing lots of multiplications and adding
multiplications and adding is basically matrix math, linear algebra.
It's lots and lots of matrix calculations, and they can all happen very efficiently in parallel with
one another.
They're all sort of independent.
So they can all run together in this very, very efficient way.
But they all have to run.
And it turns out when, when all this is becoming a big thing, in the early 2000, people were saying
what we really need is like specialized custom hardware that is that is efficient, that is designed
for matrix calculations in in fast, real time.
And you can imagine all of these scientists sitting around there thinking about this.
And while they were doing it, the other people in the room were busy playing Doom or Quake or whatever
was big in those days and, uh, using using PCs with which were like hammering away at drawing this
3D graphics.
And the way that you do these kinds of games is by doing a lot of quick polygon calculations, which
involves lots of efficient matrix maths.
So the graphics cards, everyone was trying to buy a PC with a bigger graphics card.
Those graphics cards were basically custom hardware designed to run matrix calculations, multiplies
and adds in parallel like huge numbers of them at the same time so that it could draw, uh, screens
like 60 times a second or whatever.
Uh, and so it turned out that this kind of hardware was absolutely perfect for the kinds of AI that
we were doing at the time and that we are still doing to this day.
Uh, and luckily for Nvidia.
And so that is why GPUs you probably do all that already.
So I'm sorry.
That's why GPUs are so essential to to modern day AI.
And they're quite expensive.
And what you really need more than anything is you need them to have a lot of memory, which typically
you need, like it's different to the memory of your computer.
The GPU also has Ram, as games players all know, and you need to have enough to fit all of the model
on.
And if you're training, you need a lot more than that too.
And so you need a lot of memory on your GPU for it to be able to run efficiently.
And that's a problem.
And as, uh, that's not a problem, that's expensive and is the constraint.
And Apple people will know.
Well, like I might happen to be a mac person, uh, that, uh, one of the great things about Apple
Silicon is that you have what's called a unified memory, which means that the memory that your CPU
uses is the same as the memory the GPU uses.
So you can you can share it together, which is really amazing.
But for PCs, you typically have to get a graphics card with big enough memory.
And they're expensive.
They are many thousands of dollars and boxes which have big Nvidia chips on on them.
Graphics cards are very expensive.
You can buy good ones for like $6,000 or something.
They're expensive, but there's an alternative.
There is an alternative and it's called Google Colab, where you rent GPUs in the cloud instead.
And so here are the details you get when you connect to Colab.
You choose a runtime and runtime is it's name of like the, the, uh, like the instance that you're
connecting to on the cloud.
And it includes like a kernel, like we're used to a Python process that's, that's running there.
And you can get some that are CPU based, meaning that there's no graphics card at all, there's no
GPU, so you can just use it for everyday tasks, a bit like you can on your own computer.
You can have ones with a lower spec GPU.
It's still pretty mighty.
It's still a powerful GPU.
The typical one is the Nvidia Tesla T4.
The T4 is the typical one.
It has 15GB of GPU Ram, which is a lot.
And here's the thing.
It's free.
It's it's completely free.
You can have a Google Colab with a T4 GPU, and you can have it for free at any point.
And we're just gonna do it in just a second.
Can you believe it?
And that's what you'll be using for this week, for week three and for week seven on this course when
we are running on GPUs in the cloud.
So that's amazing.
And then if you want to splash out you can pay to have a higher spec one, including an A100 for 40GB.
But when I say splash out, we're talking about like a few bucks an hour.
We're talking about like, you know, a Big Mac an hour kind of level.
I believe we'll do the maths later to see exactly how much it costs.
But, uh, it's it's, uh, I was going to say it's cheap.
And you might disagree with me there.
You might say a few dollars an hour is not cheap at all, but compare it with the cost of buying an
A100 on a PC that you're going to have for yourself.
That that is an expensive proposition.
And when you think of it in that light, to be able to run it for $2 an hour only when you need it,
and then you don't use it anymore, you don't keep it running.
You just you only pay for the minutes that you have it alive.
That's it.
It seems like it's actually a really great deal.
And I feel like we are lucky to have this ability to rent super high powered horsepower whenever we
want it.
And so, yeah, I think this is fabulous.
And that's why I always when people say to me, I want to buy a box with blah, blah, blah, blah,
I say, really, really, because you know that that you can have as many a100s as you want for a fraction
of that, uh, just, you know, every weekend.
So anyways, the, the other the final sales pitch for it and then I'll shut up the, the other point
to make is that GPUs do go obsolete pretty quickly.
The GPUs that are hot today might not be hot in a year's time.
And if you buy one, you're kind of stuck with it, but with with Google Colab, you just get the whatever
they have on offer.
They have a range of different boxes that you can get.
And so it's also a nice way to always stay current.
I guess it's like leasing a car, you know, it's like it's the buy versus versus lease kind of conundrum
that people think about.
Anyways.
That is Google Colab.
All right.
So I just took like ten minutes of your time just talking about this thing.
We need to go and see it.
Let's go and see it.
So look there's people constantly ask me how to find the link to the Google Colab that we use in these
courses.
And I've put the links everywhere, but but so one place they are is that if you go into the repo week
three, day one, there is a notebook there in in cursor with a link to the Colab.
But also if you look in the readme, there's a link to the Colab.
If you look in the class resources, there's a link to the Colab.
Like I don't know, you have to just not look anywhere, look up at the ceiling and you won't see it,
but it's everywhere else, so you got no excuses.
You can find the link to the colab and I will see you right there.