Well, here we are. 好吧，我们到了。
It is week seven. 现在是第七周。
Day five. 第五天。
Somehow it's the big reveal. 不知怎的，这是一个重大的揭露。
The results day. 出结果的日子。
Uh, but first, a reminder. 呃，但首先，提醒一下。
You can already work with open source, closed source models, with calling tools, with with assistance 您已经可以使用开源、闭源模型、调用工具以及帮助
with Rag five step strategy to solving a business problem, including curating data, fine tuning a 使用 Rag 五步策略来解决业务问题，包括整理数据、微调
frontier model, and now running Q Laura for an open source model including hyperparameter optimization, 前沿模型，现在运行 Q Laura 用于开源模型，包括超参数优化，
monitoring the training, and the fabulous weights and biases. 监控训练以及惊人的权重和偏差。
Hopefully, hopefully you didn't get as overexcited as me, but at least you enjoyed the process of 希望，希望你没有像我一样过度兴奋，但至少你喜欢这个过程
doing it. 做它。
And today, of course, it's results day. 当然，今天是结果日。
We're going to to run inference on a model which has been fine tuned. 我们将对经过微调的模型进行推理。
So with Laura matrices and stuff. 劳拉矩阵之类的东西也是如此。
So there's some some things to cover there and wrap up with, you know, so you're in a position that 所以有一些事情需要涵盖并总结，你知道，所以你处于这样的位置：
you could confidently do all of this yourself. 您可以自信地自己完成这一切。
That's the big part of it. 这是其中最重要的部分。
Um, and I'm, I'm excited to show you the results. 嗯，我很高兴向您展示结果。
Of course, what we're trying to, to get towards is get something that is close to the performance 当然，我们正在努力实现的是接近性能的东西
of a frontier model. 的前沿模型。
Recognizing we're working with llama 3.2, a 3 billion parameter model, a tiny model by by comparison 认识到我们正在使用 llama 3.2，这是一个拥有 30 亿参数的模型，相比之下，这是一个很小的模型
to multi-trillion dollar frontier models, recognizing that we've quantized it down to four bits so 到数万亿美元的前沿模型，认识到我们已经将其量化为四位，因此
it could run on a mobile phone pretty much for sure. 它几乎肯定可以在手机上运行。
And so it's very small. 所以它很小。
The question is, can we get close to frontier performance at least? 问题是，我们至少能接近前沿性能吗？
Can it beat Ed? 能打败艾德吗？
It should at least be able to beat my my abilities. 至少应该能够击败我的能力。
Maybe it could get close to GPT four one nano or something like that. 也许它可以接近 GPT 四一纳米或类似的东西。
That that's that's what we're striving for when you're trying to fine tune an open source model because 当您尝试微调开源模型时，这就是我们所努力的目标，因为
it's free and it's something that that you could use. 它是免费的，并且是您可以使用的东西。
Okay. 好的。
But first, before we we do that before we run inference, I do want to to close the loop on something 但首先，在我们进行推理之前，我确实想关闭某些东西的循环
that I've had hanging out there for a while, which is explaining what exactly is this loss calculation 我已经在那里闲逛了一段时间，这解释了这个损失计算到底是什么
that we we keep covering. 我们一直在报道。
What is the loss? 损失是什么？
How is it calculated. 是如何计算的。
Just a bit more detail on training. 只是关于训练的更多细节。
So a few theory slides before we get into action okay. 在我们开始行动之前先先看一些理论幻灯片。
Here we go. 开始了。
You remember the four steps of training. 您记住训练的四个步骤。
You know them well now. 你现在很了解他们了。
We coded them one by one. 我们将它们一一编码。
Uh, a week ago. 呃，一周前。
But now we know that you first do a forward pass. 但现在我们知道你首先要向前传球。
It's when you have some input data and you predict what will be the next token by going through that 当你有一些输入数据并且你通过这些数据来预测下一个标记是什么时
input data, the loss calculation, some calculation that says how wrong were we. 输入数据，损失计算，一些计算表明我们是多么错误。
You do the backward pass, which is when we say okay, if we tweak each of these parameters, does it 你进行向后传递，这就是我们说好的，如果我们调整每个参数，是吗？
make the loss better or worse? 使损失变得更好还是更糟？
And then we take a tiny step in the direction that reduces loss. 然后我们朝着减少损失的方向迈出一小步。
It's called the optimizer. 它被称为优化器。
You remember we used an optimizer called Adam Adam w I think we used which is very popular. 您还记得我们使用了一个名为 Adam Adam 的优化器，我认为我们使用的非常流行。
The, the original one is called SGD stochastic gradient descent. 最初的一种称为 SGD 随机梯度下降。
That's the one which is basically just saying take the learning rate times the gradients and going the 这基本上就是说用学习率乘以梯度，然后
other way and negative learning rate gradients take a step. 其他方式和负学习率梯度采取一步。
That's it. 就是这样。
So those are the four steps in training. 这就是训练的四个步骤。
Uh, you've experienced this a few times now. 呃，你已经经历过几次这样的事情了。
And what we're going to dig into is this loss calculation. 我们要深入研究的是这个损失计算。
And just a bit more perspective on this, this this whole idea. 对这整个想法有更多的看法。
So again the first step is the forward pass. 因此，第一步是向前传球。
So there's an input prompt input sequence like price is dollars. 所以有一个输入提示输入序列，比如价格是美元。
And then that's the next token. 然后就是下一个标记。
It's got to predict that is an input sequence. 它必须预测这是一个输入序列。
It's turned into token IDs. 它变成了令牌 ID。
Those token IDs are fed in to a a neural network which consists of a base llama 3.2 and then some Lora 这些令牌 ID 被输入神经网络，该网络由基础 llama 3.2 和一些 Lora 组成
adapters which have some some things which will able to adapt, able to slightly tune llama that get 适配器有一些能够适应的东西，能够稍微调整美洲驼
applied. 应用。
What comes spitting out at the end is the predicted next token. 最后吐出的是预测的下一个标记。
Let's say it says the price is $99. 假设它说价格是 99 美元。
It predicts 99 is the most likely next token. 它预测 99 是最有可能的下一个标记。
That's the forward pass. 这就是向前传球。
The loss calculation comes next. 接下来是损失计算。
The loss calculation take the predicted next token is 99. 损失计算取预测的下一个 token 为 99。
Let's say this thing actually cost $89. 假设这个东西实际上花了 89 美元。
We're off by $10. 我们优惠 10 美元。
So we need to calculate some loss. 所以我们需要计算一些损失。
And that loss has got to be related to how wrong were we if we were exactly right. 如果我们完全正确的话，这种损失一定与我们的错误程度有关。
If it was a perfect prediction, then the loss should be zero representing no error. 如果这是一个完美的预测，那么损失应该为零，代表没有错误。
It was perfect. 太完美了。
If if the if it's a if we're more wrong, the worse we are, the higher the loss should be. 如果我们错得越多，我们的情况就越糟糕，损失就应该越大。
Okay. 好的。
A bigger number means worse result. 数字越大意味着结果越差。
Sure. 当然。
You're thinking so you might be thinking, okay, why not? 你可能会想，好吧，为什么不呢？
How about we take the difference between them? 我们来看看它们之间的区别怎么样？
The absolute error. 绝对错误。
That's what we've been using all this time. 这就是我们一直在使用的。
Or maybe the mean squared error that linear regression uses. 或者可能是线性回归使用的均方误差。
But there's a problem with that that we will come to. 但我们会遇到一个问题。
But you get the general idea. 但你已经了解了总体思路。
That's what loss is all about. 这就是损失的意义所在。
Then we do something called the backward pass, which again is saying, okay, let's go to every single 然后我们做一些叫做向后传递的事情，这又是说，好吧，让我们看看每一个
parameter and say if we were to twiddle it a bit, would that make loss bigger or smaller. 参数并说如果我们稍微调整一下，损失会更大还是更小。
And that of course is called calculating the gradients because it's like a calculus thing and that we 这当然被称为计算梯度，因为它就像微积分一样，我们
only have to do that for the Laura adapters because we've frozen all of the base model. 只需对 Laura 适配器执行此操作，因为我们已经冻结了所有基本模型。
So we're only doing this, this tweaking for these, these smaller Laura adapters, they are being subjected 所以我们只是这样做，对这些较小的劳拉适配器进行调整，它们正在受到影响
to gradient based optimization. 基于梯度的优化。
It's how you'd explain it. 这就是你要解释的方式。
And doing this is something that would be very laborious and complex calculation involving tons and 这样做是一项非常费力且复杂的计算，涉及大量和
tons of maths. 大量的数学。
But there is a clever trick. 但有一个巧妙的技巧。
A trick that's known as back prop or back propagation, which is a way to calculate the gradients, 一种称为反向传播或反向传播的技巧，这是一种计算梯度的方法，
calculate how sensitive everything is to being just changed slightly by working from from the the the 计算一切对稍微改变的敏感度，从
bottom of the neural network, working all the way back up to the inputs again, and calculating all 神经网络的底部，再次回到输入，并计算所有
the gradients as a function of the gradients that came before. 梯度作为之前梯度的函数。
And it's because it uses a mathematical trick called the chain rule, which is something that you might 这是因为它使用了一种称为链式法则的数学技巧，您可能会
remember from high school, like a calculus thing, which is how you're able to calculate gradients 记得高中时，就像微积分一样，这就是你如何计算梯度的
in terms of other gradients. 就其他梯度而言。
And so it uses this approach of repeatedly applying the chain rule working backwards. 因此它使用了这种反复应用反向链式法则的方法。
And this was something that was it was invented some time ago. 这是不久前发明的东西。
But this this trick is one of the sort of secrets that has made, uh, made training neural networks 但这这个技巧是训练神经网络的秘密之一
so effective. 如此有效。
It's such an efficient way to do it that an operation that would have been very, very time consuming 这是一种非常有效的方法，以至于原本会非常非常耗时的操作
can happen rapidly. 可能会迅速发生。
And particularly it can happen very efficiently, in parallel rapidly. 特别是它可以非常高效、快速地并行发生。
And so we're able to calculate the gradients in this thing called back propagation or back prop. 因此我们能够计算称为反向传播或反向传播的梯度。
And all of that is known as the backward pass. 所有这些都称为向后传递。
Okay. 好的。
So that's backprop. 这就是反向传播。
And you've probably heard that expression. 你可能听说过这个表达方式。
If you want to know more about backprop then then it's easy to look it up and get more detail. 如果您想了解更多有关反向传播的信息，那么很容易查找并获得更多详细信息。
But I don't want to get too into the gory detail here. 但我不想在这里过多讨论血淋淋的细节。
It's actually the the algorithm itself is very old. 实际上算法本身已经非常古老了。
I think it's from the 1970s or so, but it was in the 1980s, I think 1986 that that it was there was 我认为它是从 1970 年代左右开始的，但那是在 1980 年代，我认为 1986 年就在那里了
the very famous paper. 非常有名的论文。
One of the authors was Geoff Hinton, who is like a, you know, considered one of the godfathers of 其中一位作者是杰夫·辛顿 (Geoff Hinton)，他被认为是《科学》的教父之一。
modern AI, uh, and that that really sort of brought it to the forefront of using it for neural networks 现代人工智能，呃，这确实将其带到了将其用于神经网络的最前沿
in AI. 在人工智能中。
Um, so backprop super essential. 嗯，所以反向传播非常重要。
All right. 好的。
And then optimization is the fourth step. 然后优化是第四步。
And this is where we take all of the adapters and we slightly step them in the direction. 这就是我们取出所有适配器的地方，并稍微朝这个方向移动它们。
That's going to mean that next time the loss will be less okay. 这意味着下次损失会更严重。
You get it. 你明白了。
You know about these four steps. 这四个步骤你都知道。
So now let me go one level deeper deeper so that I can explain what exactly is the loss calculation 现在让我更深入地解释一下损失计算到底是什么
itself. 本身。