Well, hello. 嗯，你好。
I'm back. 我回来了。
And you're back too, which is a great thing. 你也回来了，这是一件很棒的事情。
It means it's time for us to start day three of our journey together. 这意味着我们一起开始第三天的旅程。
And day three, we've already done building an agentic workflow for the first time. 第三天，我们已经第一次完成了代理工作流程的构建。
And we've talked about patterns. 我们已经讨论过模式。
And now we're going to talk about orchestrating between LMS and many of them. 现在我们要讨论 LMS 和其中许多之间的编排。
So this is going to be a practical day. 所以这将是务实的一天。
You'll be pleased to hear. 你会很高兴听到的。
It's about time we got some more coding. 是时候我们进行更多编码了。
And we're going to be calling a lot of LMS. 我们将调用很多 LMS。
And I just want to of course, say a few things up front. 当然，我只想提前说几句话。
We're going to be calling both paid APIs and also open source models, and we're going to be calling 我们将调用付费 API 和开源模型，并且我们将调用
them both in the cloud and also open source models locally and doing it throughout this course. 它们既在云端，也在本地开源模型，并在整个课程中进行。
And I want to be clear that you have complete flexibility to decide which models you pick, at which 我想明确的是，您可以完全灵活地决定选择哪种型号、选择哪种型号
point I'm going to have coded it one way, but a great exercise is to take what I've done and apply 我将以一种方式对其进行编码，但一个很好的练习是采用我所做的并应用
it to other models. 到其他型号。
And if you don't want to spend a dime, you don't need to. 如果你不想花一毛钱，那就不需要。
You can do this all using local models. 您可以使用本地模型来完成这一切。
Although performance might vary, you can see the results I get from the models I use, and try and 尽管性能可能会有所不同，但您可以看到我从我使用的模型中获得的结果，并尝试
see what you can achieve with free open source models as well. 看看您可以使用免费的开源模型实现什么。
And if if you want more information on what it's like to select models, whether they're open or closed 如果您想了解有关选择模型的更多信息，无论它们是开放式还是封闭式
source, apply them and deploy them, then you should take a look at my other course. 源码，应用它们并部署它们，那么你应该看看我的其他课程。
We're not going to be covering that in detail here, because otherwise I feel like it's going to be 我们不会在这里详细讨论这一点，因为否则我觉得它会是
too much of a of a rabbit hole and we'll get distracted. 太多的兔子洞我们会分心。
So I'm assuming you're coming into this knowing about the different kinds of models and having a sense 所以我假设您了解不同类型的模型并有一定的感觉
of what makes sense for closed source, open source, and so on. 什么对闭源、开源等有意义。
If you don't have that, you can either just go along with it, just just sort of pick it up as we go. 如果你没有，你也可以跟着它走，只是在我们走的时候把它捡起来。
Or of course, you could always turn to look at my other course, or I'll try and put some more background 或者当然，您可以随时查看我的其他课程，或者我会尝试提供更多背景知识
information in the guides as well, because that won't be required. 指南中的信息也是如此，因为这不是必需的。
You can just just go, go with the flow in terms of how we pick models. 你可以随波逐流地选择模型。
So let's talk about the cast of characters, the different models that we're going to be experiencing 那么让我们来谈谈角色阵容以及我们将要体验的不同模型
now. 现在。
So the first model needs no introduction really. 所以第一个模型确实不需要介绍。
It is of course, the model GPT four mini Many from open AI. 当然是来自 open AI 的 GPT 4 mini 模型。
We've already used it in a couple of calls. 我们已经在几次通话中使用过它。
It's for sure the most well known of the models out there. 它肯定是目前最知名的模型。
And of course, there's also GPT four, the bigger cousin of GPT four. 当然，还有 GPT 4，它是 GPT 4 的近亲。
And then there are the reasoning models, which are models that have been trained to think through their 然后是推理模型，这些模型经过训练可以通过其自身的能力进行思考
steps in an Agentic like way, in like a workflow of thinking through the different steps before they 以类似于 Agentic 的方式进行步骤，就像在执行之前思考不同步骤的工作流程
arrive at their conclusion. 得出他们的结论。
Because it turns out that when you ask an LLM to think through its steps, you get much better outcomes. 因为事实证明，当你要求法学硕士仔细思考其步骤时，你会得到更好的结果。
So we may take a look at some point at oh one and oh three mini, but it's less essential for this course. 因此，我们可能会看一下哦一和哦三迷你的某个点，但这对于本课程来说不是那么重要。
Most of the time. 大多数时候。
We're going to be sticking with GPT four mini now and OpenAI's great rival. 我们现在将继续使用 GPT 4 mini，这是 OpenAI 的强大竞争对手。
Of course, their competitor is anthropic. 当然，他们的竞争对手是人类。
That was actually started by a couple of people from OpenAI originally, and we'll be looking at some 这实际上最初是由 OpenAI 的几个人发起的，我们将关注一些
of their models. 他们的模型。
But Claude 37 sonnet is the model that I will spend most time on. 但克劳德 37 十四行诗是我花最多时间研究的模型。
And if you want to have a cheaper version, you can go with the Claude three haiku, which is significantly 如果你想要一个更便宜的版本，你可以选择克劳德三俳句，这是显着的
lower cost. 成本更低。
But but Sonet is also fairly cheap for Google. 但 Sonet 对于 Google 来说也相当便宜。
We're going to be using Gemini two zero flash. 我们将使用 Gemini 两个零闪光。
There is also the Pro version of that too, but I think we'll stick with flash. 还有专业版，但我想我们会坚持使用闪光灯。
And as of right now, flash is actually free at least as long as you use it within certain usage limits. 截至目前，闪存实际上是免费的，至少只要您在一定的使用限制内使用它。
I don't know how long that will be the case for, but by all means, if you want to use an open frontier 我不知道这种情况会持续多久，但无论如何，如果你想使用开放的边界
model without paying for it, then Gemini might be your path. 模型而不付钱，那么双子座可能是你的道路。
Do look into that deep Sikh. 一定要研究一下那个深刻的锡克教徒。
Deep Sikh, of course, is the Chinese upstart startup that shocked us all by coming up with such a 当然，Deep Sikh 是一家中国新贵初创公司，它提出了这样一个让我们所有人震惊的方案。
powerful model in the form of Deep Sikh V3 and R1. Deep Sikh V3 和 R1 形式的强大模型。
And it's important to understand that what made Deep Sikh so sensational was not necessarily that their 重要的是要明白，让深锡克教徒如此轰动的并不一定是他们的
model was the strongest in the world, because it wasn't. 模型是世界上最强的，因为事实并非如此。
It was slightly behind the latest from open AI, but that they developed such powerful techniques to 它稍微落后于开放人工智能的最新成果，但他们开发了如此强大的技术
train Deep Sikh to be that good, that it cost them a fraction of the spend that OpenAI had spent to 训练 Deep Sikh 变得如此优秀，他们只花费了 OpenAI 花费的一小部分
train GPT four and train GPT 40101. 训练 GPT 4 并训练 GPT 40101。
Deep seek was able to achieve very similar performance, pretty much comparable at a fraction. 深度搜索能够实现非常相似的性能，几乎可以相媲美。
I think it's like 30 times less spend. 我认为这相当于减少了 30 倍的支出。
That was the true innovation. 这才是真正的创新。
That's the remarkable thing about deep seek. 这就是深度探索的非凡之处。
And also that they open sourced the model so that you can use it. 而且他们还开源了该模型，以便您可以使用它。
But the main the major model has 671 billion parameters, which means it's far too big for anyone to 但主要模型有 6710 亿个参数，这意味着它对于任何人来说都太大了
run that on their computers. 在他们的计算机上运行它。
But there are versions of it, small versions of it called the distilled versions, which are in fact 但它有一些版本，它的小版本称为蒸馏版本，实际上是
themselves just smaller models. 本身只是较小的模型。
They're versions of Llama and Quen, two different models that have been fine tuned on data generated 它们是 Llama 和 Quen 的版本，这两个不同的模型已根据生成的数据进行了微调
by the big Deep Seek and those smaller distilled versions of Deep Seek for sure available free of charge. 由大型 Deep Seek 和那些较小的 Deep Seek 精炼版本提供，当然可以免费使用。
Grok. 格洛克。
We will also be using grok, and there are two groks. 我们还将使用 grok，有两个 grok。
Confusingly, if you don't know this grok spelt with a k at the end is the name of the model that comes 令人困惑的是，如果您不知道最后拼写为 k 的 grok 是该模型的名称
from from the company formerly known as Twitter. 来自以前称为 Twitter 的公司。
Now x x is grok and we might use grok with a k at some point as well. 现在 x x 是 grok，我们也可能在某些时候将 grok 与 k 一起使用。
Not not in today's lab, but grok with a Q is something different. 在今天的实验室里并非如此，但用 Q 来理解却是不同的。
Grok with a Q is a company that has come up with a really cheap, fast way to run inference runtime Grok with a Q 是一家公司，它提出了一种非常便宜、快速的方式来运行推理运行时
models like llama 3.3, which is the massive version of llama with 70 billion parameters. 像 llama 3.3 这样的模型，它是 llama 的大规模版本，拥有 700 亿个参数。
So you can run llama 3.3 really fast, really low cost on Groks infrastructure and along with many other 因此，您可以在 Groks 基础设施以及许多其他基础设施上非常快速、非常低成本地运行 llama 3.3
open source models including Deep Seek variants. 开源模型，包括 Deep Seek 变体。
So grok is great to use for that. 所以 grok 非常适合用于此目的。
And then Olama. 然后是奥拉马。
So Olama is itself a more of a platform. 所以 Olama 本身更像是一个平台。
It's something that you can use to run something locally that provides endpoints locally that are consistent, 您可以使用它在本地运行某些内容，并在本地提供一致的端点，
very similar to the endpoints that OpenAI and other models here have so that you can make local calls 与 OpenAI 和其他模型的端点非常相似，因此您可以进行本地调用
to an API, which is in fact going to just run an open source model locally on your computer in high 到一个 API，它实际上只是在你的计算机上本地运行一个开源模型
performance optimized cplusplus using a library called llama CPP. 使用名为 llama CPP 的库对 cplusplus 进行了性能优化。
And so a llama is something that we will use as well. 所以我们也会使用美洲驼。
Now, if some of these terms are unfamiliar to you, terms like inference. 现在，如果您不熟悉其中一些术语，例如推理等术语。
If you're not sure about that, if you don't know, if you don't fully understand the difference between 如果你对此不确定，如果你不知道，如果你不完全理解两者之间的区别
running something over a llama or grok, then I can. 在美洲驼或大神身上跑一些东西，然后我就可以了。
I can suggest a background materials, look in the guides, and also consider whether you'd like to 我可以建议背景材料，查看指南，并考虑您是否愿意
look at my LLM engineering course, which does cover all of this. 看看我的法学硕士工程课程，它确实涵盖了所有这些内容。
And the final point that I will make is that you may know that I'm something of a fanatic on leaderboards, 我要说的最后一点是，你可能知道我是排行榜的狂热分子，
on places you can go to read about metrics and performance of different models. 在您可以阅读有关不同模型的指标和性能的地方。
So much so that I was called a leaderboard, uh, humorously by my my great friend John Crone on his 以至于我被我的好朋友 John Crone 幽默地称为排行榜。
Super Data Science podcast. 超级数据科学播客。
But there is a website called called leaderboard, which I've given the web address right there. 但是有一个名为排行榜的网站，我在那里给出了网址。
And that is a great resource because it compares a number of the leading closed source and open source 这是一个很好的资源，因为它比较了许多领先的闭源和开源
models together side by side. 模型并排在一起。
And it has things like the costs and it has the context window size, if you're familiar with that, 它有诸如成本和上下文窗口大小之类的东西，如果你熟悉的话，
and it has the cost, you can calculate based on the number of input and output tokens. 它是有成本的，你可以根据输入和输出代币的数量来计算。
And it's also got the performance, the results of key benchmarks across a number of dimensions. 它还获得了性能以及多个维度的关键基准测试结果。
So I strongly encourage people to go and check out the vellum leaderboard. 所以我强烈鼓励人们去查看牛皮纸排行榜。
And there'll be a link in the resources. 资源中会有一个链接。
And it's great to have that bookmarked up. 很高兴能把它加入书签。
And as you go through different APIs, use that to gauge the the costs and capabilities associated with 当您使用不同的 API 时，请使用它来衡量与以下内容相关的成本和功能：
them. 他们。
And now I'm going to put up one more time something that I said to you in the first lecture. 现在我将再次提出我在第一堂课中对你们说过的话。
But just to really double down on this, I want to remind you that there are great resources all over 但为了真正加倍努力，我想提醒您，世界各地都有丰富的资源
the place for this course. 本课程的地点。
There's a resource for the whole course with videos and links. 有包含视频和链接的整个课程的资源。
There's GitHub, the repo that has guides. GitHub 是一个包含指南的存储库。
It has the troubleshooting. 它有故障排除。
And I'm always updating the labs to keep them up to date. 我总是更新实验室以使其保持最新状态。
I might add on new models, the models that we just went through a moment ago. 我可能会添加新模型，即我们刚才讨论过的模型。
Those are the models that right now are the latest and greatest. 这些是目前最新、最好的型号。
But as new versions come out, I will update the labs. 但随着新版本的出现，我将更新实验室。
So you've got the latest in there. 所以你已经得到了最新的信息。
And so, you know, what you've heard is, is is one story, but you'll get an even better story when 所以，你知道，你听到的是一个故事，但当你听到时，你会得到一个更好的故事
you go through the labs yourselves. 你自己去实验室。
And I will also urge you to keep a thick skin. 我也会劝你保持厚脸皮。
There's that. 就是这样。
You will hit roadblocks. 你会遇到障碍。
There will be problems. 会有问题。
That's one thing I can say for sure, but see them as this is where real learning happens. 这是我可以肯定地说的一件事，但请把它们视为真正学习发生的地方。
It's in debugging. 正在调试中。
It's in diagnosing and figuring things out, even if it's painful to start with. 它在于诊断和解决问题，即使一开始很痛苦。
It's super satisfying when you fix it and get it on track, and that is the way to do it. 当你修复它并使其走上正轨时，这是非常令人满意的，这就是做到这一点的方法。
And and if all else fails, or as I say, or if not all else fails, you just want to then reach out 如果所有其他方法都失败了，或者正如我所说，或者如果不是所有其他方法都失败了，那么您只想伸出援手
and contact me. 并联系我。
Of course, one more time. 当然，还有一次。
I've got my LinkedIn right there, but also you can email me. 我的 LinkedIn 就在那里，您也可以给我发电子邮件。
I've got my details all over the place and I like hearing from people. 我到处都有我的详细信息，而且我喜欢听取人们的意见。
I'm responsive. 我有反应。
But most importantly, I don't want you to be suffering in pain with problems. 但最重要的是，我不希望你因问题而遭受痛苦。
I want to be fixing them. 我想修复它们。
So keep this in mind. 所以请记住这一点。
And also, I mean ask ChatGPT you do tend to get great, great answers. 而且，我的意思是询问 ChatGPT，您确实往往会得到非常非常好的答案。
Honestly, it's amazing and I tend to. 老实说，这太棒了，我也倾向于这样做。
When I have a really difficult question, I will often ask ChatGPT and Claude, both because, uh, 当我有一个非常困难的问题时，我经常会问 ChatGPT 和 Claude，因为，呃，
sometimes you get answers that are too long winded or take you in too many different directions, and 有时你得到的答案过于冗长或带你走向太多不同的方向，并且
so asking a couple of different models can help. 因此询问几个不同的型号会有所帮助。
And you can also basically have like an agent workflow that you do manually. 您基本上也可以像手动执行的代理工作流程一样。
You can ask a question to ChatGPT and then to Claude, you can say, I've got this response to my question. 你可以向 ChatGPT 提问，然后向 Claude 提问，你可以说，我的问题得到了这样的答复。
Do you agree? 你同意？
Is this accurate? 这准确吗？
You can have it be the evaluator, like the evaluator optimizer pattern that we looked at a moment ago. 您可以将其作为评估器，就像我们刚才看到的评估器优化器模式一样。
So that's a that's really cool that you can do it manually. 所以这真的很酷，你可以手动完成。
And it's a great technique to get good answers from Llms when you're stuck yourself. 当你自己陷入困境时，这是一个很好的技巧，可以从法学硕士那里得到很好的答案。
Okay. 好的。
With that preamble, it's time for our next lab. 有了这个序言，我们的下一个实验室就到了。
Let's get to it. 让我们开始吧。