diff --git a/Translated_Book/ch05/5.2.ipynb b/Translated_Book/ch05/5.2.ipynb new file mode 100644 index 0000000..183d9ca --- /dev/null +++ b/Translated_Book/ch05/5.2.ipynb @@ -0,0 +1,380 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e63ef76e", + "metadata": {}, + "source": [ + "# 5.2 训练一个LLM" + ] + }, + { + "cell_type": "markdown", + "id": "c1d79d88", + "metadata": {}, + "source": [ + "​\t\t在本节中，我们最终实现了用于预训练 LLM 的代码，即我们的 GPTModel。为此，我们专注于一个简单的训练循环，如图 5.11 所示，以保持代码简洁易读。但是，有兴趣的读者可以在附录 D，向训练循环添加花里胡哨中了解更高级的技术，包括学习速率预热、余弦退火和梯度削波。" + ] + }, + { + "cell_type": "markdown", + "id": "16882c7e", + "metadata": {}, + "source": [ + "图 5.11 在 PyTorch 中训练深度神经网络的典型训练循环由几个步骤组成，在训练集中的批次上迭代多个时期。在每个循环中，我们计算每个训练集批次的损失以确定损失梯度，我们用它来更新模型权重，以便将训练集损失降至最低。" + ] + }, + { + "cell_type": "markdown", + "id": "9791e0ca", + "metadata": {}, + "source": [ + "![image-20240422143154243](../img/image-5-11.png)" + ] + }, + { + "cell_type": "markdown", + "id": "becf954d", + "metadata": {}, + "source": [ + "​\t\t图 5.11 中的流程图描述了一个典型的 PyTorch 神经网络训练工作流程，我们用它来训练 LLM。它概述了八个步骤，从迭代每个时期开始，处理批处理，重置和计算梯度，更新权重，最后是监控步骤，如打印损失和生成文本样本。如果您对使用 PyTorch 训练深度神经网络比较陌生，并且不熟悉其中任何一个步骤，请考虑阅读附录 A，PyTorch 简介中的 A.5 至 A.8 部分。" + ] + }, + { + "cell_type": "markdown", + "id": "a5e6748c", + "metadata": {}, + "source": [ + "​\t\t在代码中，我们可以通过以下train_model_simple函数实现此训练流程：" + ] + }, + { + "cell_type": "markdown", + "id": "c29235aa", + "metadata": {}, + "source": [ + "**Listing 5.3 预训练 LLM 的主要功能**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ee1deba6", + "metadata": {}, + "outputs": [], + "source": [ + "def train_model_simple(model, train_loader, val_loader, optimizer, device, num_epochs,\n", + "eval_freq, eval_iter, start_context):\n", + " \n", + "\ttrain_losses, val_losses, track_tokens_seen = [], [], [] #A\n", + "\ttokens_seen, global_step = 0, -1\n", + "\tfor epoch in range(num_epochs): #B\n", + " model.train()\n", + " for input_batch, target_batch in train_loader:\n", + " optimizer.zero_grad() #C\n", + " loss = calc_loss_batch(input_batch, target_batch, model, device)\n", + " loss.backward() #D\n", + " optimizer.step() #E\n", + " tokens_seen += input_batch.numel()\n", + " global_step += 1\n", + " if global_step % eval_freq == 0: #F\n", + " train_loss, val_loss = evaluate_model(\n", + " model, train_loader, val_loader, device, eval_iter)\n", + " train_losses.append(train_loss)\n", + " val_losses.append(val_loss)\n", + " track_tokens_seen.append(tokens_seen)\n", + " print(f\"Ep {epoch+1} (Step {global_step:06d}): \"\n", + " f\"Train loss {train_loss:.3f}, Val loss {val_loss:.3f}\")\n", + " generate_and_print_sample( #G\n", + " \tmodel, train_loader.dataset.tokenizer, device, start_context\n", + " )\n", + "\treturn train_losses, val_losses, track_tokens_seen" + ] + }, + { + "cell_type": "markdown", + "id": "1a5cf34a", + "metadata": {}, + "source": [ + "​\t\t请注意，我们刚刚创建的 train_model_simple 函数使用了两个尚未定义的函数：evaluate_model 和 generate_and_print_sample。\n", + "\n", + "​\t\tevaluate_model 函数对应于图 5.11 中的步骤 7。它会在每次模型更新后打印训练和验证集损失，以便我们可以评估训练是否改进了模型。\n", + "\n", + "​\t\t更具体地说，evaluate_model函数计算训练和验证集的损失，同时确保模型处于评估模式，在计算训练和验证集的损失时禁用梯度跟踪和辍学：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "42a2623b", + "metadata": {}, + "outputs": [], + "source": [ + "def evaluate_model(model, train_loader, val_loader, device, eval_iter):\n", + " model.eval() #A\n", + " with torch.no_grad(): #B\n", + " train_loss = calc_loss_loader(train_loader, model, device, num_batches=eval_iter)\n", + " val_loss = calc_loss_loader(val_loader, model, device, num_batches=eval_iter)\n", + " model.train()\n", + " return train_loss, val_loss" + ] + }, + { + "cell_type": "markdown", + "id": "dc84e85a", + "metadata": {}, + "source": [ + "​\t\t与 evaluate_model 类似，generate_and_print_sample 函数是一个方便函数，我们用它来跟踪模型在训练过程中是否改进。具体而言，generate_and_print_sample 函数将文本片段 （start_context） 作为输入，将其转换为令牌 ID，并将其提供给 LLM，以使用我们之前使用的 generate_text_simple 函数生成文本示例：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "599d2e5a", + "metadata": {}, + "outputs": [], + "source": [ + "def generate_and_print_sample(model, tokenizer, device, start_context):\n", + " model.eval()\n", + " context_size = model.pos_emb.weight.shape[0]\n", + " encoded = text_to_token_ids(start_context, tokenizer).to(device)\n", + " with torch.no_grad():\n", + " token_ids = generate_text_simple(\n", + " model=model, idx=encoded,\n", + " max_new_tokens=50, context_size=context_size\n", + " )\n", + " decoded_text = token_ids_to_text(token_ids, tokenizer)\n", + " print(decoded_text.replace(\"\\n\", \" \")) # Compact print format\n", + " model.train()" + ] + }, + { + "cell_type": "markdown", + "id": "5122ad4c", + "metadata": {}, + "source": [ + "​\t\t虽然 evaluate_model 函数为我们提供了模型训练进度的数字估计，但这个generate_and_print_sample文本函数提供了模型生成的具体文本示例，用于判断其在训练期间的能力。" + ] + }, + { + "cell_type": "markdown", + "id": "2feb5ef2", + "metadata": {}, + "source": [ + "**AdamW**" + ] + }, + { + "cell_type": "markdown", + "id": "3e324e55", + "metadata": {}, + "source": [ + "​\t\tAdam 优化器是训练深度神经网络的热门选择。但是，在我们的训练循环中，我们选择了 AdamW 优化器。AdamW 是 Adam 的一个变体，它改进了权重衰减方法，旨在通过惩罚更大的权重来最大限度地降低模型复杂性并防止过度拟合。这种调整使 AdamW 能够实现更有效的正则化和更好的泛化，因此经常用于 LLM 的训练。" + ] + }, + { + "cell_type": "markdown", + "id": "cfea4934", + "metadata": {}, + "source": [ + "​\t\t让我们通过使用 AdamW 优化器和我们之前定义的 train_model_simple 函数训练 10 个 epoch 的 GPTModel 实例来了解这一切。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "57ca345c", + "metadata": {}, + "outputs": [], + "source": [ + "torch.manual_seed(123)\n", + "model = GPTModel(GPT_CONFIG_124M)\n", + "model.to(device)\n", + "optimizer = torch.optim.AdamW(model.parameters(), lr=0.0004, weight_decay=0.1) #A\n", + "num_epochs = 10\n", + "train_losses, val_losses, tokens_seen = train_model_simple(\n", + " model, train_loader, val_loader, optimizer, device,\n", + " num_epochs=num_epochs, eval_freq=5, eval_iter=1,\n", + " start_context=\"Every effort moves you\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "1b89adb6", + "metadata": {}, + "source": [ + "​\t\t执行 training_model_simple 功能将启动训练过程，在 MacBook Air 或类似笔记本电脑上大约需要 5 分钟才能完成。在此执行过程中打印的输出如下：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35a1d65a", + "metadata": {}, + "outputs": [], + "source": [ + "Ep 1 (Step 000000): Train loss 9.781, Val loss 9.933\n", + "Ep 1 (Step 000005): Train loss 8.111, Val loss 8.339\n", + "Every effort moves you,,,,,,,,,,,,.\n", + "Ep 2 (Step 000010): Train loss 6.661, Val loss 7.048\n", + "Ep 2 (Step 000015): Train loss 5.961, Val loss 6.616\n", + "Every effort moves you, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and, and,, and, and,\n", + "[...] #A\n", + "Ep 9 (Step 000080): Train loss 0.541, Val loss 6.393\n", + "Every effort moves you?\" \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\" He laughed again, and threw back the window-curtains, I had the donkey. \"There were days when I\n", + "Ep 10 (Step 000085): Train loss 0.391, Val loss 6.452\n", + "Every effort moves you know,\" was one of the axioms he laid down" + ] + }, + { + "cell_type": "markdown", + "id": "3fd848ba", + "metadata": {}, + "source": [ + "​\t\t正如我们所看到的，根据训练期间打印的结果，训练损失急剧改善，从值 9.558 开始，收敛到 0.762。该模型的语言技能有了很大的提高。在开始时，模型只能将逗号附加到开始上下文中（“Every effort moves you,,,,,,,,,,,,”）或重复单词“and”。在训练结束时，它可以生成语法正确的文本。" + ] + }, + { + "cell_type": "markdown", + "id": "3264949e", + "metadata": {}, + "source": [ + "​\t\t与训练集损失类似，我们可以看到验证损失从高处开始 （9.856），并在训练期间减少。但是，它永远不会变得像训练集损失那么小，并且在第 10 个纪元之后保持在 6.372。" + ] + }, + { + "cell_type": "markdown", + "id": "24d42eaa", + "metadata": {}, + "source": [ + "​\t\t在更详细地讨论验证损失之前，让我们创建一个简单的图，并排显示训练集和验证集损失：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1380f03f", + "metadata": {}, + "outputs": [], + "source": [ + "import matplotlib.pyplot as plt\n", + "def plot_losses(epochs_seen, tokens_seen, train_losses, val_losses):\n", + " fig, ax1 = plt.subplots(figsize=(5, 3))\n", + " ax1.plot(epochs_seen, train_losses, label=\"Training loss\")\n", + " ax1.plot(epochs_seen, val_losses, linestyle=\"-.\", label=\"Validation loss\")\n", + " ax1.set_xlabel(\"Epochs\")\n", + " ax1.set_ylabel(\"Loss\")\n", + " ax1.legend(loc=\"upper right\")\n", + " ax2 = ax1.twiny() #A\n", + " ax2.plot(tokens_seen, train_losses, alpha=0) #B\n", + " ax2.set_xlabel(\"Tokens seen\")\n", + " fig.tight_layout()\n", + " plt.show()\n", + "epochs_tensor = torch.linspace(0, num_epochs, len(train_losses))\n", + "plot_losses(epochs_tensor, tokens_seen, train_losses, val_losses)" + ] + }, + { + "cell_type": "markdown", + "id": "c7aa2922", + "metadata": {}, + "source": [ + "​\t\t得到的训练和验证损失图如图 5.12 所示。" + ] + }, + { + "cell_type": "markdown", + "id": "8840557b", + "metadata": {}, + "source": [ + "图 5.12 在训练开始时，我们观察到训练集和验证集的损失都急剧减少，这表明模型正在学习。但是，训练集损失在第二个时期之后继续减少，而验证损失停滞不前。这表明模型仍在学习，但它与第 2 期之后的训练集过度拟合。" + ] + }, + { + "cell_type": "markdown", + "id": "06408771", + "metadata": {}, + "source": [ + "![image-20240422144030197](..\\img\\image-5-12.png)" + ] + }, + { + "cell_type": "markdown", + "id": "7a6cf1c9", + "metadata": {}, + "source": [ + "​\t\t如图 5.12 所示，在第一个时期，训练和验证损失都开始改善。然而，损失开始分化超过第二个时代。这种背离以及验证损失远大于训练损失的事实表明模型对训练数据过度拟合。我们可以通过搜索生成的文本片段来确认模型逐字记住了训练数据，例如“The Verdict”文本文件中的“对讽刺非常不敏感”。" + ] + }, + { + "cell_type": "markdown", + "id": "1ac40e02", + "metadata": {}, + "source": [ + "​\t\t这种记忆是意料之中的，因为我们正在使用一个非常非常小的训练数据集，并为多个时期训练模型。通常，通常只针对一个时期在更大的数据集上训练模型。" + ] + }, + { + "cell_type": "markdown", + "id": "157c5d36", + "metadata": {}, + "source": [ + "​\t\t如前所述，感兴趣的读者可以尝试在古腾堡计划的 60,000 本公共领域书籍上训练模型，其中不会发生这种过度拟合;详见附录B。" + ] + }, + { + "cell_type": "markdown", + "id": "3e4a2943", + "metadata": {}, + "source": [ + "​\t\t在下一节中，如图 5.13 所示，我们将探讨 LLM 采用的采样方法来减轻记忆效应，从而生成更新颖的文本。" + ] + }, + { + "cell_type": "markdown", + "id": "a9604871", + "metadata": {}, + "source": [ + "图 5.13 我们的模型在实现训练函数后可以生成连贯的文本。但是，它经常逐字记住训练集中的段落。以下部分介绍生成更多样化输出文本的策略。" + ] + }, + { + "cell_type": "markdown", + "id": "6937bae6", + "metadata": {}, + "source": [ + "![image-20240422144152449](../img/image-5-13.png)" + ] + }, + { + "cell_type": "markdown", + "id": "d963ca66", + "metadata": {}, + "source": [ + "​\t\t如图 5.13 所示，下一节将介绍 LLM 的文本生成策略，以减少训练数据记忆并提高 LLM 生成文本的原创性，然后我们介绍权重加载以及保存和加载来自 OpenAI 的 GPT 模型的预训练权重。" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.9" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Translated_Book/img/image-5-11.png b/Translated_Book/img/image-5-11.png new file mode 100644 index 0000000..ccf2d23 Binary files /dev/null and b/Translated_Book/img/image-5-11.png differ diff --git a/Translated_Book/img/image-5-12.png b/Translated_Book/img/image-5-12.png new file mode 100644 index 0000000..6131f64 Binary files /dev/null and b/Translated_Book/img/image-5-12.png differ diff --git a/Translated_Book/img/image-5-13.png b/Translated_Book/img/image-5-13.png new file mode 100644 index 0000000..79ddb27 Binary files /dev/null and b/Translated_Book/img/image-5-13.png differ