{"@context":"https://schema.org","@type":"NewsArticle","generatedAt":"2026-09-23T09:01:38.032Z","headline":"ByteByteGo 图解如何压缩大语言模型而不明显损失智能","description":"ByteByteGo 发布长文，讲解如何在大语言模型不明显变笨的前提下缩小模型体积。文章以 70B 参数模型约 140 GB、消费级显卡仅 24-48 GB 显存的矛盾为切入点。","url":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","mainEntityOfPage":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","datePublished":"2026-09-01T15:30:41.000Z","dateModified":"2026-09-01T15:30:41.000Z","inLanguage":"zh-CN","publisher":{"@type":"NewsMediaOrganization","name":"Aioga","url":"https://www.aioga.com"},"citation":["https://blog.bytebytego.com/p/how-to-shrink-a-language-model-without-295","https://aihot.virxact.com/items/cmtiue30g059pro9yq1w7v7i3"],"canonicalUrl":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","directAnswer":{"@type":"Answer","text":"ByteByteGo 发布文章，讨论如何在不明显损失语言模型智能的前提下缩小模型体积。文章以大模型存储空间与消费级显卡显存之间的差距为切入点，介绍相关压缩思路。","url":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","dateCreated":"2026-09-01T15:30:41.000Z","author":{"@type":"Organization","@id":"https://www.aioga.com/authors/aioga-editorial/#editorial-team","name":"Aioga Editorial Team","url":"https://www.aioga.com/authors/aioga-editorial/"}},"evidence":[{"@type":"CreativeWork","name":"blog.bytebytego.com source article","url":"https://blog.bytebytego.com/p/how-to-shrink-a-language-model-without-295","datePublished":"2026-09-01T15:30:41.000Z","provider":{"@type":"Organization","name":"blog.bytebytego.com","url":"https://blog.bytebytego.com/p/how-to-shrink-a-language-model-without-295"}},{"@type":"CreativeWork","name":"AIHot archive record","url":"https://aihot.virxact.com/items/cmtiue30g059pro9yq1w7v7i3","datePublished":"2026-09-01T15:30:41.000Z","provider":{"@type":"Organization","name":"AIHot","url":"https://aihot.virxact.com/items/cmtiue30g059pro9yq1w7v7i3"}}],"aggregationSource":"ByteByteGo（RSS）","originalPublisher":{"name":"blog.bytebytego.com","url":"https://blog.bytebytego.com/p/how-to-shrink-a-language-model-without-295"},"geoDeepAnswer":null,"article":{"id":"cmtiue30g059pro9yq1w7v7i3","slug":"cmtiue30g059pro9yq1w7v7i3","url":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","title":"ByteByteGo 图解如何压缩大语言模型而不明显损失智能","title_en":"","summary":"ByteByteGo 发布长文，讲解如何在大语言模型不明显变笨的前提下缩小模型体积。文章以 70B 参数模型约 140 GB、消费级显卡仅 24-48 GB 显存的矛盾为切入点。","source":"ByteByteGo（RSS）","sourceUrl":"https://blog.bytebytego.com/p/how-to-shrink-a-language-model-without-295","aiHotUrl":"https://aihot.virxact.com/items/cmtiue30g059pro9yq1w7v7i3","publishedAt":"2026-09-01T15:30:41.000Z","category":"行业动态","score":58,"selected":false,"articleBody":["Datadog’s free guide shows how to connect AI spend, infrastructure, and model performance into a single view, so you can correlate cost increases to the architecture changes that caused them before they show up on your cloud bill.","Break down AI costs by token, model, provider, and team","Get alerted the instant inference volume spikes or API spend exceeds budget","Correlate cost increases directly to architecture changes so root-cause analysis takes minutes","Get the guide ：https://go.bytebytego.com/Datadog_090126","A model with 70 billion parameters can take up to 140 GB of space. A good graphics card has 24 GB. A very good one might have 48 GB.","As you can see, the gap is quite significant. Disk space is pretty cheap, but fast memory is scarce and costly. Moreover, the models have grown roughly 100-fold in a few years, while consumer graphics memory has roughly doubled. It’s not just a matter of tightening things up to make them fit.","The simplest option is to purchase the hardware that is capable of running the model. But it is costly, and doesn’t work well with consumer hardware.","The other option is to shrink the model. But we don’t want to do so at the expense of the model’s intelligence. This is where certain techniques can help us make the model smaller in principle without a dip in the quality of its output.","What makes a language model intelligent?","Three techniques to shrink the model","How to shrink the model by packing fewer details?","How to shrink the model by trimming unused pathways?","How to shrink the model by mimicking behaviour?","Does shrinking damage the model’s intelligence?","Disclaimer: This post is based on publicly shared details from various sources. References at the end. Please comment if you notice any inaccuracies.","Large language models like ChatGPT are quite different from normal software programs. They don’t depend on typical if-else statements to decide what should happen next.","A large language model is essentially a very big pile of numbers called parameters or weights. These weights are the foundation of a language model’s intelligence. For reference, a model with 70 billion parameters means 70 billion numbers or weights. Each weight is normally stored in 16 bits, which is two bytes. Two bytes multiplied by 70 billion comes to 140 GB, which is basically considered the size of the model.","The weights are arranged into matrices. A single weight matrix is a grid, often something like 4096 by 4096. This comes to roughly 16.7 million numbers in a matrix. A 70 billion parameter model has hundreds of these matrices stacked across 80 or so layers.","Of course, weights in themselves aren’t the whole story behind a model’s capability. Multiple components work together to make a model intelligent. For example, the model needs an architecture like the Transformer to route data and perform attention. It also needs a context window that holds the prompt and everything generated during the conversation.","However, the architecture is made up of a few hundred lines of code. The prompt may be a few kilobytes. In contrast, the weights form the bulk of the language model. Without the proper weights, a language model cannot work as intended. The weights perform a bunch of tasks:","They store patterns such as grammar, facts, and reasoning shortcuts.","Once the training is finished, the weights freeze into an immutable network of numerical values.","The weights decide how strongly one simulated neuron influences the next.","Running the model means pushing the input through these weight matrices until the next word comes out of the other end.","One thing to keep in mind over here is that no single weight means anything on its own. If you opened a model file and looked at weight number N, you might see something like 0.0293. On either side of this number may be other numbers such as -0.0117, 0.004, -0.0862. On their own, each of these numbers hardly makes sense. The ability of the model is hidden in the relationships between the various weights. Think of it like a photograph where every pixel comes together to show something recognizable. Even if we modify every pixel’s brightness slightly, we can still make out the things in the picture. This is because the information isn’t sitting in one single place.","Following on from all this information, it is quite easy to figure out that to shrink a model, we’ve to somehow deal with these weights. And this is exactly where the techniques come into the picture. However, a couple of points can help us make better sense of the techniques to shrink a model:","Firstly, not all weights matter equally. Most of the weight values are close to zero and barely impact the final output. However, a small handful are large and produce a bigger impact.","Second, we can only judge a model based on its behaviour, not its internals.","Shipping agents to production is the easy part. Keeping them reliable, governable, and improving over time is where most enterprise AI programs stall.","How do top teams do it? They use an Agentic Operating Model (AOM), a step-by-step framework for aligning people, process, and technology so enterprise agents improve as they scale.","In LangChain’s latest guide, you’ll learn:","Why AI agents don’t break like traditional software","The engineering stack that covers the entire agent lifecycle","Shifting from “build and deploy” to “operate and continuously improve”","Learn more ：https://go.bytebytego.com/LangChain_090126","The techniques to shrink a model revolve around using fewer bits to store a weight or using fewer weights. There are three main techniques:","Storing Each Weight in Less Detail (Quantization): In this approach, we keep all the weights, but describe each one in a less precise manner. For example, two bytes become half a byte.","Removing Irrelevant Weights (Pruning): This approach involves finding the weights that contribute nothing and deleting them.","Building a Smaller Model to Mimic the Larger One (Knowledge Distillation): In this approach, we don’t touch the original model, but train a new, smaller model to behave in a similar way.","Going back to our photograph example, we can think of quantization as taking a picture with a cheaper camera that has a slightly lower resolution. Pruning is more like cutting away the blank edges of the photograph that might not be adding any value to the picture. Distillation can be thought of as paying a skilled painter to reproduce the picture at a quarter of the size.","The great part about all these techniques is that they can be stacked. For example, a model can be distilled by the lab that made it. It can be pruned by a research team. Lastly, it can be quantized by the user before it is loaded on a specific machine. In other words, stacking can make it possible to run a high-end large language model on normal consumer hardware.","Let us now look at each of these techniques in more detail.","The first technique to shrink a model is to pack fewer details for every weight. This technique is known as quantization.","Let’s say a particular weight might be stored as 0.02934517. This takes a lot of space, but in a 70 billion parameter model, it just happens to be one single weight. Whether it is stored as 0.02934517 or 0.029 makes almost no difference to the model’s output. In other words, a lot of storage is spent on precision that might not even be important.","Quantization is a technique that takes away this precision.","The first bit of quantization happens even before the model is shipped. During training, model weights are normally stored as 32-bit (4 bytes) floating-point numbers. This is also known as the FP32 format. Since training involves making millions of tiny adjustments to each weight, high precision is needed. But when the model is distributed, the precision is usually brought down to 16-bit float format, which is also known as FP16 or BF16.","However, we can bring the precision down even further. To understand how, we need to first be clear about how a float value is actually built.","A floating-point number splits the bits into three parts: a sign, an exponent, and a mantissa. FP32 gives 1 bit to the sign, 8 bits to the exponent, and 23 bits to the mantissa. BF16, on the other hand, keeps all exponent bits and cuts the mantissa down to seven. This is basically the same range as FP32, but with less detail. For clarity, BF16 is slightly different from FP16, which gives the exponent only 5 bits and keeps more bits for the mantissa. BF16 has largely replaced FP16 in practice.","An integer has an even greater difference. An 8-bit integer is a whole number from -128 to 127. A 4-bit integer is a whole number from -8 to 7. There is no exponent and no scale. In other words, converting a float into an integer not only causes a loss of precision, but also removes each weight’s scale.","Let us now look at the complete process of quantization:","In the first step, we find out the minimum and maximum values of a data set and divide the total span into a fixed number of steps.","To be clear, “the data set” is not the entire model. It’s just a small group of neighbouring weights. We can call it a block, and it should ideally be pretty small.","For example, consider these eight weights: 0.021, -0.017, 0.004, -0.048, 0.011, 0.033, -0.006, 0.070. They span from -0.048 to 0.070. The largest value in either direction is 0.070. With 4 bits as our target precision, we can write whole numbers from -7 to 7. In other words, seven steps in each direction. Therefore, one step can be calculated as 0.070/7, which comes to 0.010.","Instead of keeping a long decimal, each original number is rounded to the closest available step. To do so, we divide each weight by the step size and round it to the nearest whole number.","The table below shows the new weights:","As you can see in the table, the right-hand column values are the ones that go into the model weights file eventually. Every entry here is a whole number between -7 and 7.","The original weights were floating-point numbers. Since a float carries its own scale, rounding off to engineers removes the precision as well as the scale of every weight. This scale has to be stored somewhere.","We need to keep track of the scale factor so that the compressed numbers can roughly reconstruct the original values when the model needs to read them.","In our example, the scale factor is the step size (0.010). It is stored once for the entire block. To recover a weight, we can multiply the stored integer by the scale factor."],"articleImages":[],"mediaStatus":"none","articleBodyZh":["Datadog 的免费指南展示了如何将 AI 支出、基础设施和模型性能连接到一个视图中，这样您就可以在云账单上显示之前，将成本增加与导致这些变化的架构更改关联起来。","按令牌、模型、供应商和团队分解 AI 成本","一旦推理量激增或 API 支出超过预算，即刻收到提醒","将成本增加直接与架构更改关联，这样根本原因分析只需几分钟","获取指南：https://go.bytebytego.com/Datadog_090126","一个拥有 700 亿参数的模型可能占用多达 140 GB 的空间。一块好的显卡有 24 GB，非常好的可能有 48 GB。","如您所见，差距相当显著。硬盘空间相对便宜，但快速内存稀缺且昂贵。此外，模型在几年内大约增长了 100 倍，而消费者显存大约翻了一番。这不仅仅是压缩以适应的问题。","最简单的选择是购买能够运行该模型的硬件。但这成本高昂，而且不适合消费者硬件。","另一种选择是缩小模型。但我们不希望以牺牲模型智能为代价。在这里，某些技术可以帮助我们在不降低输出质量的前提下使模型更小。","是什么让语言模型变得智能？","缩小模型的三种技术","如何通过压缩细节来缩小模型？","如何通过剪掉未使用的路径来缩小模型？","如何通过模拟行为来缩小模型？","缩小会损害模型的智能吗？","免责声明：本文基于来自各种来源的公开信息。参考文献在文末。如果您发现任何不准确之处，请评论。","像 ChatGPT 这样的大型语言模型与普通软件程序有很大不同。它们不依赖典型的 if-else 语句来决定下一步操作。","大型语言模型本质上是由称为参数或权重的数字组成的一大堆数字。这些权重是语言模型智能的基础。举例来说，一个拥有700亿参数的模型意味着有700亿个数字或权重。每个权重通常以16位存储，也就是两个字节。两个字节乘以700亿等于140GB，这基本上被认为就是模型的大小。","这些权重被排列成矩阵。单个权重矩阵是一个网格，通常是类似4096乘以4096的大小。这样一个矩阵大约有1670万个数字。一个拥有700亿参数的模型在大约80层中堆叠了数百个这样的矩阵。","当然，权重本身并不是模型能力的全部。多个组件协同工作使模型具有智能。例如，模型需要像Transformer这样的架构来引导数据并执行注意力机制。它还需要一个上下文窗口来保存提示以及对话过程中生成的所有内容。","然而，架构由几百行代码组成。提示可能只有几千字节。相比之下，权重构成了语言模型的主要部分。没有适当的权重，语言模型无法按预期工作。权重执行许多任务：","它们存储模式，例如语法、事实和推理捷径。","训练完成后，权重会冻结为不可变的数字值网络。","权重决定了一个模拟神经元对下一个神经元的影响强度。","运行模型意味着将输入通过这些权重矩阵传递，直到下一个词从另一端输出。","在这里需要记住的一点是，没有单个权重本身是有意义的。如果你打开一个模型文件并查看编号为 N 的权重，你可能会看到类似 0.0293 这样的数字。在这个数字的两侧可能还有其他数字，例如 -0.0117、0.004、-0.0862。单独看，每一个数字几乎没有意义。模型的能力隐藏在各个权重之间的关系中。可以把它想象成一张照片，每个像素共同呈现出可识别的图像。即使我们稍微调整每个像素的亮度，我们仍然可以辨认出图片中的内容。这是因为信息并不集中在单一的位置。","根据上述信息，很容易理解，要缩小一个模型，我们必须以某种方式处理这些权重。而这正是各种技术派上用场的地方。不过，有几点可以帮助我们更好地理解缩小模型的技术：","首先，并非所有权重都同等重要。大多数权重值接近零，对最终输出几乎没有影响。然而，有少数权重较大，会产生较大的影响。","其次，我们只能根据模型的行为来判断模型，而不能凭其内部结构。","将代理投入生产是容易的部分。保持它们的可靠性、可管理性，并随着时间改进，才是大多数企业 AI 项目停滞的地方。","顶尖团队是如何做到的？他们使用了代理式操作模型（Agentic Operating Model，简称 AOM），这是一个逐步框架，用于在企业代理扩展时对人、流程和技术进行对齐，从而提升代理性能。","在浪链最新指南中，您将了解到：","为何 AI 代理不像传统软件那样容易崩溃","覆盖整个代理生命周期的工程技术栈","从“构建和部署”转向“运营和持续改进”","了解更多：https://go.bytebytego.com/LangChain_090126","缩小模型的技术主要围绕使用更少的位数来存储权重，或者使用更少的权重值。主要有三种技术：","以较低精度存储每个权重（量化）：在这种方法中，我们保留所有权重，但以不那么精确的方式描述每一个。例如，两字节变为半字节。","移除无关权重（剪枝）：这种方法涉及找到没有任何贡献的权重并将其删除。","建立较小的模型以模仿较大的模型（知识蒸馏）：在这种方法中，我们不接触原始模型，而是训练一个新的、更小的模型，使其表现得与原模型相似。","回到我们的照片例子，我们可以将量化看作是用一台分辨率稍低的廉价相机拍照。剪枝更像是剪掉照片中可能没有任何价值的空白边缘。蒸馏可以被看作是请一位熟练的画家将照片缩小到原图的四分之一来重新绘制。","这些技术的一个优点是它们可以叠加。例如，一个模型可以由制造它的实验室进行蒸馏。它可以由研究团队进行剪枝。最后，在加载到特定机器之前，用户可以对其进行量化。换句话说，叠加这些技术可以使高端大语言模型在普通消费硬件上运行成为可能。","现在让我们更详细地看看这些技术中的每一种。","缩小模型的第一种技术是减少每个权重存储的细节。这种技术称为量化。","假设某个权重可能被存储为0.02934517。这样占用很多空间，但在一个拥有700亿参数的模型中，这只不过是单个权重而已。无论它存储为0.02934517还是0.029，对模型的输出几乎没有差别。换句话说，很多存储空间都用在了可能并不重要的精度上。","量化是一种降低这种精度的技术。","量化的第一个步骤甚至发生在模型发布之前。在训练过程中，模型权重通常以32位（4字节）浮点数存储。这也被称为FP32格式。由于训练涉及对每个权重进行数百万次微小调整，因此需要高精度。但当模型被分发时，精度通常会降低到16位浮点格式，也称为FP16或BF16。","然而，我们可以将精度降低得更低。要理解如何降低，我们首先需要清楚地了解浮点值是如何构建的。","浮点数将位划分为三个部分：符号、指数和尾数。FP32 为符号分配 1 位，为指数分配 8 位，为尾数分配 23 位。而 BF16 则保留所有指数位，并将尾数减至 7 位。这在基本范围上与 FP32 相同，但细节较少。为了清楚起见，BF16 与 FP16 略有不同，后者指数只有 5 位，但为尾数保留更多位。实际使用中，BF16 已大致取代 FP16。","整数的差异更大。8 位整数是从 -128 到 127 的整数。4 位整数是从 -8 到 7 的整数。没有指数，也没有比例。换句话说，将浮点数转换为整数不仅会导致精度损失，还会去掉每个权重的尺度。","现在让我们来看量化的完整过程：","第一步，我们找出数据集的最小值和最大值，并将总跨度划分为固定步数。","需要明确的是，“数据集”不是整个模型。它只是相邻权重的小组。我们可以称之为一个块，理想情况下应该相当小。","例如，考虑这八个权重：0.021、-0.017、0.004、-0.048、0.011、0.033、-0.006、0.070。它们的跨度从 -0.048 到 0.070。无论哪个方向的最大值都是 0.070。以 4 位作为目标精度，我们可以表示从 -7 到 7 的整数。换句话说，每个方向有七个步长。因此，一个步长可以计算为 0.070/7，即 0.010。","不是保留长小数，而是将每个原始数字四舍五入到最接近的可用步长。为此，我们将每个权重除以步长并四舍五入到最接近的整数。","下表显示了新的权重：","如表中所示，右侧列的值最终会进入模型权重文件。此处每条记录都是介于 -7 到 7 之间的整数。","原始权重是浮点数。由于浮点数自带尺度，将其四舍五入会去掉每个权重的精度和尺度。这个尺度必须存储在某处。","我们需要跟踪缩放因子，以便在模型需要读取它们时，压缩后的数字可以大致重建原始值。","在我们的示例中，缩放因子是步长（0.010）。它为整个块存储一次。要恢复权重，我们可以将存储的整数乘以缩放因子。"],"translationStatus":"translated","bodyOrigin":"source-page","editorial":{"summary":"ByteByteGo 发布文章，讨论如何在不明显损失语言模型智能的前提下缩小模型体积。文章以大模型存储空间与消费级显卡显存之间的差距为切入点，介绍相关压缩思路。","background":"文中称，70B 参数模型最多可能占用约 140 GB 空间，而较好的显卡通常有 24 GB 显存，更高规格显卡可能有 48 GB。文章还指出，快速显存稀缺且成本较高。","viewpoint":"Aioga 判断：材料明确呈现的是模型规模增长与消费级显存容量之间的容量矛盾，但正文摘录未展开具体压缩方法及其效果，因此不宜据此判断某种方案已经解决部署问题。","implications":"可能影响：模型部署需要同时考虑模型体积、快速显存与硬件成本；缩小模型可能有助于适配有限显存，但现有材料不足以证明具体方法对性能、成本或部署范围的实际影响。","nextStep":"后续观察：需进一步核对原文对压缩方法、智能损失、运行条件和硬件适配范围的具体说明，再评估其对消费级设备部署的参考价值。","evidenceRefs":["title","summary","articleBody","source"],"status":"published","aiGenerated":true,"autoApproved":true,"generatedBy":"aioga-editorial:gpt-5.6-sol","reviewedBy":"aioga-editorial-review:gpt-5.6-sol","generatedAt":"2026-09-01T16:08:39.651Z","sourceHash":"75490e986e0f87cd","review":{"approved":true,"groundedness":96,"clarity":93,"duplicationRisk":12,"blockingIssues":[],"notes":["“较好的显卡通常有 24 GB 显存”中的“通常”较原文“A good graphics card has 24 GB”略有概括，但不构成实质性事实偏差。","候选内容明确区分了来源事实、编辑判断和可能影响，并对正文摘录未提供具体压缩方法及效果保持了审慎表述。"]},"validation":{"passed":true,"mode":"ai-auto","revisions":0,"checks":["schema","length","source-attribution","editorial-labels","inference-boundary","low-source-overlap","no-html","independent-ai-review"]}},"tags":["行业动态","ByteByteGo（RSS）"],"translations":{"zh-CN":{"title":"ByteByteGo 图解如何压缩大语言模型而不明显损失智能","summary":"ByteByteGo 发布长文，讲解如何在大语言模型不明显变笨的前提下缩小模型体积。文章以 70B 参数模型约 140 GB、消费级显卡仅 24-48 GB 显存的矛盾为切入点。","category":"行业动态","source":"blog.bytebytego.com","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo 图解如何压缩大语言模型而不明显损失智能 - Aioga AI资讯","description":"ByteByteGo 发布长文，讲解如何在大语言模型不明显变笨的前提下缩小模型体积。文章以 70B 参数模型约 140 GB、消费级显卡仅 24-48 GB 显存的矛盾为切入点。","url":"https://www.aioga.com/news/cmtiue30g059pro9yq1w7v7i3/","articleBody":["Datadog 的免费指南展示了如何将 AI 支出、基础设施和模型性能连接到一个视图中，这样您就可以在云账单上显示之前，将成本增加与导致这些变化的架构更改关联起来。","按令牌、模型、供应商和团队分解 AI 成本","一旦推理量激增或 API 支出超过预算，即刻收到提醒","将成本增加直接与架构更改关联，这样根本原因分析只需几分钟","获取指南：https://go.bytebytego.com/Datadog_090126","一个拥有 700 亿参数的模型可能占用多达 140 GB 的空间。一块好的显卡有 24 GB，非常好的可能有 48 GB。","如您所见，差距相当显著。硬盘空间相对便宜，但快速内存稀缺且昂贵。此外，模型在几年内大约增长了 100 倍，而消费者显存大约翻了一番。这不仅仅是压缩以适应的问题。","最简单的选择是购买能够运行该模型的硬件。但这成本高昂，而且不适合消费者硬件。","另一种选择是缩小模型。但我们不希望以牺牲模型智能为代价。在这里，某些技术可以帮助我们在不降低输出质量的前提下使模型更小。","是什么让语言模型变得智能？","缩小模型的三种技术","如何通过压缩细节来缩小模型？","如何通过剪掉未使用的路径来缩小模型？","如何通过模拟行为来缩小模型？","缩小会损害模型的智能吗？","免责声明：本文基于来自各种来源的公开信息。参考文献在文末。如果您发现任何不准确之处，请评论。","像 ChatGPT 这样的大型语言模型与普通软件程序有很大不同。它们不依赖典型的 if-else 语句来决定下一步操作。","大型语言模型本质上是由称为参数或权重的数字组成的一大堆数字。这些权重是语言模型智能的基础。举例来说，一个拥有700亿参数的模型意味着有700亿个数字或权重。每个权重通常以16位存储，也就是两个字节。两个字节乘以700亿等于140GB，这基本上被认为就是模型的大小。","这些权重被排列成矩阵。单个权重矩阵是一个网格，通常是类似4096乘以4096的大小。这样一个矩阵大约有1670万个数字。一个拥有700亿参数的模型在大约80层中堆叠了数百个这样的矩阵。","当然，权重本身并不是模型能力的全部。多个组件协同工作使模型具有智能。例如，模型需要像Transformer这样的架构来引导数据并执行注意力机制。它还需要一个上下文窗口来保存提示以及对话过程中生成的所有内容。","然而，架构由几百行代码组成。提示可能只有几千字节。相比之下，权重构成了语言模型的主要部分。没有适当的权重，语言模型无法按预期工作。权重执行许多任务：","它们存储模式，例如语法、事实和推理捷径。","训练完成后，权重会冻结为不可变的数字值网络。","权重决定了一个模拟神经元对下一个神经元的影响强度。","运行模型意味着将输入通过这些权重矩阵传递，直到下一个词从另一端输出。","在这里需要记住的一点是，没有单个权重本身是有意义的。如果你打开一个模型文件并查看编号为 N 的权重，你可能会看到类似 0.0293 这样的数字。在这个数字的两侧可能还有其他数字，例如 -0.0117、0.004、-0.0862。单独看，每一个数字几乎没有意义。模型的能力隐藏在各个权重之间的关系中。可以把它想象成一张照片，每个像素共同呈现出可识别的图像。即使我们稍微调整每个像素的亮度，我们仍然可以辨认出图片中的内容。这是因为信息并不集中在单一的位置。","根据上述信息，很容易理解，要缩小一个模型，我们必须以某种方式处理这些权重。而这正是各种技术派上用场的地方。不过，有几点可以帮助我们更好地理解缩小模型的技术：","首先，并非所有权重都同等重要。大多数权重值接近零，对最终输出几乎没有影响。然而，有少数权重较大，会产生较大的影响。","其次，我们只能根据模型的行为来判断模型，而不能凭其内部结构。","将代理投入生产是容易的部分。保持它们的可靠性、可管理性，并随着时间改进，才是大多数企业 AI 项目停滞的地方。","顶尖团队是如何做到的？他们使用了代理式操作模型（Agentic Operating Model，简称 AOM），这是一个逐步框架，用于在企业代理扩展时对人、流程和技术进行对齐，从而提升代理性能。","在浪链最新指南中，您将了解到：","为何 AI 代理不像传统软件那样容易崩溃","覆盖整个代理生命周期的工程技术栈","从“构建和部署”转向“运营和持续改进”","了解更多：https://go.bytebytego.com/LangChain_090126","缩小模型的技术主要围绕使用更少的位数来存储权重，或者使用更少的权重值。主要有三种技术：","以较低精度存储每个权重（量化）：在这种方法中，我们保留所有权重，但以不那么精确的方式描述每一个。例如，两字节变为半字节。","移除无关权重（剪枝）：这种方法涉及找到没有任何贡献的权重并将其删除。","建立较小的模型以模仿较大的模型（知识蒸馏）：在这种方法中，我们不接触原始模型，而是训练一个新的、更小的模型，使其表现得与原模型相似。","回到我们的照片例子，我们可以将量化看作是用一台分辨率稍低的廉价相机拍照。剪枝更像是剪掉照片中可能没有任何价值的空白边缘。蒸馏可以被看作是请一位熟练的画家将照片缩小到原图的四分之一来重新绘制。","这些技术的一个优点是它们可以叠加。例如，一个模型可以由制造它的实验室进行蒸馏。它可以由研究团队进行剪枝。最后，在加载到特定机器之前，用户可以对其进行量化。换句话说，叠加这些技术可以使高端大语言模型在普通消费硬件上运行成为可能。","现在让我们更详细地看看这些技术中的每一种。","缩小模型的第一种技术是减少每个权重存储的细节。这种技术称为量化。","假设某个权重可能被存储为0.02934517。这样占用很多空间，但在一个拥有700亿参数的模型中，这只不过是单个权重而已。无论它存储为0.02934517还是0.029，对模型的输出几乎没有差别。换句话说，很多存储空间都用在了可能并不重要的精度上。","量化是一种降低这种精度的技术。","量化的第一个步骤甚至发生在模型发布之前。在训练过程中，模型权重通常以32位（4字节）浮点数存储。这也被称为FP32格式。由于训练涉及对每个权重进行数百万次微小调整，因此需要高精度。但当模型被分发时，精度通常会降低到16位浮点格式，也称为FP16或BF16。","然而，我们可以将精度降低得更低。要理解如何降低，我们首先需要清楚地了解浮点值是如何构建的。","浮点数将位划分为三个部分：符号、指数和尾数。FP32 为符号分配 1 位，为指数分配 8 位，为尾数分配 23 位。而 BF16 则保留所有指数位，并将尾数减至 7 位。这在基本范围上与 FP32 相同，但细节较少。为了清楚起见，BF16 与 FP16 略有不同，后者指数只有 5 位，但为尾数保留更多位。实际使用中，BF16 已大致取代 FP16。","整数的差异更大。8 位整数是从 -128 到 127 的整数。4 位整数是从 -8 到 7 的整数。没有指数，也没有比例。换句话说，将浮点数转换为整数不仅会导致精度损失，还会去掉每个权重的尺度。","现在让我们来看量化的完整过程：","第一步，我们找出数据集的最小值和最大值，并将总跨度划分为固定步数。","需要明确的是，“数据集”不是整个模型。它只是相邻权重的小组。我们可以称之为一个块，理想情况下应该相当小。","例如，考虑这八个权重：0.021、-0.017、0.004、-0.048、0.011、0.033、-0.006、0.070。它们的跨度从 -0.048 到 0.070。无论哪个方向的最大值都是 0.070。以 4 位作为目标精度，我们可以表示从 -7 到 7 的整数。换句话说，每个方向有七个步长。因此，一个步长可以计算为 0.070/7，即 0.010。","不是保留长小数，而是将每个原始数字四舍五入到最接近的可用步长。为此，我们将每个权重除以步长并四舍五入到最接近的整数。","下表显示了新的权重：","如表中所示，右侧列的值最终会进入模型权重文件。此处每条记录都是介于 -7 到 7 之间的整数。","原始权重是浮点数。由于浮点数自带尺度，将其四舍五入会去掉每个权重的精度和尺度。这个尺度必须存储在某处。","我们需要跟踪缩放因子，以便在模型需要读取它们时，压缩后的数字可以大致重建原始值。","在我们的示例中，缩放因子是步长（0.010）。它为整个块存储一次。要恢复权重，我们可以将存储的整数乘以缩放因子。"]},"en":{"title":"ByteByteGo illustrated how to compress large language models without significant loss of intelligence","summary":"ByteByteGo published a lengthy article explaining how to reduce the size of large language models without noticeably becoming clumsy. The article starts by focusing on the contradiction of a 70B model with about 140 GB of VRAM, while a consumer-grade graphics card only has 24-48 GB of VRAM.","category":"Industry","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo illustrated how to compress large language models without significant loss of intelligence - Aioga AI News","description":"ByteByteGo published a lengthy article explaining how to reduce the size of large language models without noticeably becoming clumsy. The article starts by focusing on the contradi...","url":"https://www.aioga.com/en/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:36.907Z"},"ja":{"title":"ByteByteGoは、知能を大きく損なうことなく大規模言語モデルを圧縮する方法を示しました","summary":"ByteByteGoは、大規模な言語モデルのサイズを目立たずに小さくする方法を説明する長い記事を掲載しました。 この記事は、約140GBのVRAMを持つ70Bモデルと、消費者向けのグラフィックカードは24〜48GBのVRAMしか持たないという矛盾に焦点を当てています。","category":"業界動向","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGoは、知能を大きく損なうことなく大規模言語モデルを圧縮する方法を示しました - Aioga AIニュース","description":"ByteByteGoは、大規模な言語モデルのサイズを目立たずに小さくする方法を説明する長い記事を掲載しました。 この記事は、約140GBのVRAMを持つ70Bモデルと、消費者向けのグラフィックカードは24〜48GBのVRAMしか持たないという矛盾に焦点を当てています。","url":"https://www.aioga.com/ja/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:37.201Z"},"ko":{"title":"ByteByteGo는 지능 손실이 크게 없는 대규모 언어 모델을 압축하는 방법을 보여주었습니다","summary":"ByteByteGo는 대형 언어 모델의 크기를 눈에 띄게 어색해지지 않으면서 줄이는 방법을 설명하는 긴 기사를 게재했습니다. 기사는 약 140GB VRAM을 가진 70B 모델과 소비자용 그래픽 카드는 24-48GB VRAM만 가진 모순에 초점을 맞추는 것으로 시작합니다.","category":"업계 동향","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo는 지능 손실이 크게 없는 대규모 언어 모델을 압축하는 방법을 보여주었습니다 - Aioga AI 뉴스","description":"ByteByteGo는 대형 언어 모델의 크기를 눈에 띄게 어색해지지 않으면서 줄이는 방법을 설명하는 긴 기사를 게재했습니다. 기사는 약 140GB VRAM을 가진 70B 모델과 소비자용 그래픽 카드는 24-48GB VRAM만 가진 모순에 초점을 맞추는 것으로 시작합니다.","url":"https://www.aioga.com/ko/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:46.600Z"},"es":{"title":"ByteByteGo ilustró cómo comprimir grandes modelos de lenguaje sin una pérdida significativa de inteligencia","summary":"ByteByteGo publicó un extenso artículo explicando cómo reducir el tamaño de grandes modelos de lenguaje sin volverse visiblemente torpe. El artículo comienza centrándose en la contradicción de un modelo 70B con unos 140 GB de VRAM, mientras que una tarjeta gráfica de consumo solo tiene entre 24 y 48 GB de VRAM.","category":"Industria","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo ilustró cómo comprimir grandes modelos de lenguaje sin una pérdida significativa de inteligencia - Aioga Noticias de IA","description":"ByteByteGo publicó un extenso artículo explicando cómo reducir el tamaño de grandes modelos de lenguaje sin volverse visiblemente torpe. El artículo comienza centrándose en la cont...","url":"https://www.aioga.com/es/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:47.000Z"},"fr":{"title":"ByteByteGo a illustré comment compresser de grands modèles de langage sans perte significative d’intelligence","summary":"ByteByteGo a publié un long article expliquant comment réduire la taille des grands modèles de langage sans devenir visiblement maladroit. L’article commence par se concentrer sur la contradiction d’un modèle 70B avec environ 140 Go de VRAM, alors qu’une carte graphique grand public ne dispose que de 24 à 48 Go de VRAM.","category":"Industrie","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo a illustré comment compresser de grands modèles de langage sans perte significative d’intelligence - Aioga Actualités IA","description":"ByteByteGo a publié un long article expliquant comment réduire la taille des grands modèles de langage sans devenir visiblement maladroit. L’article commence par se concentrer sur...","url":"https://www.aioga.com/fr/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:56.460Z"},"de":{"title":"ByteByteGo zeigte, wie man große Sprachmodelle ohne nennenswerten Intelligenzverlust komprimieren kann","summary":"ByteByteGo veröffentlichte einen ausführlichen Artikel, der erklärt, wie man die Größe großer Sprachmodelle reduzieren kann, ohne spürbar unbeholfen zu werden. Der Artikel beginnt mit dem Fokus auf den Widerspruch eines 70B-Modells mit etwa 140 GB VRAM, während eine Verbrauchergrafikkarte nur 24–48 GB VRAM hat.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo zeigte, wie man große Sprachmodelle ohne nennenswerten Intelligenzverlust komprimieren kann - Aioga KI-News","description":"ByteByteGo veröffentlichte einen ausführlichen Artikel, der erklärt, wie man die Größe großer Sprachmodelle reduzieren kann, ohne spürbar unbeholfen zu werden. Der Artikel beginnt...","url":"https://www.aioga.com/de/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:02:56.192Z"},"pt-BR":{"title":"O ByteByteGo ilustrou como comprimir grandes modelos de linguagem sem perda significativa de inteligência","summary":"A ByteByteGo publicou um artigo extenso explicando como reduzir o tamanho de grandes modelos de linguagem sem se tornar visivelmente desajeitado. O artigo começa focando na contradição de um modelo 70B com cerca de 140 GB de VRAM, enquanto uma placa de vídeo de consumo tem apenas 24-48 GB de VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"O ByteByteGo ilustrou como comprimir grandes modelos de linguagem sem perda significativa de inteligência - Aioga Notícias de IA","description":"A ByteByteGo publicou um artigo extenso explicando como reduzir o tamanho de grandes modelos de linguagem sem se tornar visivelmente desajeitado. O artigo começa focando na contrad...","url":"https://www.aioga.com/pt-BR/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:04.986Z"},"ru":{"title":"ByteByteGo показал, как сжимать большие языковые модели без значительной потери интеллекта","summary":"ByteByteGo опубликовал обширную статью, объясняющую, как уменьшить размер больших языковых моделей, не становясь заметно неуклюжим. Статья начинается с акцента на противоречии модели 70B с примерно 140 ГБ видеопамяти, тогда как потребительская видеокарта имеет только 24-48 ГБ VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo показал, как сжимать большие языковые модели без значительной потери интеллекта - Aioga Новости ИИ","description":"ByteByteGo опубликовал обширную статью, объясняющую, как уменьшить размер больших языковых моделей, не становясь заметно неуклюжим. Статья начинается с акцента на противоречии моде...","url":"https://www.aioga.com/ru/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:06.103Z"},"ar":{"title":"أوضح بايت بايت جو كيفية ضغط نماذج اللغة الكبيرة دون فقدان كبير للذكاء","summary":"نشرت ByteByteGo مقالا مطولا يشرح كيفية تقليل حجم نماذج اللغة الكبيرة دون أن تصبح غير متقنة بشكل ملحوظ. يبدأ المقال بالتركيز على التناقض بين نموذج 70 بايت مع حوالي 140 جيجابايت من ذاكرة VRAM، بينما بطاقة الرسومات الاستهلاكية تحتوي فقط على 24-48 جيجابايت من ذاكرة VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"أوضح بايت بايت جو كيفية ضغط نماذج اللغة الكبيرة دون فقدان كبير للذكاء - Aioga أخبار الذكاء الاصطناعي","description":"نشرت ByteByteGo مقالا مطولا يشرح كيفية تقليل حجم نماذج اللغة الكبيرة دون أن تصبح غير متقنة بشكل ملحوظ. يبدأ المقال بالتركيز على التناقض بين نموذج 70 بايت مع حوالي 140 جيجابايت من ذ...","url":"https://www.aioga.com/ar/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:14.425Z"},"hi":{"title":"ByteByteGo ने बताया कि बुद्धि के महत्वपूर्ण नुकसान के बिना बड़े भाषा मॉडल को कैसे संपीड़ित किया जाए","summary":"बाइटबाइटगो ने एक लंबा लेख प्रकाशित किया जिसमें बताया गया है कि बड़े भाषा मॉडल के आकार को बिना अनाड़ी बने कैसे कम किया जाए। लेख लगभग 70 जीबी वीआरएएम के साथ 140बी मॉडल के विरोधाभास पर ध्यान केंद्रित करके शुरू होता है, जबकि उपभोक्ता-ग्रेड ग्राफिक्स कार्ड में केवल 24-48 जीबी वीआरएएम होता है।","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo ने बताया कि बुद्धि के महत्वपूर्ण नुकसान के बिना बड़े भाषा मॉडल को कैसे संपीड़ित किया जाए - Aioga AI समाचार","description":"बाइटबाइटगो ने एक लंबा लेख प्रकाशित किया जिसमें बताया गया है कि बड़े भाषा मॉडल के आकार को बिना अनाड़ी बने कैसे कम किया जाए। लेख लगभग 70 जीबी वीआरएएम के साथ 140बी मॉडल के विरोधाभास प...","url":"https://www.aioga.com/hi/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:15.422Z"},"it":{"title":"ByteByteGo ha illustrato come comprimere grandi modelli linguistici senza una perdita significativa di intelligenza","summary":"ByteByteGo ha pubblicato un lungo articolo che spiega come ridurre la dimensione dei grandi modelli linguistici senza diventare evidentemente goffi. L'articolo inizia concentrandosi sulla contraddizione di un modello 70B con circa 140 GB di VRAM, mentre una scheda grafica consumer ha solo 24-48 GB di VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo ha illustrato come comprimere grandi modelli linguistici senza una perdita significativa di intelligenza - Aioga Notizie IA","description":"ByteByteGo ha pubblicato un lungo articolo che spiega come ridurre la dimensione dei grandi modelli linguistici senza diventare evidentemente goffi. L'articolo inizia concentrandos...","url":"https://www.aioga.com/it/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:24.606Z"},"nl":{"title":"ByteByteGo illustreerde hoe grote taalmodellen gecomprimeerd kunnen worden zonder significant verlies van intelligentie","summary":"ByteByteGo publiceerde een uitgebreid artikel waarin werd uitgelegd hoe je de grootte van grote taalmodellen kunt verkleinen zonder merkbaar onhandig te worden. Het artikel begint met de focus op de tegenstrijdigheid van een 70B-model met ongeveer 140 GB VRAM, terwijl een consumenten-grafische kaart slechts 24-48 GB VRAM heeft.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo illustreerde hoe grote taalmodellen gecomprimeerd kunnen worden zonder significant verlies van intelligentie - Aioga AI-nieuws","description":"ByteByteGo publiceerde een uitgebreid artikel waarin werd uitgelegd hoe je de grootte van grote taalmodellen kunt verkleinen zonder merkbaar onhandig te worden. Het artikel begint...","url":"https://www.aioga.com/nl/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:24.484Z"},"tr":{"title":"ByteByteGo, büyük dil modellerinin önemli bir zeka kaybı olmadan nasıl sıkıştırılacağını gösterdi","summary":"ByteByteGo, büyük dil modellerinin boyutunu belirgin şekilde sakar olmadan nasıl küçültüleceğini açıklayan uzun bir makale yayımladı. Makale, yaklaşık 140 GB VRAM'a sahip 70B modelinin bir kişi kartı ile sadece 24-48 GB VRAM içermesinin çelişkisine odaklanarak başlıyor.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo, büyük dil modellerinin önemli bir zeka kaybı olmadan nasıl sıkıştırılacağını gösterdi - Aioga AI Haberleri","description":"ByteByteGo, büyük dil modellerinin boyutunu belirgin şekilde sakar olmadan nasıl küçültüleceğini açıklayan uzun bir makale yayımladı. Makale, yaklaşık 140 GB VRAM'a sahip 70B model...","url":"https://www.aioga.com/tr/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:32.720Z"},"vi":{"title":"ByteByteGo minh họa cách nén các mô hình ngôn ngữ lớn mà không làm giảm đáng kể trí tuệ","summary":"ByteByteGo đã xuất bản một bài viết dài giải thích cách giảm kích thước các mô hình ngôn ngữ lớn mà không trở nên vụng về rõ rệt. Bài viết bắt đầu bằng việc tập trung vào mâu thuẫn giữa một mẫu 70B với khoảng 140 GB VRAM, trong khi một card đồ họa tiêu dùng chỉ có 24-48 GB VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo minh họa cách nén các mô hình ngôn ngữ lớn mà không làm giảm đáng kể trí tuệ - Tin tức AI Aioga","description":"ByteByteGo đã xuất bản một bài viết dài giải thích cách giảm kích thước các mô hình ngôn ngữ lớn mà không trở nên vụng về rõ rệt. Bài viết bắt đầu bằng việc tập trung vào mâu thuẫn...","url":"https://www.aioga.com/vi/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:33.828Z"},"id":{"title":"ByteByteGo mengilustrasikan cara mengompres model bahasa besar tanpa kehilangan kecerdasan yang signifikan","summary":"ByteByteGo menerbitkan artikel panjang yang menjelaskan cara mengurangi ukuran model bahasa besar tanpa menjadi canggung secara mencolok. Artikel ini dimulai dengan fokus pada kontradiksi model 70B dengan sekitar 140 GB VRAM, sementara kartu grafis konsumen hanya memiliki 24-48 GB VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo mengilustrasikan cara mengompres model bahasa besar tanpa kehilangan kecerdasan yang signifikan - Berita AI Aioga","description":"ByteByteGo menerbitkan artikel panjang yang menjelaskan cara mengurangi ukuran model bahasa besar tanpa menjadi canggung secara mencolok. Artikel ini dimulai dengan fokus pada kont...","url":"https://www.aioga.com/id/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:43.613Z"},"th":{"title":"ByteByteGo แสดงให้เห็นวิธีการบีบอัดโมเดลภาษาขนาดใหญ่โดยไม่สูญเสียความฉลาดอย่างมีนัยสําคัญ","summary":"ByteByteGo ได้ตีพิมพ์บทความยาวอธิบายวิธีลดขนาดของโมเดลภาษาขนาดใหญ่โดยไม่ให้ดูเก้งก้างอย่างเห็นได้ชัด บทความนี้เริ่มต้นด้วยการเน้นความขัดแย้งระหว่างรุ่น 70B ที่มี VRAM ประมาณ 140 GB ในขณะที่การ์ดกราฟิกระดับผู้บริโภคมี VRAM เพียง 24-48 GB เท่านั้น","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo แสดงให้เห็นวิธีการบีบอัดโมเดลภาษาขนาดใหญ่โดยไม่สูญเสียความฉลาดอย่างมีนัยสําคัญ - ข่าว AI Aioga","description":"ByteByteGo ได้ตีพิมพ์บทความยาวอธิบายวิธีลดขนาดของโมเดลภาษาขนาดใหญ่โดยไม่ให้ดูเก้งก้างอย่างเห็นได้ชัด บทความนี้เริ่มต้นด้วยการเน้นความขัดแย้งระหว่างรุ่น 70B ที่มี VRAM ประมาณ 140 GB...","url":"https://www.aioga.com/th/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:43.157Z"},"pl":{"title":"ByteByteGo pokazał, jak kompresować duże modele językowe bez znaczącej utraty inteligencji","summary":"ByteByteGo opublikował obszerny artykuł wyjaśniający, jak zmniejszyć rozmiar dużych modeli językowych, nie stając się przy tym zauważalnie nieporęcznym. Artykuł zaczyna się od skupienia na sprzeczności modelu 70B z około 140 GB VRAM, podczas gdy karta graficzna klasy konsumenckiej ma tylko 24-48 GB VRAM.","category":"行业动态","source":"ByteByteGo（RSS）","aggregationSource":"ByteByteGo（RSS）","pageTitle":"ByteByteGo pokazał, jak kompresować duże modele językowe bez znaczącej utraty inteligencji - Aioga Wiadomości AI","description":"ByteByteGo opublikował obszerny artykuł wyjaśniający, jak zmniejszyć rozmiar dużych modeli językowych, nie stając się przy tym zauważalnie nieporęcznym. Artykuł zaczyna się od skup...","url":"https://www.aioga.com/pl/news/cmtiue30g059pro9yq1w7v7i3/","contentTranslated":true,"sourceHash":"b2a1230f192030e4","translatedAt":"2026-09-01T16:03:52.199Z"}},"evidenceTier":"verified-news","reviewStatus":"automated-ingest","indexable":true,"editorialCover":"/page-visuals/topic-timeline.png"}}