RAG 系统效果为何取决于嵌入模型：ByteByteGo 图解检索机制与选型要点

GlobalFoundries 的首席信息官对此有一个直接的看法：在底层数据实时且受到治理之前，你无法拥有 AI 代理。

因此，他首先重建了那一层——一个平台在三个大洲的晶圆厂之间传输数据，身份、权限和审计轨迹只处理一次，而不是每个项目都单独处理。随后，AI 代理也随之在 IT、采购和其他业务职能中部署。

加入我们，参加 9 月 10 日的活动，了解他是如何做到的，并现场提问。

保留您的座位：https://go.bytebytego.com/Redpanda_090226

RAG，即检索增强生成技术，正在帮助许多公司构建符合其特定需求的聊天机器人。然而，任何 RAG 系统的成功都取决于所使用的嵌入模型（负责将词语转换为数字的模型）的质量。

比如某个产品的文档规定，年度订阅只能在 30 天内退款。使用 RAG 构建的聊天机器人应根据这一支持文档回答客户问题。然而，当客户询问 45 天前购买的订阅是否可以退款时，聊天机器人却自信地回答“可以”。

为什么聊天机器人会在这个看似简单的问题上出错？

答案在于嵌入模型的行为，它有点像 AI 的翻译器。这个模型控制着答案搜索过程，与生成实际答案的语言模型不同。无论语言模型多么优秀，如果嵌入模型没有正确工作，都无法给出正确答案。

在本文中，我们将探讨嵌入模型在 RAG 设置中的工作原理，以及为什么它是系统中如此关键的一部分。我们将涵盖以下内容：

为什么RAG系统在回答之前会进行搜索

嵌入如何使按语义搜索成为可能

为什么相关信息不总是正确信息

为什么更好的语言模型无法弥补检索不佳的问题

是什么使嵌入模型适合 RAG 系统

如何在不盲目信任基准分数的情况下比较嵌入模型

在商业 API 和本地运行模型之间进行选择

为什么以后更换嵌入模型会变得昂贵

玛特里俄什卡嵌入如何提供对向量大小的更多控制

通用语言模型的知识取决于它在训练期间学习的内容。你不能指望这样的模型了解某个公司的私有文档、政策或内部源代码。它甚至不会知道公司在模型训练之后可能发布的最新信息。从技术上讲，每次信息发生变化时重新训练语言模型都是相当昂贵的。

这就是RAG（检索增强生成）发挥作用的地方。它将生成语言的能力与从存储知识池中查找相关信息分开。语言模型负责理解和写作部分，而外部知识集合则包含应在回答中使用的实际信息。

RAG的工作分为两个不同的阶段：索引和检索。

在索引阶段，系统准备文档：

它收集来自各种来源的文档（文件、网站、数据库等）。

它将文本划分为称为“块”的较小段落。

它将每个块发送到嵌入模型中。

它将生成的向量与原始文本和元数据一起存储。

元数据可能包含文档标题、发布日期、语言、版本等信息。

在检索阶段，系统按照以下步骤操作：

它将问题通过相同的嵌入模型发送。

它搜索与问题向量接近的文档向量。

它检索少量块。例如，最好的5块或类似数量。

它对这些块进行筛选和重新排序，并将它们放入语言模型的提示中。

最后，语言模型使用提供的提示撰写答案。

从这一切中得到的主要结论是，RAG 并不会将整个文档集合放入模型的提示中。这是因为这样会超过模型的上下文限制，并填充无关的信息。它也会增加成本和延迟。由 RAG 的嵌入模型主导的检索阶段，起到了选择步骤的作用，将成千上万的段落减少到语言模型可以检查的一小部分。

嵌入基本上只是表示一段文本的一列数字。一个实际的嵌入可能包含如 384、768、1024 或几千个数字。

这些单独的数字没有简单的标签来描述它们的含义。你不能看着某个值就说这意味着“订阅”，而另一个意味着“退款”。整体含义是分布在整个向量中的。可以把向量想象成数学空间中一个点的坐标。嵌入模型经过训练，将含义相关的文本放置在这个数学向量空间中彼此接近的位置。

这就是系统能够将像“yearly plan”“年计划”这样的术语与“annual subscription”“年度订阅”联系起来的方式。它也可以将复杂短语如“get my money back”“拿回我的钱”与“receive a refund”“收到退款”联系起来。相比之下，当问题和文档使用不同的词汇来解释同一概念时，关键字搜索会遇到困难。但是嵌入模型尝试比较文本中的概念，而不仅仅是词汇。背后的常用技术有余弦相似度、点积和欧几里得距离。简而言之，嵌入模型的目标是生成一个数学分数，表示两个向量彼此的接近程度。

嵌入模型通常返回前 k 个结果。这称为 top-k 检索。例如，如果 k 是 5，则检索返回排名前 5 的块。

嵌入模型还支持查询和文档形式不同的非对称检索。查询可能是一个简短的问题，而匹配的文档是一段较长的解释性内容。例如，查询可能是：“年订阅在 6 周后可以退款吗？”答案段落是这样的陈述：“年度订阅可在 30 天内退款。”

一个训练用来比较相似句子的嵌入模型，可能不如一个训练用来将问题与相关段落连接的模型表现得好。总而言之，嵌入模型定义了RAG系统认为相似的内容。

嵌入模型被训练来识别语义相似性。但RAG系统需要更严格的东西。它需要找到可能包含回答特定问题所需信息的段落。问题在于，一个段落可能与问题相关，但并没有明确回答问题。

例如，一个客户问退款需要多长时间，可能会收到一个解释谁有资格退款的段落。两条信息都涉及退款，但只有一条谈及实际处理时间。请参见下图，显示语义搜索空间的概念。

在这种情况下，可能出现多种失败模式：

主题相似，问题不同：查询问“批准的退款需要多长时间到帐？”检索到的段落说：“购买可以在30天内退款。”该段落与退款相关，但没有回答时间问题。

相同词语，不同实体：查询问“如何更改账单地址”。检索到的段落解释了如何更改账户的电子邮件地址。两者都在谈账户详情的更改，但它们指的是完全不同的字段。

否定：考虑这两个段落：“管理员可以删除归档项目”和“管理员不能删除归档项目”。大部分词语是相同的。这意味着它们的嵌入可能很接近。但如你所见，它们的含义完全相反。

版本和日期：知识库可能包含旧政策及其替代版本。文本几乎相同，除了日期、限制或价格。嵌入无法自动判断哪份文档是权威的。可能需要通过元数据过滤器或版本管理规则明确排除过时内容。

数值标识符：两句话“年订阅可以在30天内退款”和“年订阅可以在60天内退款”在语义上非常相似。但其中一个数字不同，这决定了最终答案。

领域特定含义：通用模型可能会误解专业词汇。例如，“capture”在日常语言中有一个意思，而在支付处理中有一个特定意思。类似地，单词“port”可以指网络、硬件或在不同平台间迁移软件。用于一般网页文本的最佳模型可能不是法律合同、医疗报告、财务文件或源代码的最佳模型。

多部分问题：客户可能会提出多部分问题。例如，可能有这样一个问题：“我可以取消订阅吗，退款需要多长时间？”回答这个问题至少需要两段内容。一段可能解释取消资格，另一段可能解释处理时间。检索到的仅包含一个主题的模型可能会生成不完整的答案。

在RAG系统中，语言模型只看到用户的问题和选定的段落。它不会看到存储在向量数据库中的每个文档。

例如，如果年度退款政策文档没有被检索到，语言模型就不会知道缺失的信息。更强大的语言模型可能会识别检索到的信息中不包含答案。这是有用的，因为它至少可以选择不提供无效信息。然而，这种检索失败的场景也可能产生几种结果：

模型根据其通用训练知识作答。

它错误地应用了相关的段落。

它表示可用的信息不足。

它错误地组合了冲突的段落。

诸如“仅从提供的文档中回答”的提示可以减少无依据的答案。但它无法让正确的文档神奇地出现。

这就是为什么在RAG系统开发过程中测试和调试非常关键。开发人员需要在更改提示或更换语言模型之前检查检索到的块。在生成阶段解决检索问题需要花费更多时间。

选择嵌入模型时最重要的关注点应是检索性能。模型应非常擅长将简短问题与可能包含答案的长句子关联起来。

以下特征很重要：

The CIO of GlobalFoundries had a straightforward view of it: you don’t get AI agents until the data underneath them is real-time and governed.

So he rebuilt that layer first — one platform carrying data across fabs on three continents, with identity, permissions, and audit trails handled once instead of per project. The agents followed, across IT, procurement, and other business functions.

Join us on September 10 to hear how he did it, and get your questions answered live.

Save your seat ：https://go.bytebytego.com/Redpanda_090226

RAG, or Retrieval-Augmented Generation, is helping a lot of companies build chatbots specific to their requirements. However, the success of any such RAG system depends on the quality of the embedding model (the model that helps translate words into numbers) that is used by the RAG system.

Consider the example where a particular product’s documentation specifies that annual subscriptions can be refunded only within 30 days. A chatbot built using RAG is supposed to answer customer queries based on this support documentation. However, when a customer asks whether a subscription bought 45 days ago can be refunded, the chatbot confidently answers “yes”.

Why did the chatbot make a mess of this seemingly simple question?

The answer lies in the behaviour of the embedding model, which is kind of a translator for AI. This model controls the answer-searching process and is different from the language model that generates the actual answer. No matter how good the language model might be, it cannot give correct answers if the embedding model doesn’t do its job properly.

In this article, we’re going to look at how this embedding model works in an RAG setup and what makes it such a critical part of the system. Here’s what we will cover:

Why a RAG system searches before it answers

How embeddings make it possible to search by meaning

Why related information is not always the right information

Why a better language model cannot repair bad retrieval

What makes an embedding model suitable for a RAG system

How to compare embedding models without trusting benchmark scores blindly

Choosing between commercial APIs and locally run models

Why changing the embedding model later becomes expensive

How Matryoshka embeddings offer more control over vector size

A generic language model’s knowledge depends on what it was taught during training. You can’t expect such a model to know about a company’s private documents, policies, or internal source code. It won’t even know about the company’s latest information that might have been published after the model was trained. Technically, it’s quite costly to retrain a language model every time there’s a change in the information.

This is where RAG helps. It separates the ability to generate language from finding relevant details from a pool of stored knowledge. While the language model handles the understanding and writing parts, the external knowledge collection contains the actual information that should be used for writing the answers.

RAG works in two different phases: indexing and retrieval.

During the indexing phase, the system prepares the documents:

It collects documents from various sources (files, websites, databases, etc.).

It divides the text into smaller passages known as chunks.

It sends each chunk into an embedding model.

It stores the resulting vector with the original text and metadata.

The metadata may contain details such as the document title, publication date, language, version, and so on.

During the retrieval phase, the system follows the following steps:

It sends the question through the same embedding model.

It searches for document vectors that are close to the question vectors.

It retrieves a small number of chunks. For example, the best 5 chunks or something like that.

It filters and reranks those chunks, placing them inside the language model’s prompt.

Finally, the language model writes an answer using the prompt that was provided.

The main takeaway from all this is that RAG doesn’t place the entire document collection into the model’s prompt. This is because doing so would exceed the model’s context limit and fill it with irrelevant information. It also increases costs and latency. The retrieval phase, led by the RAG’s embedding model, serves as the selection step, reducing thousands of passages to a small set for the language model to check.

An embedding is basically just a list of numbers that represents a piece of text. A real embedding might contain numbers such as 384, 768, 1024, or several thousand numbers.

These individual numbers don’t have simple labels describing their meanings. You cannot look at a particular value and say that this means “subscription” while another means “refund”. The overall meaning is distributed across the complete vector. Think of the vector as the coordinates of a point in a mathematical space. An embedding model is trained to place text with related meaning near each other within this mathematical vector space.

This is how the system can connect a term like “yearly plan” with “annual subscription”. It can also connect complex phrases like “get my money back” with “receive a refund”. In contrast, a keyword search struggles when a question and the document use different words to explain the same concept. But an embedding model tries to compare the ideas present within the text rather than just the vocabulary. Common techniques behind this are cosine similarity, dot product, and Euclidean distance. In a nutshell, the goal of an embedding model is to produce a mathematical score that indicates how close two vectors are to each other.

The embedding model often returns the first k results. This is known as top-k retrieval. For example, if k is 5, the retrieval returns the top 5 highest-ranked chunks.

Embedding models also support asymmetric retrieval where the query and document have different forms. The query may be a short question. But the matching document is a longer explanatory passage. For example, the query could be something like: “Can an annual subscription be refunded after 6 weeks?” The answer passage is a statement like: “Annual subscriptions can be refunded within 30 days.”

An embedding model trained to compare similar sentences may not perform as well as a model that is trained to connect questions with the relevant passages. To summarize, the embedding model defines what the RAG system considers similar.

An embedding model is trained to identify semantic similarity. But a RAG system requires something a bit more strict. It needs to find passages that might contain the information needed to answer a specific question. The problem is that a passage can be related to the question without clearly answering it.

For example, a customer asking how long a refund takes might receive a passage explaining who qualifies for the refund. Both these pieces of information concern refunds, but only one talks about the actual processing time. See the diagram below that shows the concept of semantic search space.

Multiple failure modes can pop up in this situation:

Similar Subject, Different Question: The query asks “How long will an approved refund take to arrive?” The retrieved passage says: “Purchases can be refunded within 30 days.” The passage is relevant to refunds but does not answer the timing question.

Same Words, Different Entity: The query asks “how can the billing address be changed.” The retrieved passage explains how to change the account’s email address. Both talk about changing account details. But they refer to completely different fields.

Negation: Consider the two passages: “Administrators can delete archived projects”, and “Administrators cannot delete archived projects”. Most of the words are identical. This means their embeddings may be close. But as you can see, they have opposite meanings.

Versions and Dates: A knowledge base may contain an old policy and its replacement. The text may be almost identical except for a date, limit, or price. Embeddings cannot automatically know which document is authoritative. Metadata filters or version management rules may need to exclude outdated content explicitly.

Numerical Identifiers: The two sentences “Annual subscriptions can be refunded within 30 days” and “Annual subscriptions can be refunded within 60 days” are semantically very similar. But there is a difference in one number, which determines the final answer.

Domain-specific Meanings: General models can misunderstand special vocabulary. For example, “capture” has one meaning in an ordinary language and a specific meaning in payment processing. Similarly, the word “port” can refer to networking, hardware, or moving software between platforms. The best model for general web text may not be the best model for legal contracts, medical reports, financial documents, or source code.

Multi-part questions: A customer may ask multi-part questions. For example, there could be a question like “Can I cancel the subscription, and how long will the refund take?” The answer for this requires at least two passages. One might explain eligibility for cancellation. Another might explain processing time. The model that retrieves only one subject may produce an incomplete answer.

In an RAG system, the language model just sees the user’s question and the selected passages. It doesn’t see every document that might have been stored in the vector database.

For example, if the annual refund policy document is not retrieved, the language model won’t have any idea about the missing information. A more capable language model may recognize that the retrieved information doesn’t contain the answer. This is useful because it can at least choose not to respond with invalid information. However, this failed retrieval scenario also creates several possible outcomes:

The model answers from its general training knowledge.

It incorrectly applies a related passage.

It says that the available information is insufficient.

It combines conflicting passages incorrectly.

A prompt such as “answer only from the supplied documents” can reduce unsupported answers. But it cannot make the correct document magically appear.

This is why testing and debugging are so critical during the development of an RAG system. Developers need to inspect retrieved chunks before changing prompts or swapping language models. It takes a lot more time to solve a retrieval problem in the generation phase.

The most important concern when choosing an embedding model should be retrieval performance. The model should be really good at connecting short questions with longer sentences that might contain the answers.

The following characteristics matter: