政策针对 PR 审查者、LLM 生成代码的提交者、用 LLM 发现并发布 issue 的人以及直接引用 LLM 内容的评论者,要求公开规则以减少审查负担和社区摩擦。
最近,Rust 项目中的五个团队:https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rust-forge/pull/1040#issuecomment-4438128685 采用了一项政策:https://forge.rust-lang.org/policies/llm-usage.html,这项政策最初由我起草,用于规范在向 rust-lang/rust 单仓库贡献时如何使用大型语言模型(Large Language Models):https://en.wikipedia.org/wiki/Large_language_model。值得注意的是,这项新政策并不是 Rust 官方对 LLM 的立场,也并不适用于 Rust 项目的所有地方。我为一个非常具体的目的起草了它,具体说明如下。
本文讲述了我们为何创建该政策、政策内容以及这将如何影响贡献者。
该政策影响以下群体的人:
如果你不属于这些群体之一,你不必改变工作方式。
虽然 Rust 项目是技术工件的集合,但它也是一个社区,人们在其中协作以构建、维护和扩展这些工件。当我们谈论“为 Rust 项目做贡献”时,我们部分指的是对这些工件的工作,但也意味着加入这个社区并与已经在其中的人合作。
即便在这项政策创建之前,人们也在使用 LLM 为 rust-lang/rust 做贡献。其中一些用途遵循了我们的社区规范:将消息翻译成英文,以便人们能用母语起草;发现新贡献者在 Rust 中可能编写的代码片段的差劲诊断信息;分析 RFC,看看是否遗漏了其他可能影响设计的语言部分的讨论。其中有些用途,有时是无意的,并未遵守规范。
我见过 LLM 给我们的社区带来了三个主要问题:
随着时间推移,这些问题越来越严重,直到我们不得不创建专门的渠道和管理政策来处理它们。然而,这些渠道与我们保持透明和欢迎的目标相悖,因为新贡献者根本不知道规则是什么。
新政策将这些规则正式公开,以便新贡献者知道如何加入我们的社区,而不会因为不理解原因而导致 PR 被关闭;同时,也方便现有的审核者在关闭不遵循规定的 PR 时,有明确可操作的规则可参考。
过去,如果一个开源项目收到一个经过精心打磨、测试详尽的 PR,那通常意味着在另一端有人投入了时间、精力和理解。这在多个方面影响了 Rust 的文化:
使用 LLM 后,这些信号都不再可靠。精致的 PR 不再代表努力;精致 PR 的作者不一定理解他们的代码——在自主代理的情况下,甚至不再有人在另一端;而且由于写代码变得容易得多,精致的 PR 不再表示有人可能会长期参与。
在撰写本文时,rust-lang/rust 的开启 PR 有 1,281 个。这代表了作者和审阅者投入的惊人时间。长期以来,我们一直面临一个问题:想写代码的人多于愿意审阅的人。随着 LLM 的出现,这个问题只会变得更糟。
大部分审阅工作并不仅仅是找出错误。很大一部分是决定这个方向是否是好的做法,这个 PR 是否本身就是一个好主意。换句话说,审阅主要是由决策组成:https://web.archive.org/web/20260213080731/https://siderea.dreamwidth.org/1219758.html。
将 PR 批量投给审阅者会给他们带来高心理成本。我认为大多数 LLM PR 的作者相信他们是真心在帮忙,但在我们看来,代码本身是最小的、在某些方面也是最不重要的部分。我们更关心的是作者理解代码的功能、规划它未来如何变化,以及决定它应该是什么样子。代码本身无法帮到这些。
我们经常看到有人通过将审阅意见复制到他们的 LLM 中,然后将其回复再粘贴回 GitHub 来回应评论。坦率地说:这是在浪费大家的时间。如果我们想要 LLM 的意见,我们自己会去问。我们想听的是你的想法,而不是机器的。
此外,这是对审阅者和作者之间信任的破坏。我们在审阅时的假设是,我们在与一个希望尽最大努力的人交流。粘贴 LLM 文本会引发怀疑:作者真的在乎吗?这里到底有人吗?
在这条政策之前,我们在管理方面采取的是“狂野西部”式的方法。我们有几十个 LLM PR;没有披露规则;有人尝试在他们的第一个 PR 中添加高风险的 MIR 优化:https://rustc-dev-guide.rust-lang.org/mir/optimizations.html;还有人在他们的 PR 描述中发布“Verification: git diff --check:https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---check”,好像那能起作用。虽然我们为管理员提供了“授权审阅者拒绝繁琐 PR”的依据:https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/compiler-team/issues/893,但我们的执行不一致,规则也未在任何地方公布。实际上,规则是“只要不是明显糟糕的,什么都行”。与以前的情况相比,新政策既严格得多,又清晰得多。
无论你对 LLM 是好、是坏,还是某种神秘的第三类有何看法,它们都不再能被忽视。我们的选择不是“没有政策”或者“有政策”。我们的选择是让政策成为非官方的管理笔记,还是成为我们公开坚持的内容。
为什么不干脆禁止 LLM,或者允许我们认为有益社会的 LLM 使用?因为 Rust 的治理不是那样运作的。我们没有一个仁慈的独裁者可以说:“不允许 LLM 生成的内容,无论是代码还是文字。”:https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy,或者“人工智能是一种工具,就像我们使用的其他工具一样”:https://lore.kernel.org/linux-media/CAHk-=wi4zC+Ze8e+p3tMv8TtG_80KzsZ1syL9anBtmEh5Z40vg@mail.gmail.com。
Rust 的运作基于共识。正如政策所说:
在 Rust 项目内部——也很可能永远不会——关于何时/如何/何地可以接受使用 AI 工具,没有形成共识。Rust 项目和社区的许多成员认为 AI 有价值;许多其他人则认为 AI 对社会和气候的负面影响严重到没有任何可接受的使用。还有一些人正在形成自己的意见。
尽管存在这些分歧,但我们共有许多价值观:
我们希望将来可以更改该政策。该政策包含几项条款:https://forge.rust-lang.org/policies/llm-usage.html#conditions-for-modification-or-dissolution,使其比最初采纳时更容易更改。领导委员会也在考虑创建一个子团队:https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/leadership-council/issues/308,专门处理LLM政策,这样我们就无需经过“噩梦般”的30人批准程序。
我认为这项政策中的每条规则都不完全是好的。但我确实认为,将规则记录下来总比不记录好,而且拥有一项每个人都有些不满的政策会推动我们改进治理结构。
该政策总结如下:
使用LLM回答问题、分析、提炼、改进、检查、建议、审查是可以的。但不可以用来创作作品。
第一类用途是允许的,有时需要披露;第二类用途受到严格限制。
除了作者本人外,任何人都不必阅读LLM输出,除非他们选择这样做:LLM输出不得出现在公共文档、PR描述或Github评论中,除非明确标注;审查者不必查看LLM PR,如果他们不愿意。
没有人必须使用LLM来为rust-lang/rust做贡献:政策必须首先为人类编写,机器只能作总结;LLM审查不能替代人工审查或自我审查。
你可以生成只有你自己看到的LLM内容,而无需披露,只要你不在我们预期会阅读或审查的地方发布。
使用LLM进行机器翻译、“琐碎”变更、发现错误以及审查他人工作时需要披露。我们欢迎使用你的母语发布信息;贡献时不需要英文翻译。
对于LLM生成的代码更改,有非常严格的指南:
预先安排的、非关键的、高质量、经过充分测试和审查的、最初由LLM创建的代码更改是允许的,但需披露。
该政策对 LLM 生成的变更要求比人工创作的变更更高,而不是更低:LLM 提交请求(PR)必须包含测试,毫无例外,不论这有多困难,同时还必须遵守其他各种限制;除非作者已经是该领域的专家,否则 LLM 不得生成对正确性关键的变更,即使是专家也强烈不建议这样做。
总体而言,该政策侧重于理解,帮助确保我们对代码有心智模型,而不仅仅是机械地生成正确的结果。我们的动机受 Isaac Asimov 的《职业》(Profession)影响:https://web.archive.org/web/20201109034130/https://www.abelard.org/asimov.php。没有程序员的录音带。
你必须披露 LLM 生成的内容。你可以选择不发布 LLM 内容,或者选择发布并披露其来源。你不得隐藏 LLM 的参与。
不允许骚扰。无论政策是否禁止使用 LLM,你都不得因他人使用 LLM 而骚扰他们。在与 Rust 项目互动时,你必须始终遵守行为规范:https://rust-lang.org/policies/code-of-conduct/。
请参阅政策本身:https://forge.rust-lang.org/policies/llm-usage.html 以获取更多信息。
政策的某些部分无法强制执行。这不是错误。目标并不是抓住每个违规行为,而是创建明确的界限规则:所有公开的 LLM 文本必须披露,除非政策明确豁免。这使得版主可以根据行动而不是意图来识别违规行为,并且仅在决定如何回应时考虑意图。
你必须披露在发现或报告问题时 LLM 的任何参与。如果你使用 LLM 发现问题,必须告知我们。你必须清楚引用并标明报告中哪些部分是 LLM 生成的;"不得发布 LLM 生成的评论" 的规则也适用于你。
我已经写了一份指南,供你在向 rust-lang/rust 提交包含 LLM 生成代码的 PR 时遵循。如果你遵循政策中的这一简单指南,你可以避免去考虑这些指南,甚至完全不用阅读这份清单:
请参阅政策中的“允许”部分:https://forge.rust-lang.org/policies/llm-usage.html#-allowed 以获取完整的允许内容列表。参见 rustc-dev-guide:https://rustc-dev-guide.rust-lang.org/llm-guidance/writing.html 以获取完整的指南列表。
您可以关闭不符合政策的 PR,无需任何疑问。同时请将作者引导至 #llm-mentoring:https://rust-lang.zulipchat.com/join/rlfvpemsaacs3pfi6kwqnqjb/。具体情况和建议措辞请参见开发指南:https://rustc-dev-guide.rust-lang.org/llm-guidance/reviewing.html。
您无需负责判断 PR 是否由 LLM 生成;该责任在于作者。我们将增加一个 PR 模板,要求作者说明其代码是否由 LLM 生成,以减少此类情况的发生。
如果作者声称他们的代码不是由 LLM 生成,但您仍不确定,请将 PR 私下报告给管理团队。风格不是证据,请不要指控他人使用 LLM。报告并非为了处罚,管理团队同样希望看到未违规的情况。
如果您自愿审查 LLM PR,以下部分适用于您。除非自愿,否则没有人需要审查 LLM PR。
每个人都应遵守新政策,而不仅仅是作者。这意味着您有责任检查 LLM 生成的 PR 是否涉及政策禁止的区域,例如文档、诊断或影响安全性的更改。您可以要求作者在没有 LLM 生成代码的情况下重新提交,在这种情况下本节不适用。
您需要执行的规则在开发指南中有更详细的摘要:https://rustc-dev-guide.rust-lang.org/llm-guidance/reviewing.html。官方政策:https://forge.rust-lang.org/policies/llm-usage.html 仍然是权威。
大量工作由版主、团队负责人、审查员、委员会代表以及项目内外的其他人员共同完成。其中一些工作甚至早在政策制定前几个月就已经开始。我想感谢所有直接或间接参与贡献的人。
这不是故事的结局。该政策的目标之一是帮助我们收集数据:人们是否在使用大型语言模型(LLM)做有趣且有用的事情?他们是否在学习?他们是否在重复作出贡献?这些问题的答案将帮助我们确定政策未来的变化方向。
这并不是 Rust 项目团队发布的第一个 LLM 政策,希望也不会是最后一个。尽管当前的政策仅适用于 rust-lang/rust 单一代码库,但我仍然相信,Rust 将从一项项目范围的政策中受益,该政策明确规定我们在聊天、论坛、公共交流、没有明确政策的代码库以及其他跨项目领域的期望。
Recently, five teams in the Rust project:https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rust-forge/pull/1040#issuecomment-4438128685 adopted a policy:https://forge.rust-lang.org/policies/llm-usage.html that I originally authored, governing how Large Language Models:https://en.wikipedia.org/wiki/Large_language_model can be used when contributing to the rust-lang/rust monorepo. Notably, the new policy is not an official stance on LLMs, and does not apply everywhere in the Rust project. I wrote it for a very specific purpose, described below.
This post talks about why we created that policy, what it says, and how this will affect contributors.
The policy affects the following groups of people:
If you are not in one of those groups, you don't have to change anything about how you work.
While the Rust project is a collection of technical artifacts, it is also a community of people who work together to build, maintain, and extend those artifacts. When we talk about "contributing to the Rust project", we partly mean work on those artifacts, but we also mean joining that community and collaborating with the people already there.
Even before this policy was created, people were using LLMs to contribute to rust-lang/rust . Some of those uses respected our community: translating messages to English so people could draft them in their native language; finding poor diagnostics for code snippets that new contributors to Rust might write; analyzing RFCs to see if they were missing a discussion of other parts of the language that could affect the design. Some of those uses, sometimes unintentionally, did not.
I've seen LLMs causing three main issues for our community:
As time went on, these issues grew and grew, until we had to create dedicated channels and moderation policy for how to deal with them. However, those channels work against our goals of being transparent and welcoming, because new contributors have no idea what the rules are.
The new policy formalizes those rules publicly, so that new contributors know how to join our community without getting their PRs closed for reasons they don't understand, and so that existing reviewers can easily point to the rules as an actionable reason when closing PRs that don't follow them.
It used to be that if an open source project got a polished, well-tested, detailed PR, that indicated that there was someone on the other end who had put time, effort, and understanding into the PR. That influenced Rust's culture in several ways:
With LLMs, none of these signals are reliable. Polished PRs no longer indicate effort; authors of polished PRs no longer necessarily understand their code—and in the case of autonomous agents, there is no longer someone on the other end at all; and because it's become so much easier to write code, a polished PR no longer indicates that someone is likely to stick around for the long term.
At the time of writing, there are 1,281 open PRs to rust-lang/rust . This represents a staggering amount of time invested by both authors and reviewers. We have long had the problem that there are more people who want to write code than people willing to review it. With the advent of LLMs, this problem only gets worse.
Most of the work of reviewing is not simply catching bugs. A great deal of it is deciding whether this direction is a good approach , whether the PR is a good idea at all. In other words, reviewing is made of decisions:https://web.archive.org/web/20260213080731/https://siderea.dreamwidth.org/1219758.html.
"Shotgunning" PRs at reviewers incurs a high mental cost for them. I think most authors of LLM PRs believe that they are sincerely helping, but from our perspective, the code itself is the smallest and in some ways least important part of the change. We care much more about authors understanding what the code does, planning how it will change in the future, and deciding what it should look like. The code itself cannot help with any of those.
We will often get people who respond to review comments by copy-pasting them into their LLM, then copy-pasting its response back onto GitHub. Bluntly: this is a waste of everyone's time . If we wanted an LLM's opinion, we could have asked it ourselves. We want to hear your thoughts, not a machine's.
Moreover, this is a breach of trust between the reviewer and the author. Our assumption when we review is that we're talking to a real person who wants to do their best work. Pasting LLM text creates suspicion: does the author actually care? Is there a person here at all?
Before this policy, we had a "wild west" approach to moderation. We had dozens of LLM PRs; no disclosure rules; people trying to add risky MIR optimizations:https://rustc-dev-guide.rust-lang.org/mir/optimizations.html as their first PR; and people posting "Verification: git diff --check :https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---check" in their PR description as if that did something. While we had something for moderators to point to in the form of "Empower reviewers to reject burdensome PRs":https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/compiler-team/issues/893, our enforcement was inconsistent and our rules were not published anywhere. In practice, the rule was "anything goes, as long as it's not obviously horrible". Compared to the previous situation, the new policy is both much more strict and much more clear.
Regardless of your opinions on whether LLMs are good, bad, or a secret third thing, they can no longer be ignored . Our choices are not "no policy" or "policy". Our choice is whether to have the policy be an unofficial list of moderation notes or something we stand by publicly.
Why not ban LLMs altogether, or allow any use of LLMs that we think are pro-social? Because Rust governance doesn't work that way. We do not have a benevolent dictator who can say "No LLM-generated content, whether it be code or prose.":https://ziglang.org/code-of-conduct/#strict-no-llm-no-ai-policy or "AI is a tool, just like other tools we use":https://lore.kernel.org/linux-media/CAHk-=wi4zC+Ze8e+p3tMv8TtG_80KzsZ1syL9anBtmEh5Z40vg@mail.gmail.com/.
Rust operates by consensus. As the policy says:
There is not a consensus within the Rust project—and likely never will be—about when/how/where it is acceptable to use AI-based tools. Many members of the Rust project and community find value in AI; many others feel that its negative impact on society and the climate are severe enough that no use is acceptable. Still others are working out their opinion.
Despite these differences, there are many values we all share:
We want it to be possible to change the policy in the future. The policy has several provisions:https://forge.rust-lang.org/policies/llm-usage.html#conditions-for-modification-or-dissolution that make it easier to change than it was to originally adopt. The leadership council is also considering creating a sub-team:https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/leadership-council/issues/308 that would handle LLM policy so we have fewer "nightmare" 30-person approval requirements.
I do not think every rule in this policy is wholly good. I do think that writing our rules down is better than not writing them down, and that having a policy that everyone kinda dislikes pushes us to improve our governance structures.
The policy summarizes itself this way:
It's fine to use LLMs to answer questions, analyze, distill, refine, check, suggest, review. But not to create .
Uses in the first category are allowed, sometimes requiring disclosure. Uses in the second category are heavily restricted.
No one except the author is required to read LLM output unless they choose to: LLM output isn't allowed in public docs, PR descriptions, or Github comments unless it's clearly marked; reviewers aren't required to look at LLM PRs if they don't want to.
No one is required to use LLMs to contribute to rust-lang/rust : policies must be written first for humans, and only summarized for machines; LLM reviews cannot substitute for human review or self-review.
You are allowed to generate LLM content that only you see, without disclosure, as long as you do not post it anywhere that you expect us to read or review.
Disclosure is required for machine translation, "trivial" changes, discovering bugs, and reviewing other people's work using an LLM. We welcome messages posted in your native language; English translation is not required to contribute.
There are very strict guidelines on LLM-generated code changes:
Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure .
The policy holds LLM-generated changes to a higher bar than human-authored changes, not a lower one: LLM PRs are required to have tests, full stop, regardless of how hard that is, as well as various other restrictions; LLMs must not generate soundness-critical changes unless the author is already a domain expert, and even then it's strongly discouraged.
In general, the policy focuses on understanding , helping to ensure that we have a mental model of our code, not just artifacts that mechanically do the right thing. Our motivation is influenced by Profession by Isaac Asimov:https://web.archive.org/web/20201109034130/https://www.abelard.org/asimov.php. No programmer tapes.
You must disclose LLM-generated content. You can choose to not post LLM content, or you can choose to post it and disclose its origin. You may not hide LLM involvement.
Harassment is not allowed. You may not harass people for using an LLM, regardless of whether or not their use is banned by the policy. You must follow the Code of Conduct:https://rust-lang.org/policies/code-of-conduct/ at all times when interacting with the Rust project.
See the policy itself:https://forge.rust-lang.org/policies/llm-usage.html for more information.
Some parts of the policy are unenforceable. This is not a bug. The goal is not to catch every violation, but to create a clear bright-line rule: Disclosure is required for all public LLM text, unless specifically exempted by the policy. This allows moderators to identify violations based on actions , not on intent, and only consider intent when deciding how to respond.
You must disclose any LLM involvement in discovering or reporting issues. You must tell us if you found an issue using an LLM. You must clearly quote and indicate which parts of your report were LLM-generated; the "no LLM-generated comments" rule applies to you too.
I have written a list of guidelines you should follow if you make a PR to rust-lang/rust with LLM-generated code. You can avoid thinking about them, or indeed reading the list at all, if you follow this simple guideline from the policy:
See the "Allowed" section of the policy:https://forge.rust-lang.org/policies/llm-usage.html#-allowed for a full list of what's meant by that. See rustc-dev-guide:https://rustc-dev-guide.rust-lang.org/llm-guidance/writing.html for the full list of guidelines.
You are allowed to close PRs that don't follow the policy, no questions asked. Please point the author to #llm-mentoring:https://rust-lang.zulipchat.com/join/rlfvpemsaacs3pfi6kwqnqjb/ at the same time. See the dev guide:https://rustc-dev-guide.rust-lang.org/llm-guidance/reviewing.html for exact circumstances and suggested wording.
You are not responsible for determining whether a PR is LLM-generated; that responsibility lies with the author. We will add a PR template that asks authors whether their code was LLM-generated so that this rarely comes up.
If an author claims their code is not LLM-generated, but you're still not sure, please report the PR privately to moderation. Style is not evidence; please do not accuse people of using an LLM. Reporting is not intended to be a penalty; the mod team is interested in seeing non-violations as well as violations.
If you have volunteered to review LLM PRs, the following section applies to you. No one is required to review LLM PRs unless they volunteer.
Everyone is expected to follow the new policy, not just authors. That means it is your responsibility to check whether an LLM-created PR touches an area that's disallowed by the policy, such as docs, diagnostics, or soundness-critical changes. You may request that the author redo it without LLM-generated code, in which case this section doesn't apply.
There's a more detailed summary of the rules you're expected to enforce in the dev guide:https://rustc-dev-guide.rust-lang.org/llm-guidance/reviewing.html. The official policy:https://forge.rust-lang.org/policies/llm-usage.html remains canonical.
Quite a lot of work has been put into this by moderators, team leads, reviewers, council representatives, and various other people inside and outside the project. Some of that work started months ago before the policy itself was written. I'd like to thank everyone who contributed, whether directly or indirectly.
This is not the end of the story. One of the goals of the policy is to help us gather data: Are people doing interesting and useful things with LLMs? Are they learning? Are they making repeat contributions? The answers to those questions will help us determine how the policy changes in the future.
This is not the first LLM policy published by teams in the Rust project, and hopefully it will not be the last. While the current policy only applies to the rust-lang/rust monorepo, I still believe that Rust would benefit from a project-wide policy that specifies what we expect in chats, forums, public communications, repositories without an explicit policy, and other cross-project areas.