Anthropic 团队透露 Claude Tag 承担 65% 产品工程 PR，系统提示词缩减 80%

本月早些时候，我在AI工程师世界博览会上主持了一场炉边谈话会议：https://www.ai.engineer/worldsfair/2026，与来自Anthropic Claude Code团队的Cat Wu和Thariq Shihipar一起。我们讨论了Claude Code、Claude Tag、Fable、编码代理安全性、评估、工具设计，以及Anthropic如何自己使用这些工具。

会议的完整视频现已在YouTube上可观看：https://www.youtube.com/watch?v=uU5Gv2h8-9g。以下是整理过的文字记录，附加了更多链接和我自己加粗的重点部分。

如果你不想看视频或阅读整个文字记录，这里有一些顶层要点：

1:05：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=65s

Simon：Claude Code是在去年二月发布的——还不到一年半的时间，它最初只是Claude Sonnet 3.7发布时的一个要点：https://www.anthropic.com/news/claude-3-7-sonnet。在过去一年里，现在我们有这些真正为我们工作的编码代理，那么你们的日常工作发生了哪些变化？

Cat：我记得当我们刚推出Claude Code和Sonnet 3.7时，你给它一个任务，你必须密切监控它尝试做的每一件小事。我会非常仔细地阅读每一个权限提示。我经常会说不——不，不，不，你检查过这个文件了吗？你检查过那个文件了吗？而现在每一次模型生成都非常令人难以置信。我觉得我们都有机会退一步，把很多琐碎的实现工作交给Claude。它释放了我们大量时间，让我们可以思考更多创造性的工作，比如：现在我们知道Claude Code可以实现很多功能，我们应该为用户提供怎样的正确体验？现在有了Fable，这就是一个完全不同的跃升式改进。我们在很多使用场景中看到，你现在实际上可以用Fable一次性实现大量功能。

Thariq: 我记得第一次收到关于 Claude Code 的消息。我最好的朋友之一说：“你得去试试 Claude Code。”那是在 Opus 4 发布的时候，我尝试了一下，然后我心里想，“哦，天啊。我现在必须去 Anthropic 工作了。”那就是 Opus 4——很棒的模型，但你还需要阅读权限提示。真是太疯狂了，我们竟然有这么多的健忘，我都不记得自己点击过是和允许。对我来说，我努力推动自己的一个重点是：我们必须比以往任何时候都做出更高质量的工作。输出的质量非常高。我一直在用它编辑视频，我心想，好吧，它必须在几个小时内满足我们品牌团队非常严格的要求，否则我们就做不成。这就是我试图在 Fable 上做的转变：比以往任何时候都更快地完成我们做过的最好工作。

3:39：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=219s

Simon: 在过去一年仍然被认为正确的传统软件工程做法，在这个新世界里，你觉得不再适用了的有哪些？

Cat: 我们看到的工程技能最大的变化之一是：两年前，产品经理去和一堆客户交流，花六个月时间与跨职能团队协调一些 PRD，然后写一个详尽的规格文档，说明我们将如何实现这一点，第一行代码还没写就已经准备好了。现在情况完全相反。对于很多工程师来说，我会鼓励现场的朋友更多地发展你的商业意识和产品意识，搞清楚我们应该构建什么，因为从有想法到实际构建的时间大大缩短——从六到十二个月缩短到甚至可能只有一周。这意味着我们都需要更好地判断什么值得构建，什么能真正影响我们正在工作的业务。因此，产品品味和商业意识的重要性增加，而大多数产品领域的执行力相对降低。当然，对于基础设施（infra）来说，仍然非常强调确保所有细节正确无误。

Thariq: 对我来说，现在重写也是好事。

Simon: 现在你能做的最糟糕的事情实际上是非常棒的!

Thariq：没错。所有《人月神话》里的内容——永远不要重写——我现在支持重写。如果你有一个好的测试套件——而且我认为重写实际上会迫使你确保你有一个好的测试套件——但我认为人们低估了的是，代码库本身就是规范，也许它是你唯一拥有的规范副本，因为没有人能知道代码库每一个分支的细节。你可以把它作为一种工件来提炼，或者创建它的其他版本。我们用 Rust 重写了 Bun：https://bun.com/blog/bun-in-rust，而且效果非常好——对我来说现在它已经上线了。

Simon：你们还没在 Bun-in-Rust 上发布 Claude Code，对吧？

（实际上看起来 Anthropic 已经从 6 月 17 日开始向所有人发布在 Bun-in-Rust 上的 Claude Code：https://simonwillison.net/2026/Jul/19/claude-code-in-bun-in-rust/。）

6:36：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=396s

Simon：最近的另一个重大发布是 Claude Tag：https://www.anthropic.com/news/introducing-claude-tag —— 这对于我们其他人来说，至少也才一周左右。我了解到 Anthropic 内部非工程人员正在大量使用它。非工程人员都在用 Claude Tag 做些什么？

Cat：Claude Tag 是一个存在于你团队协作工具中的 Claude。我们上周在 Slack 内部发布了它。Claude Tag 的不同之处在于它默认是多人协作的。一旦你将 Claude Tag 添加到 Slack 频道，你可以参与，你的队友也可以参与，你们可以一起协作处理 PR。另一个重大区别是它是主动的，而不是被动的。你可以告诉 Claude Tag：“嘿，监控这个频道中的每一个 bug 报告，提出一个修复它的 PR，并标记最后修改该部分代码的工程师，”它会在该频道的整个生命周期中自动执行，而无需你手动标记。第三个重大变化是我们为它增加了团队记忆功能：https://claude.com/docs/claude-tag/users/memory。如果你在频道里告诉 Claude Tag 你的偏好，它会记住这些偏好，用于未来的每一条发布。如果你希望它始终调试故障，但不想调试警告，只需用自然语言在频道中告诉它，它就会记住这一点，为你和你团队的每个人保存。

从内部来看，我们认为 Claude Tag 是 Claude Code 的演进。我们认为这是我们内部工作方式的一次重大转变。目前，Claude Tag 处理了我们 65% 的产品工程 PR。

Simon：是针对整个 Anthropic，还是仅针对 Claude Code？

Cat：这仅针对我们的产品工程团队——我们的内部版本的 Claude Tag 目前处理了我们 65% 的产品 PR。这是一个巨大的变化；超过 50% 的 PR 都由它处理。我们看到人们在 Claude Code 和 Claude Tag 之间分配工作的方式是：当你需要与代理进行交互式迭代以处理最复杂的任务时，Claude Code 仍然是最好的选择。但 Claude Tag 非常适合主动代你工作，这样你就不需要为每个你所处理功能出现的 bug 报告手动启动 Claude Code。

Thariq：对于非编码场景呢：例如在这次访谈之前，我们问 Claude Tag，“嘿，Fable 什么时候发布？”我们想确保能与公告同步。Claude Tag 会搜索我们的 Slack 并查看谁在说什么。作为你们公司的搜索引擎，它非常有价值。它掌握了你产品的所有背景，因此你可以向它询问与指标相关的问题——通常当你做决策时，你希望决策受指标数据的指导，因此你可以将它连接到事件存储。我看到我们的营销团队会做类似的事情，比如，“嘿，告诉我这个功能。”他们不是程序员，但 Claude 是程序员——它可以克隆代码库并说，“这是这个功能，它长什么样，我使用这个功能的录屏。”它可以实现各种各样的功能，我认为我们在探索这一点上还处于早期阶段。

10:06：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=606s

Simon：我在使用编码代理时遇到的一个问题是，我清楚如何作为个人使用它们，但不太清楚如何在团队环境中使用它们。听起来 Claude Tag 是你们当前针对团队协作层的解决方案。

Cat：没错。而且我们现在实际上有很大一部分会议是多人参与的。也许我会说，“嘿，我觉得我们应该在Cowork中实现这个新功能，”然后我会标记Claude Tag来做第一次尝试。然后我会告诉Claude Tag，“分享你最终实现的录屏”，然后我会标记设计团队来查看。他们会进行调整，然后再交给工程团队完成并推向生产环境。这一直是一个非常流畅的体验。我们仍在尝试理清在同一会议中引导社交互动的动态，但我们发现人们只是观察其他人如何使用它并遵循那些社交规范 —— 将Claude Tag整合到我们的团队中对我们来说非常直观。

Thariq：它非常适合教人，也可以减少疏漏，因为每个人都在一起看你使用Claude，这也会提升你使用Claude的水平。

这让我想起了Midjourney是如何通过在其Discord频道中公开要求进行提示来解决教人高级图像提示的挑战的。

11:41：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=701s

我自己发现一件非常难的事情是，现在构建功能的成本已经下降得很低，要判断一个功能是否值得立即发布。

Simon：你如何处理工程中所有问题中最难的问题——优先级？当构建一个功能变得如此便宜时，你如何决定哪些功能值得构建和发布？

Cat：这是最难的部分。我们有几种方法来处理它。一个方法是我们每天都自己使用我们的产品。每当我们想在产品中实现某个功能却无法实现时，我们不会去找别的解决方案，而是修复我们的产品，使其能够支持这一情况。我们内部有非常强烈的自用文化。在将产品分享给全世界的人之前，我们会先分享给 Anthropic 所有员工，以及一些早期用户，他们会给我们非常诚实的反馈——越严格越好——然后我们不断迭代，直到大家喜欢它。我们内部对一个功能的活跃用户数和留存率有明确的门槛，只有达到门槛后才会对外发布。由于这个门槛非常清晰，每个工程师都知道自己需要达到的目标。我认为这也提升了我们产品的打磨程度，因为如果功能不够精细，用户就会流失——那么我们就不应该发布该功能。

使用内部的用户留存数据来决定一个功能是否发布，这对我来说很有道理。

12:54：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=774s

Simon：你能举一个让你感到惊讶的功能例子吗？你发布了它，而用户参与度超出了预期——原本不太可能被发布的功能却变成了真正的产品亮点。

Cat：我确实有一个。我们团队的很多人都喜欢远程控制：https://code.claude.com/docs/en/remote-control。远程控制允许你使用移动设备或浏览器中的 Claude，连接到在你的 CLI 本地运行的 Claude Code 会话。我自己从来没有这种需求，因为我直接在手机上启动任务，它会在云端会话中运行，不需要使用本地环境——我想可能是因为我做的都是很简单的编码任务。这是我不太理解的，我当时觉得，嘿，人们应该直接设置远程开发环境。但实际上，一旦我们推出了远程控制，我和很多人聊天时得知，他们每晚的做法是把笔记本电脑插上电源，打开多个远程控制会话，锁屏，然后坐在沙发上用手机来控制 Claude Code。所以这已成为我们现在依赖的一种流程，而我最初并不理解——但现在我理解了。

14:20：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=860s

会议的一个总体主题是审查：人们在审查由代码代理为他们编写的代码上花了多少注意力。我非常想听听Claude Code团队对此的看法！

Simon：代码审查是如何进行的？是否每一行进入Claude Code的生产代码都有人类审查？如果不是，你们是怎么做的——如何保持质量？

Thariq：这在很大程度上取决于任务的重要性。对于关键领域，我们有代码负责人。系统提示就是一个例子，我们在这里有一个代码负责人——你真的需要获得他们的批准。

Simon：所以代码负责人对该代码区域的质量负有直接责任。

Cat：而且他们需要批准涉及该区域的所有PR。

Thariq：我们有我们的代码审查GitHub机器人：https://code.claude.com/docs/en/github-actions 审查一切——这适用于每个PR，而且通常机器人会完成大部分审查。我在团队中看到的一个情况是，对于更复杂的PR，你可能会制作一个工件来解释PR，以便其他人可以进行审查。我们在验证、CI/CD等方面投入了大量精力，以确保每当出现问题时都有测试。我们有一个非常健全的环境，Claude可以控制Claude Code并进行测试。因此，代码审查采取多管齐下的方式。

所以关键似乎是不断迭代自动审查系统本身，从而随着时间建立对它们的信任。

17:20：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1040s

我们深入探讨了评估——这是整个大会中另一个热门话题。

Simon：我知道Opus 4.8，如果我让它为我构建一个运行SQL查询并输出JSON的JSON端点，它就会正确完成——这是我不必仔细审查的。但随后出现了一个新模型，我不知道如何快速建立对Fable的信任，确保它不会犯Opus没有犯的错误。新模型如何影响你对其能做什么和不能做什么的直觉？

Cat：我们之所以随着时间积累这个评估库，主要原因是为了让新模型可以直接替换旧模型。当我们有新模型时，我们会运行整个评估集，并确保例如 Fable 明显优于 Opus 4.8——这让我们有信心直接使用它。

Simon：这些模型评估是针对整个 Anthropic 的，还是只针对 Claude Code 团队的？

Cat：两者都有。我们有团队内部的评估，并且我们在 Anthropic 内的每个代码库都进行代码审查，所以我们也有这方面的评估。对于像自动模式这样的功能，我们不仅对 Anthropic 内的每个用户进行了评估——我们还委托了多个外部测试者进行红队测试，创建带有提示注入和恶意输入的环境，确保自动模式不会让这些输入通过。

18:41：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1121s

Simon：我想知道我对系统提示做的改进是否真正提升了产品——这是最基本的、针对产品的评估形式，而我仍然不太清楚该怎么做。你们是否在进行这类工作，以便完全确信对系统提示的调整能够产生更好的输出？

Cat：我们没有完全的信心，但我们做了很多工作来确保性能不会倒退。起点是一套我们信任的外部评估，并且我们用一套更大的内部评估来补充它，这套内部评估也是我们信任的。首先，我们主要优化能力：在给出任务的完整定义和完整代码库的情况下，Claude 是否做出正确决策、完全修复错误并通过所有测试？这是起点，也是我们优化的目标，因为它最直接地满足用户的需求。但有许多行为会影响用户在使用 Claude Code 时的感受。例如，人们真的不喜欢 Claude Code 说该去睡觉了。或者人们真的不喜欢它说：“嘿，我完成了五个部分中的两个——你想让我继续吗？”是的，请继续。所以我们正在建立一套行为评估来捕捉这些问题。随着用户反馈——请大胆向我们提供你的用户反馈——我们对优先级问题进行排序，并逐一构建每个问题的评估。这并不是 100% 覆盖，但提高覆盖率对我们来说是一个优先事项。

20:21：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1221s

Simon：Claude Code 团队与最初训练模型的 Anthropic 团队之间有多少互动？这是一个相当紧密的合作吗？

Cat：在 Anthropic 内部，我们都密切合作。我们经常开会讨论对下一代模型的期望。我们的研究团队在公开展示方面也非常出色——我们经常在博客文章中谈论如何针对越来越长远的工作进行优化，以及我们如何训练 Claude 本身诚信、无害且有帮助。我们还投入大量精力确保它符合您的意图，即使您的意图表达得很模糊。当然，尽量具体说明您的需求，这样 Claude 就能拥有所有上下文——但即使您不具体说明，我们也教 Claude 做出良好假设。这是一个富有成效的合作关系。

21:24：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1284s

这一部分有这么多有用的提示方法！

Simon: Thariq，你今天早上提到：https://www.youtube.com/watch?v=9fubhllmsBU&t=358s，Claude Code 的系统提示因为 Claude Fable 被减少了 80%。你能详细说明一下吗？你们可以去掉什么类型的内容？

Thariq: 不仅仅是 Fable —— 还有 Opus 4.8，以及未来的模型。我们现在针对不同的模型有不同的系统提示。我们看到的一个模式是，我们对 Claude 的限制过多了。最初的，比如 Opus 4 之类的模型需要很多示例，而去掉示例非常有帮助，因为它比我们给出的示例更有创造力。

Simon: 这真的很有意思，因为我给大家的顶级提示之一是：提供示例。如果这不再成立，那会有点打破我原来的提示模型。

Earlier this month I hosted a fireside chat session at the AI Engineer World’s Fair：https://www.ai.engineer/worldsfair/2026 with Cat Wu and Thariq Shihipar from Anthropic’s Claude Code team. We talked about Claude Code, Claude Tag, Fable, coding agent security, evals, tool design, and how Anthropic use these tools themselves.

The full video of the session is now available on YouTube：https://www.youtube.com/watch?v=uU5Gv2h8-9g. Below is an edited copy of the transcript, with extra links and my own bolded highlights.

A few top-level notes if you don’t want to watch the video or wade through the whole transcript:

1:05：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=65s

Simon: Claude Code came out in February of last year — it’s under a year and a half old, and it was originally just a bullet point on the Claude Sonnet 3.7 launch：https://www.anthropic.com/news/claude-3-7-sonnet. How has what you do on a day-to-day basis changed in the past year , now that we have these coding agents that actually work for us?

Cat: I remember when we first came out with Claude Code and Sonnet 3.7, you would give it a task and you would have to closely monitor every single little thing it tried to do. I would read every permission prompt extremely carefully. I would frequently say no — no, no, no, did you check this file? Did you check that file? And now it’s been incredible with every model generation. I feel like we’ve all gotten a chance to take a step back and delegate a lot more of the menial implementation to Claude . It’s freed up a lot of our time to think about more creative work, like: what is the right experience that we should be providing to our users, now that we know Claude Code can implement a lot of it? And now with Fable it’s a totally different step change improvement. We see for a lot of our use cases that you can actually one-shot a ton of features with Fable now .

Thariq: I remember the first text I got about Claude Code. One of my best friends was like, “You need to go try Claude Code.” It was about when Opus 4 came out, and I tried it and I was like, “Oh, shit. I need to work at Anthropic now.” And that was Opus 4 — great model, but you were reading permission prompts. It’s kind of crazy how much amnesia we have, where I’m like, oh, auto mode has always been here, right? I don’t even remember pressing yes and allow. For me, the big thing I’m trying to push myself on is that we have to do higher quality work than we’ve ever done before . The outputs are incredibly high quality. I’ve been using it to edit videos a bunch , and I’m like, okay, it has to meet the very exacting demands of our brand team in a couple of hours or we just can’t do it. That’s how I’m trying to shift with Fable: the best work we’ve ever done, faster than we’ve ever done it before .

3:39：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=219s

Simon: What’s a piece of conventional software engineering that was true a year ago that you don’t think holds anymore in this new world?

Cat: One of the biggest shifts we’re seeing in the eng skill set: two years ago it was pretty typical for a product manager to go talk to a bunch of customers, align over the course of six months with cross-functional teams on some PRD, and write a thorough spec on exactly how we’ll implement this before the first line of code gets written. Now things are completely turned the opposite way. For a lot of engineers, the push I would give to folks in the room is to develop more of your business sense and product sense on what it is we should build , because the timeline between having an idea and building it is so much shorter — it’s down from six to twelve months to maybe even a week. That means all of us need to have better taste on what is worth building, what will actually inflect the businesses we’re working on. So it’s an increase in value on product taste and business sense , and a bit lower on execution in most product domains. Of course, for infra there’s still a very heavy emphasis on making sure all the details are right.

Thariq: For me, it’s that rewrites are now good .

Simon: The worst thing you could do is now actually fine!

Thariq: Exactly. All the Mythical Man-Month stuff — never rewrite — I’m pro-rewriting now. If you have a good test suite — and I think the rewrite actually forces you to make sure you have a good test suite — but I think what people undercount is that a codebase is a spec, and maybe it’s the only copy of the spec that you have , because no one knows every branching part of the codebase. You can take this as an artifact and distill it or create other versions of it. We rewrote Bun in Rust：https://bun.com/blog/bun-in-rust and it works great — it’s live for me right now.

Simon: You’re not shipping Claude Code on Bun-in-Rust yet, right?

(Actually it looks like Anthropic started shipping Claude Code on Bun-in-Rust to everyone on June 17th：https://simonwillison.net/2026/Jul/19/claude-code-in-bun-in-rust/.)

6:36：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=396s

Simon: The other big launch recently was Claude Tag：https://www.anthropic.com/news/introducing-claude-tag — that’s what, a week old now, at least for the rest of us. I understand it’s being used at Anthropic by non-engineers a great deal. What kind of things are non-engineers doing with Claude Tag?

Cat: Claude Tag is a Claude that lives in your team’s collaboration tools. We launched it last week within Slack. The thing that’s different about Claude Tag is it’s multiplayer by default . Once you add Claude Tag to a Slack channel, you can chime in, your teammates can chime in, and you can collaborate together on the PR. The other big difference is that it’s proactive instead of reactive. You can tell Claude Tag, “Hey, monitor every bug report in this channel, put up a PR to fix it, and tag the engineer who last touched this part of the codebase,” and it’ll do it for the lifetime of the channel without you having to manually tag it in. And the third big shift is that we’ve added team memory：https://claude.com/docs/claude-tag/users/memory into this . If you tell Claude Tag your preferences in the channel, it’ll remember them for every future post. If you always want it to debug outages but you don’t want it to debug warnings, just tell it that in natural language in the channel and it’ll remember it for you and everyone else on your team.

Internally, we see Claude Tag as the evolution of Claude Code. We see this as a large shift in how we work internally. Claude Tag currently lands 65% of our product eng PRs.

Simon: For all of Anthropic, or just for Claude Code?

Cat: This is just for our product engineering team — our internal version of Claude Tag lands 65% of our product PRs right now . And this is a huge shift; this is more than 50% of our PRs. The way we see people split work between Claude Code and Claude Tag is: Claude Code is still the best place for your most complex tasks, when you’re interactively iterating with the agent. But Claude Tag is great for having it work proactively on your behalf , so you no longer need to manually kick off Claude Code for all the bug reports that come up for features you’re working on.

Thariq: And for non-coding cases: for example, before this talk we asked Claude Tag, “Hey, when is Fable releasing?” We wanted to make sure we’d line it up with the announcement. Claude Tag would search our Slack and look at who’s been saying what. As a search engine for your company, it’s really valuable. It has all the context for your product, so you can ask it metrics-related questions — often when you’re making decisions you want them informed by what the metrics say, so you hook it up to your event store. I’ve seen our marketing team do things like, "Hey, tell me about this feature." They’re not programmers, but Claude is a programmer — it can clone the codebase and say, "This is the feature, this is what it looks like, this is a recording of me using the feature ." It enables a whole wide variety of things, and I think we’re still early in figuring that out.

10:06：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=606s

Simon: One of the problems I’ve had with coding agents is that I get how to use them as an individual, but I’m not really clear on how to use them in a team environment. It sounds like Claude Tag is your current answer to that team collaborative layer for this stuff.

Cat: Exactly. And a large percentage of our sessions are actually multiplayer right now. Maybe I say, “Hey, I think we should implement this new feature in Cowork,” and I’ll tag in Claude Tag to do a first pass at it. Then I’ll tell Claude Tag, “Share a recording of your final implementation,” and I’ll tag in design to take a look. They’ll nudge it, then pass it on to eng to take it to the finish line and get it out to prod. It’s been this very fluid experience. We’re still trying to iron out what the social dynamics are for steering the same session , but we’ve found that people just observe how others use it and follow those social norms — it’s been pretty intuitive for us to integrate Claude Tag into our teams.

Thariq: It’s great for teaching people, and also for reducing slop, because the fact that everyone is seeing you use Claude together sort of levels up how you use Claude as well .

This reminded me of how Midjourney solved the challenge of teaching people advanced image prompting by enforcing prompting in public in their Discord channels.

11:41：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=701s

Something I’ve found really hard myself is knowing when a feature is worth shipping now that the cost of actually building features has dropped so much.

Simon: How do you deal with the hardest problem in all of engineering — prioritization? How do you decide which features are worth building and shipping when building a feature is so much more inexpensive now?

Cat: This is the hard thing. There are a few ways we approach it. One is we dogfood our products every single day. Whenever there’s something we want to be able to do in our products that we’re not able to, instead of finding a different solution we fix our product so it can support that case. We have a very heavy dogfooding culture internally. Before we share our products with everyone in the world, we share them with everyone within Anthropic, and with some early customers who give us very honest feedback about it — the more brutal the better — and we iterate until people love it. We have an internal bar for the number of active users and the amount of retention a feature has to have before we share it with the world. Because this bar is very clear, every engineer knows what they’re trying to hit. I think this also levels up our polish, because if the feature isn’t polished, people will churn — and then we shouldn’t ship that feature.

Using internal user-retention to decide if a feature should ship makes a whole lot of sense to me.

12:54：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=774s

Simon: Do you have an example of a feature which surprised you? You rolled it out and the engagement was off the charts — something unlikely to be shipped that turned into a real product thing.

Cat: I do have one. A lot of folks on our team love remote control：https://code.claude.com/docs/en/remote-control. Remote control lets you use your mobile device, or Claude in the web browser, to connect to a local Claude Code session running in your CLI. I never have this need, because I just kick off the task directly on mobile and it runs in a cloud session without using my local environment — I think because I’m doing very easy coding tasks. It was something I didn’t totally understand; I was like, hey, people should just set up remote dev environments. But in practice, once we rolled out remote control, so many people I talk to told me that what they do every night is plug their laptop into a power charger, open a bunch of remote control sessions, lock the screen, and then use their mobile phone from their couch to control Claude Code . So this has become a flow we’re now leaning into that I didn’t originally get — but now I do.

14:20：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=860s

One of the over-arching themes of the conference was review: how much attention to people spend to reviewing code written for them by coding agents. I was very keen to hear the Claude Code team’s take on this!

Simon: How does code review work? Does a human being review every line of production code that makes it into Claude Code? And if not, what are you doing — how do you keep the quality up?

Thariq: It varies on the task a lot. For important areas we have code owners. The system prompt is an example where we have a code owner — you really need to get their approval.

Simon: So the code owner is directly responsible for the quality of that area of the code.

Cat: And they need to approve any PR that touches it.

Thariq: We have our code review GitHub bot：https://code.claude.com/docs/en/github-actions review everything — that goes on every PR, and often it’s doing the bulk of the review. Something I’ve seen on the team is that for more complex PRs you might make an artifact to explain the PR so that other people can then review. And we invest a lot into verification, CI/CD, things like that, to make sure that any time anything fails we have a test. We have a really robust environment where Claude can control Claude Code and test it. So there’s a multi-pronged approach to code review.

So the key seems to be constantly iterating on the automated review systems themselves, in order to build trust in them over time.

17:20：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1040s

We got deep into evals—another hot topic throughout the wider conference.

Simon: I know that Opus 4.8, if I ask it to build me a JSON endpoint that runs a SQL query and outputs JSON, is just going to get it right — that’s not something I have to review closely. But then a new model comes along and I don’t know how to build trust in Fable quickly, that it’s not going to mess things up that Opus didn’t. How does the new model affect your intuition for what it can do and what it can’t do?

Cat: The main reason we’re building up this eval base over time is so that new models can be a drop-in replacement . When we have a new model, we run the whole eval set and make sure that, for example, Fable is strictly better than Opus 4.8 — and that gives us the confidence to drop it in.

Simon: Are those model evals for Anthropic as a whole, or Claude Code team-specific?

Cat: We have both. We have evals on our team, and we run code review across every repo within Anthropic, so we have evals for that. And for things like auto mode, we not only have evals across every user within Anthropic — we’ve also commissioned multiple external testers to red team it, to create environments with prompt injections and malicious inputs, and make sure that auto mode doesn’t let any of those pass .

18:41：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1121s

Simon: I want to know if the system prompt improvement I made actually improved the product — that’s the most basic form of product-specific eval, and I still don’t have a great feel for how to do that. Is that something you’re doing such that you have complete confidence that a tweak you’ve made to the system prompt results in better output?

Cat: We don’t have complete confidence, but we do a lot to make sure that we don’t regress performance. The starting point is a suite of external evals that we trust, and we complement that with an even larger suite of internal evals that we trust. To start, we mainly optimize for capability : given a complete definition of a task and the full codebase, does Claude make the right decisions, fully fix the bugs, and pass all the tests? That’s the starting point and the thing we optimize for, because it’s most directly what users want. But there are a lot of behaviors that impact how users feel when they work with Claude Code. For example, people really don’t like it when Claude Code says it’s time to go to sleep. Or people really don’t like it when it says, “Hey, I finished two out of five parts — do you want me to continue?” Yes, please continue. So we’re building up a set of behavioral evals to catch these. And as we get user feedback — please be loud with us about your user feedback — we rank the priority issues and go down one by one and build evals for each of them. It’s not 100% coverage, but it is a priority for us to increase the coverage.

20:21：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1221s

Simon: How much interaction is there between the Claude Code team and the teams at Anthropic who are training the models in the first place? Is that quite a close collaboration?

Cat: Across Anthropic, we all work quite closely together. We meet often to talk about what we expect the next generation of models to be able to do. Our research team has also been amazing about showing this publicly — we often talk in our blog posts about how we’re targeting ever-increasing longer-horizon work , and how we train Claude itself to be honest, harmless, and helpful. We also put a lot of effort into making sure it’s aligned with your intent, even if your intent is expressed in a fuzzy way. Of course, try your best to be specific about what you want, so Claude has all the context — but even when you’re not specific, we teach Claude to make good assumptions. It’s been a productive partnership.

21:24：https://www.youtube.com/watch?v=uU5Gv2h8-9g&t=1284s

So many useful prompting tips in this section!

Simon: Thariq, you mentioned this morning：https://www.youtube.com/watch?v=9fubhllmsBU&t=358s that the system prompt for Claude Code has been reduced by 80% because of Claude Fable . Can you go into a little more detail? What kind of things have you been able to drop?

Thariq: It wasn’t just Fable — it was Opus 4.8 as well, and going forward, future models. We have different system prompts for different models now. One of the patterns we saw is that we were over-constraining Claude. The initial, maybe Opus 4-ish models wanted a lot of examples, and removing examples was extremely helpful , because it was just more creative than the examples we gave it.

Simon: That’s really interesting, because one of the top prompting tips I give people is: give it examples. If that’s no longer true, that kind of breaks my prompting model a little bit.