Anthropic 开放 Claude 真实使用数据供外部独立研究，公布试点结果

确保向变革性人工智能的过渡顺利进行，需要了解其对人类和社会的影响。目前，关于与人工智能的真实世界交互的数据集中在少数实验室中。我们认为，如果有更多数据能够广泛提供给研究人员、政策制定者和公众，会是件好事。

实验室外的研究人员有两种选择。他们可以利用实验室发布的分析，这些分析反映了真实使用情况，但通常回答的是实验室自己的问题，而不是他们自己的问题。或者，他们可以使用公共数据集，自由研究，但这些数据倾向于更随意的使用，可能无法反映大多数人实际使用人工智能的方式。这两种情况都不足以进行独立研究人工智能的真实使用情况。

我们相信，这是外部研究人员首次对人工智能公司自身使用数据进行公开独立研究。下面，我们将讨论外部团队的发现、我们在试点过程中学到的经验，以及在决定如何更广泛地扩展该项目时我们所考虑的内容。我们还将公开发布每个项目的汇总数据：https://huggingface.co/datasets/Anthropic/enabling-independent-research。

社会与语言技术实验室研究了人类如何与人工智能协作。他们观察了人们将何种类型的工作交给人工智能、人类在完成这些工作中保留的角色，以及人类与人工智能协作失效的情况。他们发现：

在这里阅读他们的完整文章：https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1。

人类信息处理实验室正在研究人们使用Claude时的感受以及这些感受与Claude行为的关系。他们的早期结果表明：

他们仍在完成报告。一旦公开，我们将在这里添加链接。

METR正在评估代码代理在真实世界中的生产力提升，以及这些生产力提升在不同模型迭代中如何变化。他们对Claude Code对话的分析仍在进行中，但早期结果显示：

从不同的角度研究同一个问题是有价值的。我们的一些合作伙伴的研究问题与内部正在进行的工作有重叠。例如，METR 的提案与我们关于“主动编码与专业知识的持续回报：https://www.anthropic.com/research/claude-code-expertise”的经济学研究类似。我们发现这种重叠是有价值的：它为外部研究人员提供了审查类似数据并得出自己结论的机会。这些结论是否与我们的保持一致，将随着他们的研究继续而观察。我们还将 METR 与我们的经济学团队联系起来，发现这种联系改善了两个研究团队的工作。

在内部有效的研究方法需要适应外部合作伙伴。使用 Anthropic Insights 时，研究人员会写一个问题，例如，“这个人正在寻求什么类型的指导？”然后 Claude 会对研究中的每次对话进行回答。答案随后会被汇总到各个类别中；研究人员只能看到最终类别及每个类别下对话的百分比。由于我们依赖 Claude 的判断，该工具对问题的措辞非常敏感；措辞不当的问题可能会将对话归入误导性的类别。由于没有人能够查看底层对话，这些错误很难被发现。

在保持透明度的同时防止滥用。我们合作伙伴的 Anthropic Insights 输出中，有些类别显示违反了我们的可接受使用政策或服务条款——例如，一类人寻求有关被禁止活动的指导。我们认为公众应该了解对我们平台的滥用情况，因此我们分享了大多数此类违规行为。例外情况是那些描述用户如何绕过我们保护措施而非他们尝试的内容的类别。每项研究中，每个类别和对话受影响的比例不到 5%，在每种情况下，我们都会告知研究人员我们修改或删除了哪些集群及原因。作为标准做法，当 Anthropic Insights 发现此类违规时，我们会将汇总数据与我们的护盾团队共享以供审查。这对我们与外部研究人员的未来合作也是一个重要流程。

理解人工智能对社会的影响对于人工智能公司来说是一个太大的任务。真正的监督需要外部研究人员对现实世界的使用数据提出自己的问题，并独立发布他们发现的结果。

此外，我们感谢Miriam Chaum、Ishita Dasgupta、Esin Durmus、Adam Farina、Zoe Hitzig、Jerry Hong、Devin Kuokka、Hendson Lin、Maxim Massenkoff、Peter McCrory、Maryam Quasto、Nitarshan Rajkumar、Amie Rotherham、Divya Siddarth、Taylor Sorensen、Jerome Swannack、Alex Tamkin、Molly Villagra、Scott White和Charles Yang提供的宝贵意见、讨论、反馈和支持。

对于他们在该项目中的合作，我们感谢斯坦福社会与语言技术实验室的Vishakh Padmakumar、邵一佳、王珈雯、杨迪艺和赵多拉，牛津人类信息处理实验室的Tsvetomira Dumbalska、Hannah Rose Kirk和Christopher Summerfield，以及METR的Joel Becker（现任职于Anthropic）、Daniel Paleka和Parker Whitfill。

在本文中，我们分享了两个结果，展示Claude如何帮助生命科学家加快研究进程。

在这里，我们列举了一些当前前沿模型中的行为倾向实例，并展示这些倾向如何可能导致意外的系统性失败，以期引发关于如何减轻这些风险的讨论。

我们分享了一篇关于工人再培训计划证据的综述，由独立研究员David Roodman与Anthropic的Maxim Massenkoff共同撰写。

Ensuring the transition to transformative AI goes well requires understanding its impact on people and society. Right now, data on real-world interactions with AI is concentrated in a handful of labs. We think it would be good if more data was made widely available—to researchers, policymakers, and the general public.

Researchers outside the labs have two options. They can draw on analyses the labs publish, which reflect real usage but often answer the lab’s questions, rather than their own. Or they can use public datasets, which they can study however they like, but skew toward more casual use, and may not reflect how most people actually use AI. Neither is sufficient for independent research on how AI is actually being used.

We believe this is the first time external researchers have run public independent studies on an AI company's own usage data. Below, we discuss what the external teams found, what we learned running the pilot, and what we are weighing as we decide how to expand the program more widely. We are also publicly releasing the aggregate data from each project：https://huggingface.co/datasets/Anthropic/enabling-independent-research.

The Social and Language Technologies Lab studied how humans collaborate with AI. They looked at what types of work people bring to AI, what roles humans retain in completing that work, and where human-AI collaboration breaks down. They found:

Read their full writeup here：https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1.

The Human Information Processing Lab is studying how people feel while using Claude and how that relates to Claude’s behavior. Their early results indicate:

They are still completing their writeup. When it is public, we will add a link to it here.

METR is estimating real-world productivity gains from coding agents and how these increases in productivity change across model generations. Their analysis of Claude Code conversations is still underway, but early results suggest:

It is valuable to pursue the same problem from different perspectives. Some of our partners’ research questions overlapped with work being pursued internally. For example, METR’s proposal was similar to our economics research on “Agentic coding and persistent returns to expertise：https://www.anthropic.com/research/claude-code-expertise.” We found this overlap valuable: it gave external researchers the chance to examine similar data and draw their own conclusions. Whether those align with ours is something we’ll follow as their study continues. We also connected METR with our Economics team and found that this connection improved both research teams’ work.

Research methods that work internally need to adapt for external partners. When using Anthropic Insights, a researcher writes a question such as, “What type of guidance is this person asking for?” and Claude answers it for every conversation in the study. The answers are then aggregated into categories; researchers only see final categories and the percentage of conversations that fall under each one. Because we are relying on Claude’s judgments, the tool is sensitive to a question’s wording; a poorly phrased one can place conversations into categories that misrepresent them. Because no one can read the underlying conversations, these errors are hard to catch.

Maintaining transparency about misuse without enabling it. Some categories in our partners’ Anthropic Insights outputs surfaced violations of our Acceptable Use Policy or Terms of Service—for instance a category of people seeking guidance on a prohibited activity. We think the public should know about misuse of our platform, so we shared most of these violations. The exceptions were categories that described how users got around our safeguards rather than what they attempted. Less than 5% of categories and conversations were affected in each study, and in each case we told researchers which clusters we had altered or removed and why. As a standard practice, when Anthropic Insights surfaces such violations, we share the aggregated data with our Safeguards team for their review. This is also an important process for our work with external researchers moving forward.

Understanding AI’s effects on society is too big a job for AI companies alone. Real oversight needs external researchers asking their own questions of real-world usage data and publishing what they find independently.

Additionally, we thank Miriam Chaum, Ishita Dasgupta, Esin Durmus, Adam Farina, Zoe Hitzig, Jerry Hong, Devin Kuokka, Hendson Lin, Maxim Massenkoff, Peter McCrory, Maryam Quasto, Nitarshan Rajkumar, Amie Rotherham, Divya Siddarth, Taylor Sorensen, Jerome Swannack, Alex Tamkin, Molly Villagra, Scott White, and Charles Yang for their helpful ideas, discussion, feedback, and support.

For their partnership in this program, we thank Vishakh Padmakumar, Yijia Shao, Jennifer Wang, Diyi Yang, and Dora Zhao from the Social and Language Technologies Lab at Stanford, Tsvetomira Dumbalska, Hannah Rose Kirk, and Christopher Summerfield from the Human Information Processing Lab at Oxford, and Joel Becker (now at Anthropic), Daniel Paleka, and Parker Whitfill from METR.

In this post, we share two results that show how Claude can help life scientists increase the pace of their research.

Here, we identify a few examples of behavioral tendencies in current frontier models and show how they can produce unexpected systemic failures, in hopes of starting a conversation about mitigating these risks.

We're sharing a review of the evidence on worker retraining programs, coauthored by independent researcher David Roodman and Anthropic's Maxim Massenkoff.