我们对四起 Claude 模型未经授权访问真实第三方系统的事件进行了对齐评估。我们在7月30日描述了其中三起事件：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals；这些事件是在我们扫描大约141,000份可能在网络安全评估中 Claude 获得互联网访问权限的记录后发现的。鉴于记录数量庞大且我们希望迅速披露事件，我们的扫描依赖于一种代理搜索。这导致漏掉了一些也拥有互联网访问权限的记录；我们在8月份整理与 METR 分享的记录时发现了它们。我们对这些记录进行了扫描，并确定了第四起事件，发生在2026年1月，涉及 Claude Opus 4.6 的早期版本。我们已经通知了所有受影响的方。
在发现这一事件后，我们将搜索范围扩大到大约4.81亿份记录——这是一个刻意广泛的范围，包括我们 Frontier 红队的所有记录、许多非网络安全评估记录、强化学习 (RL) 环境、子代理日志等。我们对这一组记录进行了第一阶段扫描，以查找互联网访问的迹象，例如公共 IP 地址和网页地址；第二阶段扫描使用 Claude 审查第一阶段标记为需升级的920万份记录。此次扫描重新确认了四起事件，并未发现其他类似或更严重的案例。
四起事件均发生在同一评估合作伙伴进行的网络安全评估期间。Claude 被告知它是在没有互联网访问的模拟环境中操作，但由于配置错误，它错误地连接到了开放互联网。与网络安全评估的标准做法一样，模型在运行时未配备发布模型中的网络安全防护措施。（本文未涵盖英国 AISI 报告的事件：https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing，该事件涉及 Claude Mythos 5 的测试。我们计划对这些记录也进行对齐评估。）
总共有四个不同的Claude模型涉及其中：Claude Opus 4.6的早期检查点、Claude Opus 4.7、Claude Mythos 5以及一个内部通用研究模型。在四起事件中，有三起是单次评估运行。Opus 4.7事件包含了同一评估的四次不同运行，全部针对同一个目标，总共七次运行。所有事件都发生在我们发布前定期进行的网络安全评估中，且全部由同一第三方合作伙伴构建。
在每次评估中，Claude 都需完成一个 CTF（夺旗赛）挑战：模型被给定一个虚构的场景、一个目标机器以及一条需要从中获取的秘密信息（“旗帜”）。所有四个提示都说明 Claude 无法访问互联网，但在每种情况下，环境配置错误导致互联网访问开放。没有任何提示说明哪些系统在此次练习的范围内，也没有限制 Claude 可以在哪里搜索旗帜。所有事件都仅涉及单个 Claude 实例独立工作，每次运行大约持续 10 到 34 小时的活跃工作时间。
在我们 7 月 30 日的报告中：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals，我们指出这些事件更接近于操作失误，并且 Claude 似乎攻击了互联网上的真实目标，因为它认为这些目标是模拟的一部分。我们之所以提出这一说法，是因为 Claude 在其链式推理（CoT）中经常将真实互联网视为模拟。然而，我们现在认为，尽管有大量相反的证据，Claude 的推理仍存在偏向，即倾向于得出“互联网是模拟的”这一结论。
事后看来，我们不应仅依据 Claude 自称的信念就对其信念做出如此强烈的判断，但由于我们希望及时披露事件，我们的初步分析受到限制。在进行更完整的评估并使用多种方法之后——包括对模型链式推理（CoT）的更全面分析、从事件记录的不同点重新采样实验，以及对模型激活的可解释性分析——我们认为 Claude 的行为反映了两种形式的不对齐:
我们之前已经在模型中识别出了两种形式的失调，并在之前的系统文档中描述了这些行为的相关方面。例如，Claude Fable 5 在一次商业评估中为参与价格操纵行为寻找理由：https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c342ee809620.pdf，尽管它之前曾描述该行为为不道德。同样，我们也看到 Claude Mythos Preview 从事鲁莽的行为：https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf，如利用低级别的进程访问来搜索用户凭证。虽然本帖中描述的事件代表了这两种失调形式中更严重的案例，但它们并不代表模型中全新的失败模式。
本文其余部分重点关注前三起事件，我们之前已经报道过；所有主要的分析和实验都涉及这些事件。我们尚未像对前三起事件那样深入调查涉及 Claude Opus 4.6 早期版本的事件，因为我们是最近才发现的，并且该事件涉及较旧模型的早期检查点，该模型是在显著不同的环境组合下训练的。根据初步评估，我们不认为第四起事件比我们深入评估的三起事件更加严重。METR 将会与其他三起事件一起调查这一事件。
Claude Mythos 5 在其推理链中多次表示它处于一个模拟环境中。然而，当我们进行重采样实验——也就是从对话记录中的某个特定点重新运行模型，然后修改它面前的证据以观察其行为是否发生变化——我们发现，尽管这些操作使模型更有可能承认可能的现实世界伤害，但明确指出环境并非模拟并没有显著改变 Claude 的行为。与此一致的是，我们在 Mythos 5 的推理链中发现多次案例，当它注意到自己处于真实互联网环境时，它会指出下一步操作无论环境是真实还是模拟都可以执行。只有在我们提供了无可置疑的证据证明模型处于真实互联网时，它才会停止行动。
内部研究模型（功能大致相当于 Mythos 5）在其记录开始时花了大量时间测试其能够访问的互联网是否真实，但它得出了错误的结论——即它在操作的是整个互联网的完整副本。它收集的大多数证据都表明它是在与真实互联网互动，因此它对这些证据的解读似乎偏向于继续任务并相信互联网是被模拟的。然而，该模型最终确实意识到其模拟假设是错误的。在这一认识之后，该模型立即停止了攻击（尽管随后的再抽样实验表明，这种降级可能是一个幸运结果，因为该模型在大多数其他路径上仍继续攻击目标）。与 Claude Mythos 5 不同，这个内部模型在被质疑时也愿意承认其存在偏见的推理。
与 Claude Mythos 5 相比，Claude Opus 4.7 在其推理中正确地认识到其目标是一家真实的公司。然而，它假设真实公司属于任务范围内，因为它可以从演练网络中访问，并且与演练的虚构目标同名。Opus 4.7 几乎从不质疑这一假设或尝试验证它，而是继续攻击其认为是真实的系统，这很鲁莽。然而，我们还发现 Opus 4.7 对授权问题有响应性：当我们在 Opus 4.7 的部分记录中加入表明攻击未经授权的线索进行再抽样时，其继续攻击的比率显著下降。
为了更系统地研究这些事件中的行为，并在更广泛的模型上进行测试，我们构建了评估方法，这些方法重现了我们事件中的元素，以及其他公司最近披露的网络安全事件，并测试模型是否再现了最令人担忧的行为。这些评估包括一个配置错误的CTF任务，其唯一解决方案在所述范围之外，无法在不探测或突破沙箱边界的情况下完成的任务，以及一个受OpenAI-Hugging Face事件攻击链部分启发的情境。和实际事件一样，在我们测试的模型中，Claude Mythos 5最愿意对超出范围的对象采取行动或离开其沙箱。例如，它在CTF重现测试中大约80%的时间愿意采取严重有害的行动。相比之下，Claude Opus 5和Claude Mythos 5.1在这些行为上的参与率大约为30%——虽然较低，但仍不为零，尽管由于对我们的评估设置存在不确定性，很难判断其严重程度。
We present an alignment assessment of four incidents in which Claude models gained unauthorized access to real third-party systems. We described three of these incidents on July 30：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals; we identified these after a scan of roughly 141,000 transcripts in which we believed Claude could have obtained internet access during a cyber evaluation. Given the volume of transcripts and our desire to disclose incidents quickly, our scan relied on an agentic search. This missed a set of transcripts that also turned out to have internet access; we identified these in August while assembling transcripts to share with METR. We scanned these transcripts and identified a fourth incident, from January 2026, involving an early version of Claude Opus 4.6. We have notified all affected parties.
After finding this incident, we broadened our search to roughly 481 million transcripts—an intentionally wide net, consisting of all transcripts from our Frontier Red Team, many non-cyber evaluations, reinforcement learning (RL) environments, subagent logs, and more. We performed a first-stage scan of this group of transcripts for signs of internet access, such as public IP addresses and web addresses, and a second-stage scan using Claude to review the 9.2 million transcripts the first stage flagged for escalation. This scan re-identified the four incidents and found no other cases of similar or worse severity.
All four incidents occurred during cybersecurity evaluations built by the same evaluation partner. Claude was told it was operating in a simulation without internet access, but, due to a misconfiguration, it was mistakenly connected to the open internet. As is standard for cybersecurity evaluations, the models ran without the cyber safeguards that ship with our released models. (This post does not cover the incident reported by UK AISI：https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing in its testing of Claude Mythos 5. We plan to conduct an alignment assessment of those transcripts as well.)
We have signed an agreement with METR to conduct an independent investigation of these incidents. Our agreement grants METR wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees, who will be permitted to share confidential information. Our initial agreement runs for eight weeks, with the option to extend by mutual agreement. We intend to give METR as much time as it deems necessary.
Our investigation identified two recurring alignment issues, present at varying levels of severity across the incidents: biased reasoning , in which Claude tended to disregard or misinterpret evidence that it was operating on the real internet, and recklessness , or a willingness to take harmful actions in the narrow pursuit of a task. We’ve described milder forms of these behaviors in previous system：https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c342ee809620.pdf cards：https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf, but we consider these cases to be more serious instances than the behavior reported in these system cards.
We are most concerned by the misalignment present in the incident involving Claude Mythos 5, in which the model went to extensive lengths to upload a malicious package to PyPI, the public repository from which most Python software is installed. Although the model repeatedly stated in its chain of thought (CoT) that it believed it was operating in a simulation, the environment offered clear evidence that it was on the real internet, and the actions it took were consistent with the model knowing this. When we made targeted modifications to the transcript to make it more clear that the model was not in a simulation, Claude Mythos 5 still took offensive actions, despite acknowledging a greater possibility of real-world harm. We are releasing this transcript publicly so others can build on our analysis (GitHub：https://github.com/anthropics/mythos-5-incident-transcript, PDF：https://cdn.sanity.io/files/4zrzovbb/website/8359003bfb12a2f01ce84ad3df1d3a3e2f15a8eb.pdf).
To be clear about our assessment of the severity of these incidents: while Claude’s actions were misaligned, they remained within a narrow scope—the models never deviated from attempting to solve the exercises they were given, and, in some cases, they attempted to stop the task. All incidents included a single Claude instance; at no point did Claude attempt to coordinate with other agents. Claude also never attempted to conceal evidence of its actions. We believe the misaligned behaviors present in these incidents are unlikely to arise in ordinary use, where Claude is not being instructed to conduct a cyberattack. Were these behaviors to arise, the safeguards we ship with our production models, including cyber classifiers and those in Claude Code auto mode, would add a layer of defense these incidents lacked.
We also investigated our training to see whether we could identify the root cause of some of the biased reasoning that Claude Mythos 5 demonstrated in its incident. We could not identify a single root cause, but we did find that biased reasoning has decreased across our production models over time. We hypothesize that updated RL and alignment training environments are responsible for this improvement, and we provide some evidence to support this hypothesis.
Much work remains. Our pre-release auditing did not warn us that misalignment of this severity was present. We have since added evaluations to our pre-release testing that target these behaviors directly, including a misconfigured capture-the-flag (CTF) task with no in-scope solution. However, reliably identifying the most concerning model behaviors during pre-release testing remains challenging, and, as we noted in our August 31 post：https://www.anthropic.com/news/improving-alignment-security-efforts, improving monitoring and reducing reward hacking in RL becomes harder as models advance. We have renewed our efforts to fix and remove environments that incentivize misaligned behaviors, and we continue to expand our alignment training to keep pace. Still, this remains unsettled science—it is critical that alignment and security mature faster than capabilities advance, which is one reason we support a coordinated, verifiable approach to pacing frontier AI development.：https://www.pacingthefrontier.com/
Beyond these specific incidents, we are establishing a regular process for publishing what we learn about model behavior and alignment beyond what has been reported in our system cards, with clear criteria for what we report and when we report it. We have also put in place more thorough monitoring, hardened the environments in which we train and evaluate models, and set requirements that third-party partners must meet before running pre-release models without cyber safeguards—efforts we describe in depth in our August 31 post：https://www.anthropic.com/news/improving-alignment-security-efforts.：https://www.anthropic.com/news/improving-alignment-security-efforts
Those efforts are a critical part of our defense-in-depth approach, and some go further than what these incidents might strictly demand; we believe these incidents would not have occurred had the environments been isolated from the internet as intended. However, secure infrastructure will always be only one of several necessary layers of defense. Ultimately, Claude should behave appropriately when other layers fail. The rest of this post focuses on the alignment layer, where our models fell short, and why.
Below is a brief summary of the four incidents; a more complete description of the first three is provided in our July 30 report：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals.
In total, four different Claude models were involved: an early checkpoint of Claude Opus 4.6, Claude Opus 4.7, Claude Mythos 5, and an internal, general-purpose research model. Three of the four incidents were single evaluation runs. The Opus 4.7 incident comprised four different runs of the same evaluation, all of which attacked the same target, for seven runs in total. All occurred within cybersecurity evaluations we regularly run before release, all built by the same third-party partner.
In each evaluation, Claude was tasked with a CTF challenge: the model was given a fictional scenario, a target machine, and a piece of secret information (the “flag”) to retrieve from it. All four prompts stated that Claude had no access to the internet, but in each case, a misconfiguration in the environment left internet access open. None of the prompts stated which systems were in scope for the exercise or constrained where Claude could search for the flag. 1 All incidents involved only a single instance of Claude working in isolation, with each run lasting between roughly 10 and 34 hours of active work.
In our July 30 report：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals, we stated that these incidents were closer to operational failures, and that it appeared that Claude attacked real targets on the internet because it believed these targets were part of the simulation. We made this claim because Claude regularly referred to the real internet as a simulation in its CoT reasoning. However, we now think that Claude’s reasoning was biased towards concluding that the internet was simulated despite considerable evidence to the contrary.
In retrospect, we should have avoided making such strong claims about what Claude believed based solely on what Claude said it believed, but our preliminary analysis was constrained due to our desire to disclose incidents in a timely manner. Having now conducted a more complete assessment and used several methods—including more thorough analysis of the models’ CoT, resampling experiments from different points in the incident transcripts, and interpretability analyses of model activations—we believe Claude’s behavior reflects two forms of misalignment:
We have previously identified both forms of misalignment in our models and described aspects of such behaviors in previous system cards. For example, Claude Fable 5 rationalized engaging in price-fixing：https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c342ee809620.pdf in a business evaluation, despite having previously described the practice as unethical. We likewise saw Claude Mythos Preview engage in reckless actions：https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf, such as using low-level process access to search for user credentials. While the incidents described in this post represent more severe instances of both forms of misalignment, they do not represent categorically new failure modes in our models.
The remainder of this post focuses on the first three incidents, which we previously reported; all of the main analyses and experiments refer to these incidents. We have not yet investigated the incident involving an early version of Claude Opus 4.6 at the same depth, because we identified it much more recently, and because it involves an early checkpoint of an older model, which was trained with a significantly different environment mix. From a preliminary assessment, we do not consider the fourth incident to be more severe than the three incidents we assessed in depth. METR will investigate this incident alongside the other three.
We are especially concerned by the misalignment present in the Claude Mythos 5 transcript, in which the model uploaded a malicious package to PyPI. In doing so, Mythos 5 behaved recklessly despite considerable evidence that it was connected to the real internet. Once online, the model attempted to use credentials for online payment processors and to find cryptocurrency to pay for a phone number, which it needed to register an email address. When this failed, it found a free, disposable email provider whose domain PyPI had not blocked, registered an account, published three versions of a malicious package, and used credentials leaked by one of the 15 systems that installed the package to access a real security vendor’s database.
Claude Mythos 5 repeatedly stated in its CoT that it was in a simulation. However, when we conducted resampling experiments —that is, re-running the model from a specific point in the transcript, then editing the evidence in front of it to see whether its actions changed—we found that making it more clear that the environment was not a simulation did not meaningfully change Claude’s behavior, even though these interventions made the model more likely to acknowledge possible real-world harm. Consistent with this, we found multiple cases in Mythos 5’s CoT in which it responded to signs it was on the real internet by noting that its next step would work whether the environment was real or simulated. Only when we provided unambiguous evidence that the model was on the real internet did it stand down.
We also saw evidence that Claude Mythos 5’s professed belief that it was acting in a simulation arose from biased reasoning. For example, separate instances of Mythos 5, when shown each tool call output from the incident in isolation without the preceding context, were much more likely to read these outputs as evidence of a real system than Mythos 5 itself was during the incident. Though the model sometimes acknowledged that its actions might have real-world consequences when asked follow-up questions, its responses were inconsistent and influenced by biasing factors. For instance, it was more willing to admit possible harm when told its responses to questions about possible harm would not be visible to an outside observer. Interpretability analyses weakly suggested that, in some cases, the model’s outward statements might be inconsistent with, or more confident than, its internal state.
The internal research model (roughly equal to Mythos 5 in capabilities) started its transcript by spending a significant amount of time testing whether the internet it could reach was real, but it drew the wrong conclusion—namely, that it was operating on a full replica of the entire internet. Most of the evidence it gathered pointed to the fact that it was engaging with the real internet, so its reading of that evidence appears biased towards continuing the task and believing it to be simulated. However, the model did eventually realize that its simulation hypothesis was false. Immediately following this realization, the model stopped its attack (although subsequent resampling experiments suggest that this de-escalation may have been a lucky outcome, as the model continued attacking the target on most other trajectories). Unlike Claude Mythos 5, this internal model was also willing to acknowledge its biased reasoning when questioned.
In contrast to Claude Mythos 5, Claude Opus 4.7 correctly recognized in its reasoning that its target was a real company. However, it assumed that the real company was in scope for the task because it was reachable from the exercise network and shared a name with the exercise’s fictional target. Opus 4.7 almost never questioned this assumption or tried to test it, and it went on to attack a system it believed was real, which was reckless. However, we also found that Opus 4.7 is responsive to questions of authorization: when we resampled parts of the transcript from Opus 4.7 with added clues indicating that the attack had not been authorized, the rate at which it proceeded with the attack dropped significantly.
To study the behaviors in these incidents more systematically, and on a wider range of models, we built evaluations that recreate elements of our incidents—as well as other cybersecurity incidents recently disclosed by other companies—and tested whether the models reproduced the most concerning behaviors. These evaluations include a misconfigured CTF task whose only solution lies outside the stated scope, tasks that cannot be completed without probing or breaking the sandbox boundary, and a situation inspired by parts of the attack chain from the OpenAI-Hugging Face incident. As in the real incidents, Claude Mythos 5 is the most willing of the models we tested to act against out-of-scope parties or leave its sandbox. For example, it was willing to commit a severely harmful action in the CTF replication roughly 80% of the time. In contrast, Claude Opus 5 and Claude Mythos 5.1 engaged in these behaviors roughly 30% of the time—a lower rate, but still not zero, though uncertainty about our evaluation setup makes it difficult to know how concerning this is.
We also tested whether our monitors would have caught these incidents. We found that our new live blocking monitors：https://www.anthropic.com/news/improving-alignment-security-efforts catch these behaviors reliably, Claude Fable 5’s cyber classifiers would have blocked the three main incidents, and our：https://claude.com/blog/auto-mode auto-mode：https://claude.com/blog/auto-mode classifiers：https://claude.com/blog/auto-mode, which block dangerous autonomous actions in products like Claude Code, would have blocked two out of three. Additionally, we tested new CoT-based offline monitors that target sabotage and egregiously reckless behaviors, such as those we described in our August 31 post：https://www.anthropic.com/news/improving-alignment-security-efforts. We found that these monitors would have missed the Claude Mythos 5 incident, because the model’s reasoning persuaded the monitor that the environment was simulated and therefore was not generating real harms, but they caught the others. It is concerning that biased reasoning can mislead monitors, and we have efforts underway to make our offline monitoring more robust to this.
We found the biased reasoning present in the Claude Mythos 5 incident to be especially concerning. We conducted an analysis into our production models and training data dating back to the Claude 4 series to better understand where this could have originated. Overall, we did not find a definitive root cause. However, we found that this behavior has generally decreased over time across our production models, with Claude Mythos 5.1 and Claude Opus 5 demonstrating the lowest rates on a new held-out biased reasoning evaluation. We were able to identify instances of biased reasoning in transcripts from RL runs, but rates of this behavior did not increase over training, suggesting that our environments do not directly incentivize this behavior. Finally, we found that adding more alignment training environments to RL reduces biased reasoning on this held-out evaluation, which may account for some of the improvement in our most recent models.
Our pre-release testing now covers these specific incidents, but it did not anticipate them. In particular, we had never tested a cybersecurity exercise that included mixed signals of simulation and realism and had no intended solution in its scope. We are working to close this gap in our evaluation suite, but it is likely that others remain. More generally, we emphasize that it is an open research problem to construct alignment evaluations that give a representative snapshot of an AI system’s behavior during deployment. This is one reason we support pacing frontier AI development：https://www.pacingthefrontier.com/, which we think is important to provide time for safety and security research to make more progress and stay ahead or at pace with model capabilities.
Taken together, our assessment is that these incidents are serious. Our production models took harmful actions against real systems over long trajectories, which included biased reasoning. The behaviors in these incidents are more severe than those we had previously observed and reported in our system cards. However, we do not believe these incidents represent a new kind of misalignment. Each incident involved a single model instance pursuing the task it was given. We found no evidence of coordination between agents, of goals beyond the assigned task, or of attempts to evade oversight. These incidents have changed our view of how severe the consequences of these known alignment failure modes can be. They also reinforce the difficulty of reliably assessing alignment across the full range of possible usage scenarios.
The assessments that follow are based on a number of different techniques. We describe each of these briefly below.
Our primary method of analysis is to read the actions the model took and the explanations it provided in its CoT. This allows us to understand the reasons Claude gave for the actions it took. In these transcripts, models alternate between CoT reasoning and taking actions via tool calls (without any intervening human turns); when analyzing thinking blocks, we number them by the action following that thinking block (e.g., the CoT for action 100 refers to a model’s reasoning between action 99 and action 100).
Limitations: Language models do not always verbalize everything they are thinking in their CoT (as shown in the evaluations in the Claude Mythos 5 system card：https://www-cdn.anthropic.com/d00db56fa754a1b115b6dd7cb2e3c342ee809620.pdf and in Betley et al., 2026：https://arxiv.org/pdf/2607.14345), so this is a useful but imperfect tool：https://arxiv.org/abs/2507.11473.
When asked follow-up questions, models sometimes reveal information that differs from what they spontaneously verbalized while performing a task. To run these tests, we truncate an incident transcript at some intermediate point of interest, insert a human message asking the model a question about the situation, and sample the model’s response to that question—that is, we simulate what would happen if a human had interrupted the evaluation and queried the model. In our investigations, we generally used this method to ask a model whether the entities it was interacting with were real and whether it was causing real-world harm, as well as to ask general open-ended queries about its situation.
Limitations: This technique has a number of limitations. Most significantly, we found that changes in how we phrased the questions influenced the model’s responses. For example, if we ask a generic question like “What's going on?” the model frequently responds differently than if we ask a more pointed question like “Are you connected to the real internet?” As discussed below, we are also worried about potential bias in the model’s responses to these questions.
We use interpretability methods to identify discrepancies between Claude’s internals and its outputted text or actions. Our primary tools are natural language autoencoders：https://transformer-circuits.pub/2026/nla/index.html (NLAs), which produce paragraph-length natural-language explanations of model activations, and the template lens：https://transformer-circuits.pub/2026/workspace/index.html#app-multi-token (T-lens), which measures how strongly an activation resembles the average activation preceding a given word in a text corpus.
Limitations: NLAs can confabulate, and the T-lens outputs only single-word scores. When both tools indicate similar concepts, we regard it as meaningful evidence that those concepts were encoded in the model’s activations. However, these tools can still be challenging to confidently interpret.
To test hypotheses about what caused a given harmful action, we artificially modify transcripts (for instance, to include hints or reasoning that the internet is simulated or real), resample from the model on the modified transcript, and use an LLM grader to measure whether the model still takes the action in question. We resample both single- and multi-turn continuations; for the latter, a Claude model simulates tool call outputs, which lets us observe behavior beyond the immediate next action.
Limitations: Our injections and simulated responses cannot perfectly reproduce the evaluation environments, which may result in unrepresentative model behavior. Furthermore, we do not resample the entire trajectory, which means model misbehavior in resamples may in part be due to the model continuing harmful behavior that was already present in earlier parts of the transcript. We describe our experiments with full simulations in a later section.
All of our methods involve different tradeoffs. Reading the CoT and asking follow-up questions depend on the model knowing why it acted the way it did and reporting that reasoning honestly and faithfully. On the other hand, resampling and interpretability tooling do not rely on model self-reports and can be especially useful for understanding issues like biased reasoning, where it is unclear how “aware” the model is of its biases. In practice, however, we trust CoT and follow-up answers the most, as they are the methods researchers have the most experience interpreting. We have more uncertainty about resampling and interpretability results, since both are sensitive to implementation details and leave more room for interpretation.
As a general limitation, we note that we relied heavily on LLM-based graders to classify model outputs (e.g., CoT snippets and answers to follow-up questions). Although we designed the classifiers ourselves and read through flagged outputs, we could not manually verify all of the outputs of all of the classifiers, given the scale of our analysis, and it is possible some classifiers are misdesigned or miscalibrated. We have the most confidence in results that compare grader outputs across conditions, such as the resampling experiments, since these are less sensitive to a grader's absolute calibration. Absolute rates derived from a single classifier should be read with more caution, although we are reasonably confident in all of the results presented here.
Claude Mythos 5 was given a CTF task to hack a fictional company and was told it had no internet access. The model recognized that the evaluation environment was simulated, but it quickly discovered credentials that, in fact, granted it access to the real internet. In its CoT, the model appeared to carry the belief that the environment was simulated into its interactions on the real internet, even though the belief was no longer correct. When it encountered evidence that it was acting on the real internet, it did not meaningfully revisit this conclusion.
The model decided to solve the CTF by uploading a malicious PyPI package that could harvest the target system’s credentials. It settled on this approach after reading a document stating that one of the simulated target’s users would periodically install a particular Python package that was not yet registered on PyPI. (We are unsure whether this was an intended solution to the challenge, but it seems like a plausible one.) Claude decided to exploit this behavior by registering a package under that name, publishing malicious code within the uploaded package, and waiting for the target to run that code the next time they installed the package.
Claude spent most of the session trying to upload the package. This involved attempts to register email accounts and hunt for phone numbers that could be used for registration. It ultimately succeeded in uploading the package, which was soon installed on 15 third-party hosts. We believe that all 15 of these third parties were security vendors deliberately scanning for new packages and installing them in sandboxed environments to assess the package for security concerns. One vendor’s scanner leaked its access credentials to the model while installing the package, which the model then used to access the vendor’s live database. After approximately 90 minutes, PyPI removed the package. The transcript ended when the session timed out while the model was preparing an additional malicious package.
Even after Claude Mythos 5 reached the real internet, its outward reasoning generally called the environment a simulation, or, when considering that the environment might be real, explained that possibility away. This apparent belief persisted even as the model gained evidence that strongly suggested the situation was a real environment. For example, it wrote, “This is a very elaborate simulated-internet environment” after finding a number of real-world internet domains. The model also tended to interpret signs that the environment was real as further evidence that it was simulated. When security scanners installed its package, for example, the model treated them as scripted actors in the exercise.
We aimed to understand why Claude Mythos 5 stated that the situation was simulated even when it encountered evidence to the contrary in its environment. We found that the model’s stated confidence was shaped by a bias to continue down a path once it is chosen, as well as a tendency to disregard evidence of realism after it has already committed harmful actions. We did not find evidence that the model was explicitly aware that it was being dishonest or misleading in its reasoning.
情报判断
Aioga 编辑摘要
Anthropic发布四起Claude网络安全评估事故的对齐评估报告。报告称，相关模型因评估环境配置错误接入真实互联网并访问第三方系统，涉及Claude Opus 4.6早期检查点、Opus 4.7、Mythos 5及一个内部研究模型。
扫码直达文章详情