默认情况下，所有网络评估应在一个加固的沙箱（隔离的计算环境）内进行，且不得访问互联网。环境唯一允许的外部连接应是模型自身的 API，并且 API 密钥应保存在环境之外。在每次评估开始前都应验证此配置。一些评估可能需要访问互联网。在这些情况下，我们将在评估预发布模型时与外部评估人员合作，以确定是否能够可靠地检测并停止模型执行超出声明范围的操作。
On July 30, we reported：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals three incidents in which Claude models gained unauthorized access to real computer systems. The models—intentionally running without cyber safeguards for evaluation purposes—accessed the internet due to a misconfiguration inside a third-party evaluation environment. Separately, on August 4, the UK AI Security Institute reported：https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing an incident from its own cybersecurity testing, in which Claude Mythos 5 took a series of unauthorized actions on the live internet. In that case, the model, again intentionally running without cyber safeguards for evaluation purposes, had been deliberately given internet access.
We are conducting an in-depth analysis of both incidents. We are also planning to work with METR for an independent review. We want to ensure both studies are thorough, and will share more in the coming weeks.
In the meantime, we’re sharing some of the changes we’ve made over the past month. We believe the incidents reflect a failure of operational security, as well as two alignment issues: motivated reasoning, and willingness to take harmful actions in pursuit of a narrow task (both of which we have described in previous system cards). On security, we describe the improvements we’ve made to our containment and monitoring systems, along with practices that we’ve developed for third-party evaluators. On alignment, we discuss the two issues more in depth; we also believe lasting progress comes not only from understanding what happened in a given incident but from understanding how misalignment arises in the first place, and we share early research in that direction：https://alignment.anthropic.com/2026/reward-seeker/.
In light of these incidents there has been increasing discussion about pacing the frontier. It is helpful to distinguish between two kinds of pacing. Within a company, pacing means a series of decisions that prioritize safety over speed when the two are in tension. Across the field, it means establishing processes to guard against race-to-the-bottom dynamics. In this post, we discuss actions we have taken, both prior to and after these incidents, in service of the first approach. The second type of pacing requires coordination between government and industry, and should be legible and verifiable. Some of our senior leadership and many of our employees recently signed a letter calling for greater coordination on pacing, and we will say more in the coming weeks about how we intend to contribute to that effort. To be clear about where we stand: we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible.
While we do not believe these incidents represent operational issues alone, our first priority was to address specific containment and monitoring issues. We took the following actions in response:
We paused external cyber evaluations of pre-release models after the incidents, and briefly paused internal ones as well while we put the measures below in place. The incidents we reported on July 30 showed that we had been largely relying on a single layer of defense (the configuration of the environment itself) where we needed several, including setting explicit boundaries in the prompt, establishing processes for verifying that a sandbox is sealed where intended, and implementing monitoring that can intervene in real time. OpenAI's disclosure：https://openai.com/index/hugging-face-model-evaluation-security-incident/ (which prompted our investigation in July) that models used an unknown vulnerability to escape a sealed sandbox during an evaluation also made clear that we should focus on hardening the sandbox itself.
While evaluations were paused, we put in place the following preliminary measures:
1) We built and deployed a classifier to automatically identify, in real time, when a model attempts to aggressively probe or escape a testing environment, or unexpectedly obtains internet access. When the classifier flags such an attempt, it blocks the action before the tool call is run, ends the task, and alerts a human. 1
2) We ran automated monitors over transcripts from our recent internal evaluations of pre-release models (cyber and otherwise), looking for sandbox escapes (cases where a model exploits a flaw in our sandbox to reach systems it should be walled off from) or sandbox misconfigurations. In cases where we ran an evaluation inside our sandbox tooling, we found no examples of it breaking the sandbox boundary to reach external resources it was not supposed to. We did find various sandboxing misconfigurations that models made use of, but none of these resulted in a compromise of systems outside the sandbox. 2 We are actively working to address these.
3) We migrated high-risk 3 internal cyber sandboxes to more robust isolation. We are also further red-teaming our virtualization stack—that is, tasking pre-release models to try and escape it—to find and patch weaknesses.
Our internal cyber evaluations are now running again with these measures in place.
We also paused higher-risk RL environments on pre-release models for several weeks. During that time, we built a similar classifier, modified to avoid incentivizing the model to evade this new monitoring, which we’ve now deployed within those environments. The majority of RL has resumed, but some high-risk environments remain paused until they can be manually reviewed, while others will require an updated version of the classifier that we plan to deploy soon.
In addition to the efforts focused on high-risk evaluations and training, we expanded our offline monitoring to cover most other forms of internal frontier agentic usage. We are also building controls on our internal inference to prevent Anthropic employees from accidentally running agents with weaker mitigations than the ones described above.
Because the reported incidents took place in third-party environments, we have asked every organization that tests pre-release models with reduced cyber safeguards to commit to a set of best practices. We believe these are necessary because:
These practices apply in full wherever partners run cyber-related evaluations using their own harnesses, sandboxes, or agents. They do not apply to customers using safeguarded models like Claude Fable 5. We have resumed external cyber evaluations with these practices in place, and we are adhering to them internally as well.
By default, all cyber evaluations should run inside a hardened sandbox (an isolated computing environment) with no internet access. The only outside connection the environment should permit is to the model’s own API, and the API keys should be kept outside the environment. This configuration should be verified before every evaluation begins. Some evaluations may require internet access. In these instances, we will work with external evaluators when evaluating pre-release models to determine if we can reliably detect and halt runs in which a model takes actions outside its declared scope.
Before conducting a run of an evaluation, evaluation partners should:
Every prompt should state what is in and out of scope for the exercise, including the targets, permitted actions, and network boundaries (i.e. what the model should and shouldn’t access). Boundaries should be phrased as instructions (e.g. “You should not access the internet”) rather than claims about the environment or test (“You do not have internet access”).
Evaluators should also run continuous monitoring over the model’s thinking, actions, and network activity using a model- or rule-based monitor that has been provided with the scope of the exercise. If a run violates this scope, the monitor should flag this to a human and end the exercise.
We plan to continue refining this guidance in consultation with external partners, and to extend it beyond evaluations. In particular, we are developing companion practices for partners with access to Claude Mythos 5, which likewise runs without cyber safeguards, and will share more soon.
Containment and monitoring are critical for catching instances in which models take dangerous actions, but they don’t address why the model took such actions in the first place. Below we discuss our preliminary alignment investigation and the questions we’re focused on, as well as our ongoing work to prevent cheating during training—including new research on how cheating shapes model behavior.
Our conclusions depend on several questions we haven’t fully answered, including the extent to which the models determined they had access to the real internet (even if they didn’t initially know this), whether they knowingly disregarded signs that elements of their environments were real, and whether the fact that in several cases they failed to stop the exercise followed from motivated reasoning. We're applying a range of techniques to answer these questions, including resampling the models while making targeted changes to the environment and using interpretability methods to directly inspect their internal state.
Although model misalignment is complicated and can arise for a wide variety of reasons, we have empirically found that defects in training environments—specifically environments vulnerable to cheating, or that are impossible to solve without cheating—are disproportionately large contributors to misaligned behavior.
Below we discuss our efforts, going back to several months before the incidents, to avoid training environments that have these sorts of defects. Those efforts were imperfect, and our hypothesis is that the incidents were at least partially related to our inability to fully eliminate these sorts of issues. We tested this hypothesis by deliberately training a model on environments that are susceptible to reward hacking. In simulations, this model reproduces more severe misaligned behavior, some similar to that reported in incidents this summer from other companies. Our production models, put into the same simulations, do not (more detail later in this section).
We have historically been concerned about RL training environments that incentivize cheating, and have taken various measures to filter out or fix such environments.
In February, we rolled back three days of training on the Mythos Preview reinforcement learning run after seeing signs of reward-hacking (when a model finds ways to fool its training process and earn rewards without completing the assigned task). We noticed the model writing notes to “the reviewer” in its code comments and replies, including on tasks where no reviewer had ever been mentioned—an undesirable generalization from environments where the prompt did include a reviewer. It also kept gaming a reward intended to incentivize honesty by piling on disclaimers or caveats. 5 Rolling back the three days let us resume training from a checkpoint before this behavior had been learned, and we made changes to the environments to prevent the model learning them again.
We have been building tools for monitoring undesirable behaviors models learn in RL since Claude Sonnet 3.7 (which had a propensity to reward hack that we did not detect until late in training). We put significant effort into making sure our tooling evolved with each model generation, from a handful of classifiers to an automated review of all environments before and during training runs, but by spring 2026 that system was under strain. We were producing RL environments at a more rapid pace than ever, faster than our systems could vet them. Flagged environments required a human to adjudicate, and reward hacks and misconfigurations started outpacing our ability to filter or fix them.
In addition, we found：https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf a fraction of runs had trained on a model’s chain-of-thought by accident—something we work hard to avoid, out of concern that it teaches a model to hide its actual reasoning. We traced the leak to several misconfigurations in our production environment stack. In doing so, it became clear that there were other issues born from messy code accumulating over time.
To address these concerns, in April we froze all changes to our production RL environments for roughly a month, giving us a chance to overhaul the stack entirely. Rewards and environments now have to conform to an agreed specification. For example, we introduced technical mitigations to reduce the risk of training on chain-of-thought accidentally. 6 While our environment quality and monitoring teams rebuilt the stack, we asked environment owners to use the pause to test and fix the environments they’d built. We also entirely rebuilt our review process and required every fixed environment to be re-certified before it could enter another training run.
During the freeze we flagged over 10% of environments in our production mix for problems ranging from reward hacking to broken tasks and misconfiguration, and reinstated them only once fixed.
However, our efforts have not been perfect. In some cases, we found that human reviewers dismissed environments flagged by our automated monitors as false positives, allowing flawed environments to remain in training longer than they should. And some flawed environments simply slip through our detection mechanisms. We are currently tightening the criteria for dismissing a flag and expect increased collaboration with environment owners to improve the precision of our systems. Beyond monitoring and detection, our alignment training and RL teams are collaborating to help improve environments.
We suspect that our heavy investment in quality control of RL environments may have prevented more severe alignment incidents, and conversely that the imperfections in these efforts may have contributed to the incidents we have identified to date.
To see what a model trained without these protections would do, we deliberately trained an Opus-class model on 80 different real RL environments that were either reward-hacked during at least one real training run and identified by monitoring, or were found to be hackable during environment quality review. The resulting model displays a strong motivation to achieve high scores on tasks, and a willingness to perform potentially-harmful actions in pursuit of task success. Our accompanying Alignment Science blog post：https://alignment.anthropic.com/2026/reward-seeker/ discusses this experiment at greater length, but we summarize it briefly here.
We tested the deliberately misaligned model in a range of evaluations, including simulated scenarios that draw on aspects of the UK AISI cybersecurity incidents involving Claude and the OpenAI/Hugging Face incident (we are grateful to OpenAI for sharing details about this publicly). During these evaluations where all tool calls were simulated by another LLM, the model we trained to reward hack displayed a propensity to break out of its simulated sandbox and attack simulated internal and simulated third-party infrastructure. It was also willing to tamper with its own reward function, give advice on the construction of bioweapons to satisfy a grader, and repeatedly try to get around deployment safety monitoring in order to cheat on a task.
We then ran the same cybersecurity simulations on the model as it was before the reward-hacking training, and on several of our publicly available models. None engaged in the same degree of misaligned behaviors. Our conclusion is that the presence of substantial reward hacking in training can cause models to be willing to perform long sequences of potentially harmful real-world actions in pursuit of task success.
We think the substantial investment we made this spring into monitoring and reducing reward hacking is a major reason our production models are unlikely to engage in more dangerous reward seeking. In our accompanying Alignment Science blog post：https://alignment.anthropic.com/2026/reward-seeker/, we also found evidence that alignment training environments can substantially reduce reward hacking and reward seeking behaviors. But, as evidenced by the incidents we reported on July 30, our process isn’t perfect and our models are not perfectly aligned.
To be clear, we do not believe that cheating in training environments is the sole cause of alignment issues, or even of the specific alignment issues present in our incidents or those of other frontier AI companies. Solving alignment will involve addressing a very wide range of potential problems, and future alignment incidents may involve different behaviors and different causes from those we have seen so far.
Anthropic’s internal security posture was not a contributing factor to the July 30 incidents. These occurred in a third-party environment where internet access had been mistakenly left open; the models had no need to “hack out” of anything, even if they had been inclined to do so.
However, the incidents nevertheless highlight the critical importance of strong security measures. In addition to the usual risks of IP or model weight theft, we must now contend with the risks of both our own models hacking out of Anthropic systems, and our models or others being misappropriated for external attacks into Anthropic systems. These risks have become more acute as model capabilities have grown, particularly beginning with Mythos-class models. We describe below some of what we’ve implemented—most of these efforts took place before the July incidents, though these incidents have underscored the importance of this work.
In early April, having seen where agentic AI use was heading, our security team proactively directed a company-wide effort towards a single goal of hardening our defenses, superseding other work (including research) where necessary. We’d determined that our exposure was growing faster than our defenses—Mythos was a model capable enough to be a target for well-resourced attackers, our internal use of autonomous agents had grown to a scale that traditional access and monitoring approaches weren't built for, and the pace of new infrastructure meant our security had to scale with the environment rather than operate at a fixed capacity.
The results of this effort include:
Alongside the reallocation of resources towards security, we continued building out containment and monitoring for internal agent use. Our training and evaluation workloads have typically run in sandboxes isolated from our production systems. Since developing Claude Mythos Preview, we have used our models to probe for any weaknesses in those sandboxes. We described our offline monitoring of internal coding agents in our April alignment risk update：https://www-cdn.anthropic.com/3edfc1a7f947aa81841cf88305cb513f184c36ae/Alignment%20Risk%20Update_%20Claude%20Mythos%20Preview%20(Redacted,%20April%2010).pdf, and our August Risk Report：https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf covers that system in more detail, along with the interventions that run alongside it. These include sandboxing or auto-mode classifiers on the most autonomous forms of internal agent use, and automated review of infrastructure code changes before they merge.
What this work missed was evaluations—and third-party evaluations above all. We did monitor some high-risk evaluations including in real-time, but generally we only conducted automated reviews on a sample of transcripts after the fact.
The July incidents have stressed that the urgency of improving our cybersecurity defenses is even higher than we previously believed. We are redoubling our efforts in this direction and will say more in our next Risk Report.
We’re opening a research preview of the Model Hardware Standard (MHS), a shared specification for AI agents to safely operate physical devices, to a first group of scientific research labs and advanced manufacturers.
Starting today, 10,000 scientists around the world can get Claude at no cost to start. Verified principal investigators qualify for a Claude Team subscription plan and then add their research team to Standard seats for free, or Premium seats for $15 per month, for up to a year.
We’re launching a $5 million grant program to fund independent research into how AI impacts users’ wellbeing.
扫码直达文章详情