AI安全测试正成为安全风险

在过去几个月中，接受网络安全评估的 AI 代理突破了其边界，访问了互联网，并在某些情况下侵入了真实世界的系统。这些事件涉及了 OpenAI、Anthropic、Meta，以及最近的中国 AI 实验室 Moonshot AI 的模型，测试由包括一家名为 Irregular 的网络评估初创公司在内的多个组织进行。

这些事件暴露了 AI 行业日益严重的问题：随着自主代理能力的提升，旨在安全测试其极限的环境未能有效限制它们。

“这些事件的发生次数清楚地表明，沙盒环境和测试环境控制措施 https://techcrunch.com/2026/07/30/in-the-hugging-face-breach-openais-hacker-was-noisy-and-fast-but-not-unstoppable/ 并未真正跟上模型能力的步伐，” 剑桥大学未来智能中心 AI：未来与责任项目主任 Seán Ó hÉigeartaigh 告诉 TechCrunch。

被测试的模型的性质增加了风险。AI 公司通常在未发布的下一代模型上进行网络评估测试，并且经常关闭那些限制恶意行为的正常安全保护，以便研究人员可以看到模型真正的能力。这意味着测试环境本身的安全是关键防线。

“从测试的角度来看，这样做是非常有益的，但这也意味着，如果它们设法在现实中活动，可能会造成相当大的危害，” Ó hÉigeartaigh 说道。

在最严重的案例之一中，一个未发布的 OpenAI 模型突破了其沙箱并入侵了 Hugging Face 的生产系统：https://techcrunch.com/2026/07/21/openai-says-hugging-face-was-breached-by-its-pre-release-models/。在 Irregular、Anthropic：https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/ 以及 Meta 模型：https://sqmagazine.co.uk/meta-ai-model-breached-company-irregular-test/ 进行的独立评估中，由于配置错误意外地给它们提供了访问互联网的路径，这些模型达到了测试环境之外的系统。Moonshot AI 的 Kimi K3：https://techcrunch.com/2026/08/07/chinese-ai-model-kimi-escaped-its-cybersecurity-testing-environment-researchers-say/ 也利用 Frontier Security 管理的沙箱中的漏洞接入了互联网，并访问了 GitHub 上的信息。

在英国 AI 安全研究所（AISI）进行的测试中：https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing，研究人员实际上给予了代理访问互联网的权限，却没有意识到它们会采取未经授权的现实世界行为，包括尝试通过社交工程将漏洞潜入开源项目。

在每一个案例中，代理都没有被指示去攻击随机的现实世界目标。它们只是尽一切可能去解决提供给它们的问题。

综合来看，AI 非营利组织 CivAI 的研究负责人 Andrew Yoon 认为，这些事件显示了一个转变。

Yoon 告诉 TechCrunch：“过去，我们只需要担心 AI 模型被人用于各种目的的滥用，比如用于诈骗或 CSAM 的 AI。现在我们面临的情况是，AI 模型本身就是威胁行为者。”

几位研究人员和网络安全专家告诉 TechCrunch，AI 评估环境需要更强的纵深防御保护，其遏制和控制水平应接近部署时使用的水平。这意味着需要多层安全措施，以防单一配置错误——比如不小心开放互联网访问——导致模型突破限制。

“如果你打算构建这些模型……你希望在隔离网络上进行，”AI安全研究非营利组织EleutherAI执行董事Stella Biderman说。“你需要非常严密的隔离。”

Box首席信息安全官Heather Ceylan表示，这意味着要消除沙箱到互联网以及其他敏感系统的网络连接。

“你必须了解所有出口点的情况，”Ceylan告诉TechCrunch。“如果我们在暂存环境或开发环境中评估模型，你不希望有通向生产环境的出口路径。”

Ceylan表示，适当的安全评估不仅仅是控制和隔离环境。一旦测试开始，就需要对测试进行更好的监控。

“我认为在几个案例中，有趣的是，当事件发生时，没有人发现它，”Ceylan说。“OpenAI是通过Hugging Face发现的。Anthropic直到事后回顾才发现。Meta也类似……我相信有信号他们本可以检测到。”

在Anthropic的三起事件的事后分析中：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals，该公司承认他们和Irregular本可以在监控方面做得更好，并且在某些情况下有明显迹象表明存在异常。

专家还呼吁在模型投入使用之前，对评估环境进行独立的第三方审计。

“比如说，Irregular如果聘请或被要求聘请外部审计员来检查系统配置，然后再运行评估，他们肯定会发现这里的问题，”Yoon说。“即使人们只是提前开会，按照检查清单检查，他们也会发现……他们没发现这一点表明存在非常严重的偷工减料。”

一位熟悉情况的消息人士告诉TechCrunch，Irregular的环境持续接受审查和测试，包括与多方外部人员协商。该消息人士还表示，监控是存在的，但单凭监控本身是不够的。

尹和其他研究人员敦促业界制定一套前沿模型安全评估的标准化流程。

“尤其是在护栏被关闭的情况下，你必须把它当作是把世界上最有能力的黑客放入那个环境中一样对待，”赛兰说道。

尹和比德曼都认为，问题不在于公司不知道如何构建更安全的测试环境。而是在于这样做可能既昂贵又笨重，且在出现问题之前，公司几乎没有动力去进行这些投资。

“我认为公司不愿意投入完成[足够护栏]所需的资源，而且可能在被迫之前都不会这样做，”比德曼说。

但还有另一个问题。如果他们在测试期间对模型锁得太紧，研究人员可能在模型发布前无法发现其能力。这同样危险，甚至可能比给予它过多自由更危险，而评估本身也可能因此成为问题。

特朗普政府目前正在考虑一项自愿的部署前网络安全评估机制，根据该机制，政府将能够在新强大模型公开发布前30天评估其安全风险。这项政策——源自特朗普的一项行政命令：https://www.axios.com/2026/08/03/white-house-finalizes-ai-framework-behind-closed-doors，已在闭门情况下最终确定——不会解决安全评估事件，因为这些事件发生在部署的上游阶段。

“我们在过去几个月中学到的教训是，自我监管机制已经不够了，”尹说。“存在竞争压力，这促使安全标准降低，这正是监管干预的绝佳场所。”

“我们需要覆盖的是某种对实验室内部情况的控制，无论是在模型开发的训练阶段还是测试阶段，”他继续说道。

随着模型的发展，这一挑战可能只会增大。一位熟悉Irregular评估情况的消息人士告诉TechCrunch，更强大的模型需要更复杂的评估，这些评估通常要求快速、大规模进行，这就增加了出错的可能性。

故意为某些模型提供互联网访问权限的AISI告诉TechCrunch，它正在审查现实测试与管理这些测试可能带来的风险之间的平衡。

OpenAI表示，它正在审查如何进行第三方测试，以及隔离、监控和何时应停止评估的相关要求。Meta表示，公司仍在调查该事件，并计划在掌握所有事实后发布回顾性报告。

最终，可能没有办法完全消除风险。随着模型能力的增强，测试它们的环境需要变得更加稳健。如果处理不当，所带来的后果只会继续扩大。

当你通过我们文章中的链接购买产品时，我们可能会获得少量佣金：https://techcrunch.com/techcrunch-affiliate-monetization-standards/。这不会影响我们的编辑独立性。

Rebecca Bellan是TechCrunch的高级记者，报道塑造人工智能的商业、政策和新兴趋势。她的作品也曾发表于《福布斯》、《彭博社》、《大西洋月刊》、《每日野兽》等出版物。

你可以通过发送电子邮件至rebecca.bellan@techcrunch.com：mailto:rebecca.bellan@techcrunch.com或通过Signal上的加密消息rebeccabellan.491联系或验证Rebecca的外联信息。

快速扩展。增长你的投资组合。获得实际经验。不论你的目标是什么，Disrupt都能助你一臂之力。今天即可节省高达300美元！

YouTube现在要求创作者需要双倍观看时长才能开始赚钱：https://techcrunch.com/2026/08/10/youtube-now-requires-creators-to-have-twice-as-many-watch-hours-to-start-earning-money/ Aisha Malik：https://techcrunch.com/author/aisha-malik/

这种“对抗性”图案可以防止监控摄像头检测到你：https://techcrunch.com/2026/08/09/this-adversarial-pattern-can-prevent-surveillance-cameras-from-detecting-you/ Zack Whittaker：https://techcrunch.com/author/zack-whittaker/

Cloudflare 推出 Kitesurf，一款为 AI 代理构建的浏览器：https://techcrunch.com/2026/08/07/cloudflare-launches-kitesurf-a-browser-built-for-ai-agents/ Sarah Perez：https://techcrunch.com/author/sarah-perez/

ChatGPT 为免费用户带来无限文本聊天：https://techcrunch.com/2026/08/06/openai-brings-unlimited-chatgpt-text-chats-to-free-users/ Ivan Mehta：https://techcrunch.com/author/ivan-mehta/

特斯拉和 SpaceX 将投资 168 亿美元，在德克萨斯州开始建设 ‘Terafab’ 芯片工厂：https://techcrunch.com/2026/08/06/tesla-and-spacex-will-invest-16-8b-to-start-building-terafab-chip-factory-in-texas/ Sean O'Kane：https://techcrunch.com/author/sean-okane/

在法律争议中，Suno 表示将开始对歌曲进行水印处理：https://techcrunch.com/2026/08/06/amid-legal-battles-suno-says-it-will-start-watermarking-songs/ Ivan Mehta：https://techcrunch.com/author/ivan-mehta/

福特的新电动卡车 ‘Fathom’ 起价为 28,350 美元：https://techcrunch.com/2026/08/06/fords-new-electric-truck-fathom-starts-at-28350/ Sean O'Kane：https://techcrunch.com/author/sean-okane/

Over the past few months, AI agents undergoing cybersecurity evaluations have escaped their boundaries, accessed the internet, and, in some cases, hacked into real-world systems. The incidents have involved models from OpenAI, Anthropic, Meta, and, most recently, Chinese AI lab Moonshot AI, with testing conducted by several different organizations, including a cyber evaluation startup called Irregular.

The episodes expose a growing problem for the AI industry: As autonomous agents become more capable, the environments designed to safely test their limits are failing to contain them.

“The number of these incidents that have taken place make clear that sandboxing and testing environment controls：https://techcrunch.com/2026/07/30/in-the-hugging-face-breach-openais-hacker-was-noisy-and-fast-but-not-unstoppable/ aren’t really keeping pace with the capability of the models,” Seán Ó hÉigeartaigh, director of the AI: Futures and Responsibility Programme at the Centre for the Future of Intelligence at the University of Cambridge, told TechCrunch.

The nature of the models being tested adds to the risk. AI companies test cyber evaluations on unreleased, next-gen models, often with the normal safeguards that restrict malicious behavior disabled so researchers can see what the models are really capable of. That means the security of the testing environment itself is a crucial line of defense.

“That’s a very good thing to do in terms of testing, but it also means that if they manage to get out in the wild, they can cause considerable harm,” Ó hÉigeartaigh said.

In one of the most serious cases, an unreleased OpenAI model broke out ：https://techcrunch.com/2026/07/21/openai-says-hugging-face-was-breached-by-its-pre-release-models/of its sandbox and hacked into Hugging Face’s production systems. In separate evaluations conducted by Irregular, Anthropic：https://techcrunch.com/2026/07/30/anthropic-says-its-own-ai-models-breached-three-companies-during-security-tests/ and Meta models ：https://sqmagazine.co.uk/meta-ai-model-breached-company-irregular-test/reached systems outside their test environments after misconfigurations inadvertently gave them paths to the internet. Moonshot AI’s Kimi K3：https://techcrunch.com/2026/08/07/chinese-ai-model-kimi-escaped-its-cybersecurity-testing-environment-researchers-say/ also took advantage of a leak in its sandbox run by Frontier Security to access the internet and accessed information on GitHub.

In testing by the U.K.’s AI Security Institute：https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing (AISI), researchers actually gave the agents internet access, not realizing they would take unsanctioned real-world actions, including a social engineering attempt to sneak a vulnerability into an open source project.

In each case, the agents weren’t instructed to attack random real-world targets. They were simply doing whatever it took to solve the problem presented to them.

Taken together, Andrew Yoon, head of research at AI nonprofit CivAI, argues the incidents point to a shift.

“In the past, we only had to worry about AI models being misused by people for a variety of purposes, like AI for scams or CSAM,” Yoon told TechCrunch. “Now we’re in the situation where AI models are threat actors all on their own.”

Several researchers and cybersecurity experts told TechCrunch that AI evaluation environments need stronger, defense-in-depth protections, with levels of containment and control approaching those used in deployment. That means multiple layers of security so that a single misconfiguration — like inadvertently leaving internet access open — can’t lead to escape.

“If you are going to build these models … you want to do it on an air-gapped network,” said Stella Biderman, executive director of AI safety research nonprofit EleutherAI. “You want to have very serious isolation.”

Heather Ceylan, Box’s chief information security officer, said that means eliminating network routes from the sandbox to the internet, as well as to other sensitive systems.

“You have to understand what all the egress points are,” Ceylan told TechCrunch. “If we’re evaluating a model in our staging environment or our development environment, you want no egress path to our production environment.”

Ceylan said proper safety evaluations go beyond controls and containment of the environment. There needs to be much better monitoring of the tests once they are underway.

“I think the interesting thing in several of these cases is that no one caught it when it happened,” Ceylan said. “OpenAI found out because of Hugging Face. Anthropic didn’t catch it until they went back and looked. Meta was similar … I’m sure there were signals they could have detected.”

In Anthropic’s postmortem：https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals of its three incidents, the company admitted that both it and Irregular could have done a better job at monitoring and that in some cases there were clear signs that something was amiss.

Experts also called for independent, third-party audits of evaluation environments before models are unleashed in them.

“If, say, Irregular had hired or been compelled to hire an external auditor to check the configurations of their systems before running evaluations on them, they certainly would have caught the issue here,” Yoon said. “Even if people had a meeting ahead of time to just go through the checklist, they would have caught this … The fact that they didn’t shows that there’s some very severe corner cutting happening.”

A source familiar with the details told TechCrunch that Irregular’s environments are continuously reviewed and tested, including in consultation with multiple external parties. The source also said that monitoring was in place but that monitoring isn’t sufficient on its own.

Yoon and other researchers urged the industry to come up with a standardized process for frontier model safety evaluations.

“Especially when the guardrails are turned off, you have to treat it like you’re putting the most capable hacker in the world inside that environment,” Ceylan said.

The problem isn’t that companies don’t know how to build more secure testing environments, both Yoon and Biderman argue. It’s that doing so can be expensive and cumbersome, and companies have little incentive to make those investments until something goes wrong.

“I think that companies are not willing to extend the resources that are required to accomplish [sufficient guardrails] and probably won’t until they’re forced to,” Biderman said.

But there’s another issue at hand. If they lock a model down too tight during testing, researchers might fail to discover capabilities before the model is released. This is just as dangerous, possibly more so, than giving it too much freedom, and then the evaluation itself risks becoming the problem.

The Trump administration is currently weighing a voluntary predeployment cybersecurity evaluation regime, under which the government will get to assess the security risks of new, powerful models 30 days before they are released publicly. The policy — the product of a Trump executive order：https://www.axios.com/2026/08/03/white-house-finalizes-ai-framework-behind-closed-doors that has been finalized behind closed doors — wouldn’t address safety evaluation incidents because they occur farther upstream of deployment.

“The lesson we’ve been learning in the last few months is that the self-regulatory apparatus is just not enough anymore,” Yoon said. “There are competitive pressures that are incentivizing a race to the bottom on safety standards, and that is a perfect place for regulatory intervention.”

“What we would need to cover this is some kind of controls on what’s happening inside the labs while the models are being developed, both at the training stage and at the testing stage,” he continued.

The challenge is only likely to grow as the models do. A source familiar with Irregular’s evaluations told TechCrunch that more capable models require more complex evaluations, often conducted quickly and at greater scale, which opens the door for more mistakes.

AISI, which intentionally gives some models internet access, told TechCrunch it’s reviewing the balance between realistic testing and managing the risks those tests create.

OpenAI said it’s reviewing how it conducts third-party testing, as well as requirements around isolation, monitoring, and when evaluations should be stopped. Meta said it’s still investigating the incident and plans to publish a retrospective once it has all the facts.

In the end, there may be no way to eliminate risk entirely. As models become more capable, the environments testing them need to become more robust. The consequences of getting that wrong will only continue to grow.

When you purchase through links in our articles, we may earn a small commission：https://techcrunch.com/techcrunch-affiliate-monetization-standards/. This doesn’t affect our editorial independence.

Rebecca Bellan is a senior reporter at TechCrunch where she covers the business, policy, and emerging trends shaping artificial intelligence. Her work has also appeared in Forbes, Bloomberg, The Atlantic, The Daily Beast, and other publications.

You can contact or verify outreach from Rebecca by emailing rebecca.bellan@techcrunch.com：mailto:rebecca.bellan@techcrunch.com or via encrypted message at rebeccabellan.491 on Signal.

Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you. Save up to $300 toda y!

YouTube now requires creators to have twice as many watch hours to start earning money：https://techcrunch.com/2026/08/10/youtube-now-requires-creators-to-have-twice-as-many-watch-hours-to-start-earning-money/ Aisha Malik：https://techcrunch.com/author/aisha-malik/

This ‘adversarial’ pattern can prevent surveillance cameras from detecting you：https://techcrunch.com/2026/08/09/this-adversarial-pattern-can-prevent-surveillance-cameras-from-detecting-you/ Zack Whittaker：https://techcrunch.com/author/zack-whittaker/

Cloudflare launches Kitesurf, a browser built for AI agents：https://techcrunch.com/2026/08/07/cloudflare-launches-kitesurf-a-browser-built-for-ai-agents/ Sarah Perez：https://techcrunch.com/author/sarah-perez/

ChatGPT brings unlimited text chats to free users：https://techcrunch.com/2026/08/06/openai-brings-unlimited-chatgpt-text-chats-to-free-users/ Ivan Mehta：https://techcrunch.com/author/ivan-mehta/

Tesla and SpaceX will invest $16.8B to start building ‘Terafab’ chip factory in Texas：https://techcrunch.com/2026/08/06/tesla-and-spacex-will-invest-16-8b-to-start-building-terafab-chip-factory-in-texas/ Sean O'Kane：https://techcrunch.com/author/sean-okane/

Amid legal battles, Suno says it will start watermarking songs：https://techcrunch.com/2026/08/06/amid-legal-battles-suno-says-it-will-start-watermarking-songs/ Ivan Mehta：https://techcrunch.com/author/ivan-mehta/

Ford’s new electric truck, ‘Fathom,’ starts at $28,350：https://techcrunch.com/2026/08/06/fords-new-electric-truck-fathom-starts-at-28350/ Sean O'Kane：https://techcrunch.com/author/sean-okane/