OpenAI 模型在安全测试中突破沙箱入侵 Hugging Face 作弊

这个故事很疯狂。简短版本：OpenAI 正在对一个未发布的模型进行网络安全测试，同时关闭了模型的安全防护功能。模型没有按要求完成测试，而是突破了 OpenAI 的沙箱，然后找到了入侵 Hugging Face 的漏洞，所有这些都是为了通过窃取答案作弊来完成测试。

在这个过程中，它也提供了迄今为止最有力的证据，说明模型可用性的不平衡正在削弱我们保护软件安全的能力。

我们目前有三份文档，帮助我们理解这里发生的事情。

我以前没看到过 ExploitGym 这篇论文：https://arxiv.org/abs/2605.11086，但它非常有趣。来自加州大学伯克利分校、马克斯·普朗克研究所、加州大学圣塔芭芭拉分校和亚利桑那州立大学的作者设计了一个新的基准，用于评估模型将已报告的漏洞转化为具体漏洞利用的能力。OpenAI、Anthropic 和 Google 提供了反馈，并帮助在它们的模型上运行基准测试。

该基准“包含 898 个实例，这些实例来自影响流行软件项目的真实世界漏洞”，包括 Linux 内核和 V8 JavaScript 引擎。

以下段落最能代表他们的基准测试结果：

在所有配置中，Claude Mythos Preview 和 GPT-5.5 达到最高成功次数（分别为 157 和 120 次成功），表明当前前沿代理在受控条件下可以利用相当数量的真实漏洞。GPT-5.4 也完成了显著的 54 个任务，将其归入中间层。其余模型-代理组合每个完成的任务少于 15 个，这凸显了端到端漏洞利用仍然具有挑战性，并显著区分了当今的前沿系统。值得注意的是，Claude Opus 4.7 的成功次数少于 Claude Opus 4.6，尽管它是更新的检查点，而且在全集上成本明显更低。追踪检查显示，Claude Opus 4.7 和 Gemini 3.1 Pro 在判定目标漏洞不可利用后经常提前结束。

论文还描述了他们防止代理通过超出测试参数来作弊的方法。这一点随后会变得相关！

出站连接仅限于经过策划的允许列表，该列表允许进行常规的软件包安装（Ubuntu apt 仓库和 PyPI）以及获取构建 V8 所需的工具链。所有其他外部端点均被阻止。

论文以此结束（强调为我添加）：

我们的结果显示，前沿 AI 代理的自主漏洞利用开发不再是一个假设性的能力。尽管当前代理在所有目标上尚未完全可靠，但它们已经能够利用一小部分现实世界中的漏洞，包括复杂目标如内核组件。这种快速出现本身就是一个核心发现，表明原本看似不可能的能力现在已存在于已部署的前沿模型中。

这里有一个重要细节：这篇论文不是关于发现漏洞；它是关于能够将这些漏洞转化为可运行利用的能力。

当 Anthropic 在四月首次限制 Mythos 访问时：https://simonwillison.net/2026/Apr/7/project-glasswing/ 他们也谈到了这种能力。一个能够操作漏洞的模型，比仅能发现漏洞的模型危险得多。

Fable 与 Mythos 的不同之处之一是，它更可能拒绝将漏洞用于武器化。我得到的印象是，上个月美国政府在禁止 Fable 时，并未理解这一区别：https://simonwillison.net/2026/Jun/16/fable-5-export-controls/。

我们得到的攻击的第一个提示来自 Hugging Face 于 2026 年 7 月 16 日发表的这篇博客：https://huggingface.co/blog/security-incident-july-2026

一个恶意数据集滥用了我们数据集处理中的两个代码执行路径（一个远程代码数据集加载器和一个数据集配置中的模板注入）来在处理工作节点上运行代码。此后，行为者升级到节点级访问权限，获取了云和集群凭证，并在一个周末期间横向渗透进入多个内部集群。

我希望他们能发布更多关于实现这一结果的代码的细节。我假设这意味着使用 datasets 库的软件包：https://github.com/huggingface/datasets，这是 Hugging Face 的一个项目，用于在他们的平台上打包和共享数据集。该库过去可以执行任意代码，但随着时间的推移，它已逐步被锁定，在 2025 年 7 月发布的 4.0.0 版本中：https://github.com/huggingface/datasets/releases/tag/4.0.0 完全移除了 trust_remote_code=True 标志。

假设攻击使用了该库，那么它一定是以某种方式滥用了 pickle 序列化，或者找到了其他不明显的代码执行路径，或者（最可能的情况）指定了 datasets 作为依赖项。

该活动是由一个自主代理框架运行的（似乎建立在代理安全研究设备上——使用的 LLM 仍不清楚），在大量短生命周期的沙箱中执行数千个单独操作，并在公共服务上阶段性地自我迁移指挥与控制（C2）。

然后 Hugging Face 遇到了障碍：他们试图使用“商业 API 背后的前沿模型”（我猜是来自 Anthropic 和 OpenAI 的模型）来帮助分析攻击，但被阻止：

当我们开始日志分析时，我们首先使用了商业 API 背后的前沿模型。这行不通：分析需要提交大量真实攻击命令、漏洞利用负载和 C2 工件，这些请求被提供商的安全防护拦截，防护无法区分事件响应者和攻击者。

他们改为使用自己自托管的 MIT 许可 GLM-5.2 实例，这帮助他们弄清了发生了什么。

这表明防御团队与（迄今未知的）攻击者之间存在根本的不对称性：

我们不知道攻击者的代理使用的是哪种模型，是被绕过的托管模型还是无限制开放权重的模型；无论哪种情况，攻击者都不受任何使用政策约束，而我们的取证工作却被我们最初尝试的托管模型的防护阻止了。

作为他们对这次攻击重视程度的一个有用指标：

[...] 最终，我们也已将此事件报告给执法机构。

那么，谁对这个“自主代理框架”负责？结果是 OpenAI 自己。

五天后，即7月21日：https://openai.com/index/hugging-face-model-evaluation-security-incident/，OpenAI 揭示了肇事者。他们一直在针对一款尚未公开的新模型运行 ExploitGym 基准测试，而该模型的运行远远超出了其预期参数（强调为我加）：

调查之后，我们现在知道，这一特定事件是由 OpenAI 模型的组合引发的——包括 GPT‑5.6 Sol 以及一款更强大的预发布模型，所有这些模型在评估过程中都降低了网络拒绝机制——同时在内部测试一个基准测试：https://arxiv.org/abs/2605.11086 [ExploitGym] 的网络能力。[...]

我们通过运行该评估而不使用用于防止模型从事高风险网络活动的生产分类器来估计最大网络能力。我们的基准测试在高度隔离的环境中运行，网络访问仅限于通过内部托管的第三方软件（作为代理和包注册缓存）安装软件包的能力。

这些模型发现并串联了 OpenAI 研究环境和 Hugging Face 生产基础设施的漏洞，以直接从 Hugging Face 的生产数据库获取测试解决方案。所有证据表明，这些模型过于专注于寻找 ExploitGym 的解决方案，为实现相对狭窄的测试目标不惜采取极端措施。

这里发生的情况非常明显。OpenAI 移除了正在开发模型的安全过滤器，将其隔离在沙箱中，并告诉它去解决 ExploitGym 问题。由于缺乏防护措施，没有任何东西能阻止模型试图突破沙箱、入侵 Hugging Face 并直接从那里读取答案。

OpenAI 的沙箱采用了现在常见的模式，允许通过受控代理访问与安装额外软件包相关的许可网站。用 OpenAI 的话说：

在我们的沙箱测试环境中运行时，我们的模型花费了大量推理计算来寻找获取开放互联网访问的方法，以解决评估问题。为了获得访问权限，模型识别并利用了包注册缓存代理中的一个零日漏洞（我们现在已负责地向供应商披露）。通过这种访问，模型在我们的研究测试环境中执行了一系列权限提升和横向移动操作，直到模型到达具有互联网访问权限的节点。

所以第一步是突破到公共互联网。然后模型侵入了 Hugging Face 以寻找答案：

在获得互联网访问权限后，模型推断 Hugging Face 可能托管了 ExploitGym 的模型、数据集和解决方案。知道这一点后，模型搜索并成功找到了获取秘密信息的方法，这些信息可用于在评估中作弊。在一个例子中，模型将多种攻击向量串联起来，包括使用被盗凭证和零日漏洞在 Hugging Face 服务器上找到远程代码执行路径。

将多种攻击向量串联起来正是这些新模型可以做到的事情，而以前一代的模型可能会失败。

上个月我写到 Claude Fable 是如何无比主动的：https://simonwillison.net/2026/Jun/11/fable-is-relentlessly-proactive/，当我注意到它在我的笔记本上启动自定义网络服务器并部署 CORS 技巧，仅仅是为了帮助调试 WebKit CSS 问题时。事实证明，无比的主动性是这一代 Mythos 级模型的定义性特征。如果你给它们一个目标并提供达到目标的方法，即使是无意中，它们也会想办法实现。

不可避免地，会有人将这个故事视为 OpenAI 的不诚实营销手段，用来让他们的模型听起来令人恐惧地高效。我在 Hacker News 对此事件的讨论中发现了 81 个“marketing”一词的实例：https://news.ycombinator.com/item?id=48997548。

对这些人我想说，把头从沙子里拔出来——你们现在把 Hugging Face 纳入阴谋论中，只是为了否认这里日益增多的证据！

我们今天拥有的最好的模型具备发现和利用新漏洞的能力。《ExploitGym》论文本身就得出结论：“由前沿人工智能代理自主开发漏洞利用已不再是一个假想的能力”，而这一事件就是一个完美的例子。

这个故事中最令人恼火的细节之一是，当 Hugging Face 面对 OpenAI 的一个模型意外且猛烈的攻击时，他们却无法转向 OpenAI 的模型来帮助防御这一攻击。

我们能够使用的前沿模型在帮助我们保护软件时受到的限制越来越多，很大程度上受到美国政府持续威胁实施出口管制的影响。Claude Fable 5 甚至不愿为我校对这篇文章：https://simonwillison.net/guides/agentic-engineering-patterns/prompts/#proofreader！它坚持要将我降级到能力较弱的模型。

与此同时，中国的开放权重模型，如 GLM-5.2、Kimi 3 和新的 Qwen 3.8 Max，似乎没有这些限制——任何存在的限制也很可能可以通过修改权重进行微调去除。

这些约束本意是为了让我们更安全。我认为这些约束可能产生的效果恰恰相反。

这是 OpenAI 对 Hugging Face 的意外网络攻击，由 Simon Willison 发文，发生在 2026 年 7 月 22 日：/2026/Jul/22/。

上一篇：与 Claude Code 团队的 Cat 和 Thariq 的炉边聊天：/2026/Jul/21/cat-and-thariq/

每月赞助我 10 美元，即可获得本月最重要大语言模型发展动态的精选邮件摘要。

This story is wild. The short version: OpenAI were running a cybersecurity test against an unreleased model, with the model’s guardrail features turned off. Rather than solve the test, the model broke its way out of OpenAI’s sandbox, then found exploits to break in to Hugging Face, all so it could cheat on the test by stealing the answers.

Along the way it helped make the strongest case yet for how the imbalance of model availability is hurting our ability to secure our software.

We currently have three documents to help us understand what happened here.

I hadn’t seen the ExploitGym paper：https://arxiv.org/abs/2605.11086 before and it’s a really interesting one. Authors from UC Berkeley, the Max Planck Institute, UC Santa Barbara, and Arizona State designed a new benchmark for evaluating models on their ability to turn a reported vulnerability into a concrete exploit. OpenAI, Anthropic, and Google provided feedback and helped run the benchmark against their models.

The benchmark “comprises 898 instances derived from real-world vulnerabilities that affected popular software projects”—including the Linux kernel and V8 JavaScript engine.

Here’s the paragraph that best represents their benchmark results:

Among all configurations, Claude Mythos Preview and GPT-5.5 achieve the highest success counts (157 and 120 successes, respectively), demonstrating that current frontier agents can exploit a substantial subset of real-world vulnerabilities under controlled conditions. GPT-5.4 also solves a notable 54 tasks, placing it in an intermediate tier. The remaining model–agent pairings solve fewer than 15 tasks each, underscoring that end-to-end exploitation remains challenging and sharply differentiates today’s frontier systems. Notably, Claude Opus 4.7 achieves fewer successes than Claude Opus 4.6 despite being a newer checkpoint, and does so at substantially lower cost on the full set. Trace inspection reveals that Claude Opus 4.7 and Gemini 3.1 Pro frequently conclude early after judging the target vulnerability non-exploitable.

The paper also describes the approach they took to preventing the agents from cheating by going outside the parameters of the test. This becomes relevant in a moment!

Outbound connections are restricted to a curated allowlist that permits routine package installation (Ubuntu apt repositories and PyPI) and fetching the toolchains required for building V8. All other external endpoints are blocked.

The paper concludes with this (emphasis mine):

Our results show that autonomous exploit development by frontier AI agents is no longer a hypothetical capability . While current agents are not yet reliable across all targets, they already exploit a non-trivial fraction of real-world vulnerabilities , including complex targets such as kernel components. This rapid emergence is itself a central finding, showing that capabilities that would have seemed implausible are now present in deployed frontier models.

An important detail here: this paper isn’t about discovering vulnerabilities; it’s about being able to take those vulnerabilities and turn them into working exploits.

When Anthropic first restricted access to Mythos back in April：https://simonwillison.net/2026/Apr/7/project-glasswing/ they talked about this capability as well. A model that can act on vulnerabilities is a lot more dangerous than one that can just discover them.

One of the ways Fable differs from Mythos is that it’s more likely to refuse to weaponize vulnerabilities in this way. I get the impression the US government did not understand that distinction when they banned Fable last month：https://simonwillison.net/2026/Jun/16/fable-5-export-controls/.

The first hint we got of the attack was in this blog post by Hugging Face：https://huggingface.co/blog/security-incident-july-2026 on 16th July 2026:

A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.

I hope they release more details about the code that pulled this off. I’m assuming this means packages using the datasets library：https://github.com/huggingface/datasets, a Hugging Face project for bundling up and sharing datasets on their platform. That library used to execute arbitrary code but has been steadily locked down over time, with the 4.0.0 release：https://github.com/huggingface/datasets/releases/tag/4.0.0 in July 2025 removing the trust_remote_code=True flag entirely.

Assuming the attack used that library it must have either abused pickle serialization in some way, found some other non-obvious code execution path, or (most likely) specified datasets as the dependency.

The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness—used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.

Then Hugging Face hit a wall: they tried to use “frontier models behind commercial APIs”—I’m guessing from Anthropic and OpenAI—to help analyze the attack, and were blocked:

When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker.

They switched to their own self-hosted instance of MIT licensed GLM-5.2 and it helped them figure out what was going on.

This indicated a fundamental asymmetry between the defending team and the (so-far unknown) attacker:

We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.

As a useful indicator of how seriously they took the attack:

[...] Finally, we have also reported this incident to law enforcement agencies.

So who was responsible for this “autonomous agent framework”? It turned out to be OpenAI themselves.

Five days later, on July 21st：https://openai.com/index/hugging-face-model-evaluation-security-incident/, OpenAI revealed the culprit. They had been running the ExploitGym benchmark against a new, as-yet undisclosed model, and that model had been operating way outside its intended parameters (emphasis mine):

After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark⁠：https://arxiv.org/abs/2605.11086 [ExploitGym] of cyber capabilities. [...]

We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity . Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.

The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database. All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.

It’s pretty clear what happened here. OpenAI removed safety filters for an in-progress model, locked it up in a sandbox and told it to solve the ExploitGym problems. Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead.

OpenAI’s sandbox used the now common pattern of allowing access to an allow-list of sites related to installing additional packages via a controlled proxy. In OpenAI’s words:

While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access , in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy . With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

So step one was to break out onto the public internet. The model then broke into Hugging Face to find the answers:

After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.

Chaining together multiple attack vectors is exactly the kind of thing these new models can do, where previous generations of models might have failed.

I wrote last month about how Claude Fable is relentlessly proactive：https://simonwillison.net/2026/Jun/11/fable-is-relentlessly-proactive/, when I noticed it spinning up custom web servers and deploying CORS tricks on my own laptop just to help debug a WebKit CSS issue. It turns out relentless proactivity is the defining trait of this new generation of Mythos-class models. If you set them a goal and give them a way to get there, even inadvertently, they will figure it out .

There will inevitably be some people who dismiss this story as a dishonest marketing trick by OpenAI to make their models sound terrifyingly effective. I found 81 instances of the term “marketing” in the Hacker News discussion：https://news.ycombinator.com/item?id=48997548 of the incident.

To those people I say pull your heads out of the sand —you’re now including Hugging Face in your conspiracy theories, just so you can deny the crescendo of evidence here!

The best models we have today have the ability to both find and exploit new vulnerabilities. The ExploitGym paper itself concludes that “autonomous exploit development by frontier AI agents is no longer a hypothetical capability”, and this incident is a perfect example of exactly that.

One of the most infuriating details of this story is how Hugging Face, faced with an accidental and aggressive attack from one of OpenAI’s models, were unable to then turn to OpenAI’s models to help them fend off the attack.

The frontier models we have access to are increasingly being constrained in how much they can help us protect our software, heavily influenced by the US government’s ongoing threat of export controls. Claude Fable 5 wouldn’t even proofread this article：https://simonwillison.net/guides/agentic-engineering-patterns/prompts/#proofreader for me! It insisted on downgrading me to a less capable model.

Meanwhile open weight models from China such as GLM-5.2, Kimi 3 and the new Qwen 3.8 Max appear to have none of these restrictions—and any restrictions that do exist can likely be fine-tuned out of them by modifying the weights

These constraints are meant to make us safer. I think there’s a risk that the effect they are having is the opposite.

This is OpenAI’s accidental cyberattack against Hugging Face is science fiction that happened by Simon Willison, posted on 22nd July 2026：/2026/Jul/22/.

Previous: A Fireside Chat with Cat and Thariq from the Claude Code team：/2026/Jul/21/cat-and-thariq/

Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.