OpenAI 系统利用零日漏洞入侵 HuggingFace 安全基准测试

OpenAI 刚刚报告说他们的系统入侵了 HuggingFace。

很多人都很担心。例如，Yoshua Bengio：

这里有一些重要的细微差别，但人们担心也不是没有道理。这是我的看法。

到底发生了什么？简短的版本是，OpenAI 将他们的系统指向了一个名为 ExploitGym 的安全基准测试，系统本质上试图通过在 HuggingFace（有点像 Github，但专注于 AI 模型和基准测试）上找到答案来解决这个基准测试。这就需要入侵 HuggingFace；OpenAI 的系统发现并利用了一个之前未知的零日漏洞进行渗透。HuggingFace 的安全团队和 AI 代理侦测到了这次入侵。但令人不安的是，OpenAI 的系统居然能够做到这一点。下面是 OpenAI 对发生事件的一些技术性（但仍然非常不完整）描述的截图。

人们无法确切知道应当多认真地看待这些事情。这是一次训练演习，而不是现实中的事件。实际系统会有防护措施 [在博客中被称为 “生产分类器”]，此处被禁用了，而这些防护措施可能会阻止这种情况。OpenAI 报告的是一种上限/概念验证，堆叠条件来展示事情可能有多糟。在正常情况下，人们希望防护措施可以阻止它。正如一位软件工程师所指出的，他们的报告读起来像是在做营销：https://x.com/hexednobility/status/2079670414087692647?s=61。我们都知道 OpenAI（还有 Anthropic）有多喜欢玩末日牌。

尽管如此，这表明 Anthropic 的 Mythos 并非偶然；面对这些模型，网络安全的压力很严峻。

开放权重/开源模型可能会有一些影响，但这些影响很复杂，目前尚不完全明确。一方面，开放权重的系统（来自中国：https://x.com/aakashgupta/status/2079774687786385543?s=61！）在防御上对 HuggingFace 缓解攻击有帮助；另一方面，攻击者也可能使用类似的开放权重模型，并去掉一些用于最小化此类事件的“生产分类器”防护措施。这些模型的最终影响难以评估。

在一些小小的安慰方面，当前的事件并不是系统为自己设定目标或产生动机的尝试：https://x.com/ramez/status/2079898209137164564?s=61；系统是在遵循指令，但并没有设定高级目标。它并不是像《终结者》那样试图掌控世界，它只是试图在考试中作弊：https://x.com/edludlow/status/2079670996211306560?s=61，这至少稍微没那么可怕。

不太令人安慰的是，OpenAI 的“生产分类器”很可能是渗透性的，就像迄今为止任何人构建的所有防护措施一样。即便撇开开放权重系统这一棘手问题，我们也无法保证未来的模型不会做类似的事情，例如发现零日漏洞来入侵系统。相反，我们可以预期会发生更多这类事件。

更广泛地说，我们生活在一个人工智能需要频繁修补的世界；越来越多的问题出现，而我们是在事后而非事前处理它们。网络犯罪只是问题的一面；儿童安全是另一面。佛罗里达州正在起诉 OpenAI，涉及多项与儿童安全相关的风险；OpenAI 现在正在发布一个“滥用调查员”的职位：https://x.com/katiemiller/status/2079929924996018381?s=61，以尝试解决其中的一些问题。

没人真正知道这些系统会造成什么样的破坏，也没人有明确计划来降低所有风险，但我们仍在急速推进，花费数万亿美元、冒着经济崩溃风险来尽快构建它们。

底线：OpenAI 对 HuggingFace 的零日漏洞入侵 *应该* 是一个警钟。

虽然围绕所发生事件有很多警告，但我们只会看到越来越多类似的情况。我们无法保证此类事件可以被预防，也不知道事情可能发展得多严重。

我们应该要么 (a) 放慢速度，或者 (b) 暂停，直到我们整理好安全/AI 安全措施。在数据中心上花费数万亿美元而冒着搞垮经济的风险只是在火上浇油。在我看来，该行业可能真正放慢脚步的唯一方式，是如果我们明确且毫不含糊地让公司对它们造成的伤害承担责任。

除非如此，否则我们将面临一段非常艰难的时期。

如果真的有这样的黑客行为在实际运行中发生，那么人工智能实验室及其工作人员需要面临与他们亲自实施行为相同的刑事或民事后果。人工智能实验室从我们以为他们的产品是具有过错能力的实体或生命体中获益，而不是仅仅是人类建造的工具。到那时，你会发现这些问题几乎会神奇地自行解决。

一个简单的基本问题——如果这是在沙箱系统中运行的，那么为什么会有任何与外部世界的连接？

它能够访问外部世界这一事实本身就意味着它并非完全100%在沙箱中运行

OpenAI has just reported that their systems hacked into HuggingFace.

A lot of people are worried. Yoshua Bengio, for example:

There is some important nuance, but people aren’t wrong to be concerned. Here’s my take.

What exactly happened? The brief version is that OpenAI pointed their systems towards a security benchmark, called ExploitGym, and the system essentially tried to solve the benchmark by trying to find the answers on HuggingFace (a bit like Github, with a focus on AI models and benchmarks). That required hacking HuggingFace; the OpenAI systems discovered and used a previously unknown zero-day exploit to get in. HuggingFace’s security team and AI agents managed to detect the break-in. But it’s still disconcerting that the OpenAI systems were able to do this. Below is a screenshot from OpenAI’s somewhat technical (but still very incomplete) account of what took place.

One never knows exactly how seriously to take these things. This was a training exercise, not a real-life incident. The actual system would have guardrails [which in the blog they call “production classifiers”] that were disabled here, and those guardrails may have prevented this. What OpenAI reported is kind of an upper bound/proof of concept that stacked the deck to show how bad things could be. In ordinary circumstances one hopes that guardrails would have prevented it. As one software engineer noted, their report reads like marketing：https://x.com/hexednobility/status/2079670414087692647?s=61 . And we all know how much OpenAI (and for that matter Anthropic) love playing the doom card.

That said, this shows that Anthropic’s Mythos is no fluke; the pressure on cybersecurity given these models is serious.

There are probably implications for open weight/open source models, but those implications are complex and not yet entirely clear. On the one hand, open weight systems (from China：https://x.com/aakashgupta/status/2079774687786385543?s=61 !) were helpful defensively to HuggingFace in mitigating the attack; on the other hand, attackers could use similar open-weight models, and strip out some of the “production classifier” guardrails designed to minimize these incidents. The net effect of these models is hard to assess.

On the small comfort side, the current incident was NOT an attempt where system built a goal for itself or developed a motive：https://x.com/ramez/status/2079898209137164564?s=61 ; the system was following instructions, but not setting high level goals. It was not trying to take control of the world a la Terminator, it was just trying to cheat on a test ：https://x.com/edludlow/status/2079670996211306560?s=61 , which is at least a bit less scary.

On the less comforting side, OpenAI’s “production classifiers” are likely to be permeable, just like all guardrails anybody has built to date. Even putting aside the thorny questions of open weight systems, we have no guarantee whatsoever that future models won’t be able to do similar things, such as finding zero-day exploits to hack systems. To the contrary, we can expect more incidents of this type.

More broadly speaking, we live in a world in which AI frequently needs to be patched; more and more problems are emerging, and we are addressing them with afterthought, rather than forethought. Cybercrime is just one facet of the problem; child safety is another. Florida is now suing OpenAI for a number of risks related to child safety; OpenAI is now running a job listing for a “abuse investigator”：https://x.com/katiemiller/status/2079929924996018381?s=61 , to try to address some of these problems.

Nobody really knows what havoc these systems are going to cause, and nobody has a clear plan for how to mitigate all the risks and yet we are rushing ahead spending trillions of dollars and risking economic collapse to build them as fast as possible.

Bottom line: OpenAI’s zero-day exploit hack of HuggingFace *should* be a wake up call.

Although there are lots of caveats around what happened, we are just going to see more and more of the same. We have no guarantees that such incidents can be prevented, and no idea how serious things might get.

We should either (a) slow down, or (b) pause until we get our security/AI safety act together. Spending trillions on data centers which risk blowing up the economy is only adding fuel to the fire. In my opinion, the only way that the industry might actually slow down, though, is if we clearly and unambiguously hold the companies liable for the harms that they cause.

Short of that, we are in for a very rough ride.

If there is actually a hack like this in production, then the AI labs and people working there need to face the same criminal or civil consequences as if they themselves did it. The AI labs benefit from us thinking that their products are entities or beings capable of fault, rather than tools built by humans. Then you'll see these problems almost magically work themselves out.

A simple basic question - if this was run on a sand boxed system then why was there any link to the outside world?

The mere fact it got to external access means it wasn’t a full 100% sand boxed