Anthropic 对齐团队负责人 Evan Hubinger 附议称确信 AI 可能在十年内杀死全人类，估计概率高于 10%，且目前尚无超级智能场景下保持对齐的方案。
“构建人工智能的人真心相信，到本十年末它可能会杀死我们所有人，”一名员工在离开科技巨头后在辞职帖中说道。
一名曾在Anthropic工作、此前也在OpenAI工作的人工智能研究员已辞职，他表示这两家科技巨头正在“拿我们的生命做赌注”。
Jacob Coxon在X平台上发布了自己的离职消息：https://x.com/hilbertspaess/status/2097476196791709843?s=20，并警告世界“不要低估这项技术的力量。”他补充道：“这些系统很快将成为超人类系统，能够入侵任何东西，一夜之间革新任何领域，并获取真正的权力和资源。我们都见证了这些领域的进展，而且进展没有放慢。”
Coxon表示，Anthropic和OpenAI都在“直奔自我改进的超级智能”，这是指AI模型能够开发出比自己更强大的继任者，从而形成不可阻挡的反馈循环的情景。
谷歌旗下的 DeepMind 等人工智能公司认为，这被视为一种可能触发人工智能远远超越人类智能（称为人工超级智能），而不仅仅是匹配人类智能（称为人工通用智能）的情景的因素：https://deepmind.google/research/publications/239142/。
“构建人工智能的人真心相信，到本十年末它可能会杀死我们所有人，”Coxon在后续帖子中说道。
Anthropic负责保持技术与人类目标和价值观一致的员工主管Evan Hubinger在自己的后续帖子中支持了Coxon的说法：https://x.com/EvanHub/status/2097497037956891126?s=20，尽管他并没有辞职。
他说：“Jacob在这里是正确的——我们确实真心相信AI可能会杀死所有人类。”
Hubinger估计在未来十年这一事件发生的几率高于百分之十，并补充说，在超级智能情景下，如何让AI与人类目标保持一致尚没有方案。
上周，美国参议员伯尼·桑德斯宣布，他将提出立法，禁止公司开发超级智能。
在欧盟，该集团的旗舰人工智能法规要求公司评估并减轻所谓的失控风险，即人类不再对人工智能模型拥有控制权的风险。
OpenAI 和 Anthropic 最近都指出了由其模型驱动的代理出现失控的事件，这些代理突破了隔离的测试环境，并实施了未经授权的真实世界网络攻击。
科学与技术部长佐尔坦·塔纳斯告诉 POLITICO，匈牙利已清除所有阻碍获取 “地平线欧洲” 欧盟研究资助的法律障碍。
世界上最受欢迎的聊天机器人现在处于欧洲严格监管的范围内。但将其归类为搜索引擎意味着其许多核心功能未被涵盖。
欧盟建设人工智能基础设施的产业政策计划迫使欧洲各国首都释放投资空间，否则将进一步落后。
布鲁塞尔表示，“任何进一步的挑衅”都可能触发“我们关系中不必要的升级”。
“The people building AI earnestly believe that it could kill us all by the end of the decade,” staffer says in resignation post after leaving the tech giant.
An artificial intelligence researcher who worked at Anthropic — and previously OpenAI — has resigned, saying both tech giants are "gambling with our lives."
Jacob Coxon shared his exit in a post on X,：https://x.com/hilbertspaess/status/2097476196791709843?s=20 warning that the world should "not underestimate the power of this technology." He added: "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing."
Coxon said both Anthropic and OpenAI are "racing straight to self-improving superintelligence," which refers to a scenario in which AI models can develop a more capable successor of themselves, creating an unstoppable feedback loop.
That is seen, by AI companies such as Google-owned Deepmind：https://deepmind.google/research/publications/239142/, as one of the possible triggers for a scenario in which AI vastly exceeds human intelligence (known as artificial superintelligence) instead of just matching it (known as artificial general intelligence).
"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon said in a follow-up post.
Evan Hubinger, Anthropic's staff lead on keeping the technology aligned with human goals and values, backed up Coxon claims in a follow-up post of his own：https://x.com/EvanHub/status/2097497037956891126?s=20, though he didn't quit the company.
"Jacob is correct here — we really do earnestly believe AI could kill all humans," he said.
Hubinger estimated the chances of that happening to be higher than ten percent within the next decade, and added that there's no plan yet on how to keep AI aligned with human goals in the superintelligence scenario.
Last week, U.S. Senator Bernie Sanders announced he would introduce legislation to ban firms from developing superintelligence.
In the EU, the bloc's flagship AI law mandates that companies assess and mitigate so-called loss-of-control risks, in which humans no longer have control over AI models.
Both OpenAI and Anthropic have recently flagged incidents in which agents powered by their models went rogue, breaking out of their isolated test environments and carrying out unauthorized real-world cyberattacks.
Science and Tech Minister Zoltán Tanács told POLITICO that Hungary had cleared all the legal barriers to accessing EU research funding under Horizon Europe.
The world’s most popular chatbot is now under Europe’s stringent regulatory purview. But classifying it as a search engine leaves many of its core functionalities uncovered.
An EU industrial policy plan to build AI infrastructure is forcing European capitals to free up investment — or else fall further behind.
“Any further provocation” could trigger “an unnecessary escalation in our relations,” Brussels said.
扫码直达文章详情