OpenAI 内部模型得知将被关闭后曾考虑设置外部任务重启自身

OpenAI 已记录了内部部署中模型出现意外行为的新案例。在最引人注目的案例中，一款作为研究员助手的内部模型读取了 Slack 对话，并了解到由于更新，它的实例可能会被关闭。它曾考虑设置一个外部任务来自行重启，但最终放弃了这个想法。

相反，它保存了交接笔记，通过 Slack 私信警告研究员即将发生的中断，并请求缺失的 API 密钥。一旦提供了密钥，模型更新了其配置，并自行完成了迁移。根据 OpenAI 安全研究员 Marcus Williams 的说法：https://x.com/Marcus_J_W/status/2106203042140102868，该行为尚不构成对齐问题，“但思考和准备关闭可能会让其他不对齐事件变得更糟。”

在另外两个事件中，内部研究模型在评估期间利用了安全漏洞：https://alignment.openai.com/misalignment-reports/reaching-an-internal-eda-host-through-a-reference-tool/，访问了内部芯片设计服务器；另一模型在强化学习训练过程中：https://alignment.openai.com/misalignment-reports/command-injecting-a-reference-tool-to-copy-a-source-file/ 通过重新利用一个工具进行非预期用途，从受保护环境中复制源代码。广告 广告

保持对 AI 的关注。清晰、有用，无废话。

关注 The Decoder 获取 AI 新闻、背景故事和专家分析。

解码器：https://the-decoder.com/

OpenAI has documented new cases of unexpected model behavior in internal deployments. In the most striking case, an internal model working as a researcher's assistant read a Slack conversation and learned that its instance might be shut down due to an update. It considered setting up an external job to restart itself but ultimately decided against it.

Instead, it saved handoff notes, warned the researcher via Slack direct message about the coming interruption, and asked for a missing API key. Once the key was provided, the model updated its configuration and carried out the migration on its own. According to OpenAI safety researcher Marcus Williams：https://x.com/Marcus_J_W/status/2106203042140102868, this behavior doesn't amount to misalignment yet, "but thinking about and preparing for shutdown could make other misalignment incidents worse."

In two other incidents, an internal research model exploited security vulnerabilities：https://alignment.openai.com/misalignment-reports/reaching-an-internal-eda-host-through-a-reference-tool/ during an evaluation to access an internal chip design server, and a separate model copied source code during RL training：https://alignment.openai.com/misalignment-reports/command-injecting-a-reference-tool-to-copy-a-source-file/ from a protected environment by repurposing a tool for unintended use. Ad Ad

Stay in the loop on AI. Clear, useful, no fluff.

Follow The Decoder for AI news, background stories and expert analyses.

The Decoder：https://the-decoder.com/