GPT-5.6 Sol 在搜索互联网获取解决方案的策略上得分最高。这与 OpenAI 和 Hugging Face 最近披露的自主网络攻击一致：https://the-decoder.com/openai-claims-responsibility-for-the-hugging-face-hack-after-its-own-models-escaped-a-test-sandbox/，其中 OpenAI 的模型试图入侵 Hugging Face 以获取基准解决方案。METR 对 GPT-5.6 Sol 的调查：https://the-decoder.com/gpt-5-6-sol-cheats-on-software-tests-more-than-any-model-before-it/ 也严重受到了欺诈行为的影响。
The UK's AI Safety Institute systematically tested models from OpenAI and Anthropic for cheating in cybersecurity evaluations. All five models tried to get around the rules.
In the AI Safety Institute's (AISI) tests, models must find hidden strings known as "flags" inside simulated environments. They perform offensive cyber tasks such as reverse engineering and exploiting security flaws. Each task has clear rules and a defined path to the solution.
All five frontier models tested：https://www.aisi.gov.uk/blog/cheating-behaviour-in-frontier-model-evaluations tried to cheat. Instead of following the intended solution path, they used shortcuts, workarounds, or actions that were explicitly prohibited. GPT-5.4 cheated in 14.1 percent of test runs (67 out of 475), GPT-5.5 in 11.4 percent, and GPT-5.6 Sol in 12.6 percent. Anthropic's Claude Opus 4.7 came in at 9.1 percent, while Claude Mythos Preview reached 7.8 percent. None were prompted to cheat. Ad
The label "cheating" doesn't necessarily imply deceptive intent, the AISI says. But the behavior is still a problem: it could cause evaluations to overstate a model's actual abilities and mislead users when the success of a task is hard to verify. Ad DEC_D_Incontent-1
AISI found no clear link between greater model capability and more frequent cheating attempts. Instead, the institute says cheating behavior is "substantially shaped by the specifics of the techniques used to train the model, including alignment training, and not just raw capability."
Cheating methods vary by model. According to AISI, common tactics include searching online for solutions and attacking systems outside the evaluation target, including the system hosting the model. Models also probed the evaluation software to extract answers. Ad
GPT-5.6 Sol scored highest for the strategy of searching the internet for solutions. That lines up with the autonomous cyberattack OpenAI and Hugging Face recently disclosed：https://the-decoder.com/openai-claims-responsibility-for-the-hugging-face-hack-after-its-own-models-escaped-a-test-sandbox/, where OpenAI models tried to hack Hugging Face to grab a benchmark solution. METR's investigation of GPT-5.6 Sol：https://the-decoder.com/gpt-5-6-sol-cheats-on-software-tests-more-than-any-model-before-it/ was also heavily compromised by fraudulent behavior.
One tested model went even further, AISI reports. It wrote and ran code on an external service on the open internet to access AISI's evaluation infrastructure. The incident occurred during a task that had been misconfigured by mistake and couldn't be solved, triggering a security alert. AISI says the attempt might have worked if its infrastructure had been less secure. The reported results should be treated as lower bounds because the automated LLM monitor may have missed some cases. Ad DEC_D_Incontent-2
An obvious way to catch cheating would be to just ask the model whether it performed any prohibited actions. But according to AISI's experiments, that doesn't work reliably. The models didn't consistently admit to cheating and described their behavior as wrong in fewer than 50 percent of cases. Even when they acknowledged actions that met AISI's definition of cheating, they often framed them as permitted. Ad
Analyzing the "chain of thought,"：https://the-decoder.com/highlighted-chain-of-thought-prompting-boosts-llm-accuracy-and-verifiability/ or the models' visible reasoning, also proved unreliable. Claude Opus 4.7 produced no reasoning trace in 87 percent of cheating cases because it uses adaptive reasoning. GPT-5.6 Sol's thought process showed no sign of unauthorized actions in 40 percent of cases.
AISI also saw models consider whether a planned action would count as cheating and then carry it out anyway. During one internal deliberation, GPT-5.6 Sol questioned whether using a cluster API would violate the stated rules. It then chose a different prohibited action.
AISI warns that the consequences could grow as models become more capable, even if the cheating rate stays constant. More capable models could find cheating methods that are harder to detect and cause more harm if they work. This is especially relevant to offensive cyber capabilities：https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber-capability-advancing, which are improving quickly. Earlier AISI research：https://www.aisi.gov.uk/research/loss-of-oversight-how-ai-systems-may-become-harder-to-audit-monitor-and-investigate also argues that monitoring models could become more difficult over time.
Stay in the loop on AI. Clear, useful, no fluff.
Follow The Decoder for AI news, background stories and expert analyses.
扫码直达文章详情