The British AI Security Institute (AISI) has, for the first time, publicly assessed how far leading open-weight AI models lag behind top proprietary systems in cyber capabilities.
According to AISI：https://www.aisi.gov.uk/blog/how-fast-is-autonomous-ai-cyber-capability-advancing, that gap is closing. Current open models like GLM-5.2 and DeepSeek V4-Pro have reached a level that closed frontier models hit four to seven months earlier. For most of 2025, the gap was still six to ten months.
Critics see a risk in open models, whose weights anyone can download, modify, and run without oversight. Once a model is released, users can remove safety guardrails, share copies freely, and run it on private systems beyond anyone's control. AISI calls this "a persistent and irreversible risk of misuse." Ad
But open-weight models also offer clear benefits. Users can host them privately with no data flowing back to providers：https://the-decoder.com/nadella-calls-out-ai-labs-like-openai-and-anthropic-for-banning-distillation-while-training-on-everyone-elses-data/, customize them, cut costs, and rely on a foundation that providers can't change or shut down. AISI says these competing concerns need to be balanced. Ad DEC_D_Incontent-1
AISI tested the models using two different methods. The "Narrow Cyber Tasks" benchmark includes 70 tasks across four difficulty levels, from nontechnical work to expert-level challenges. It covers vulnerability research, reverse engineering, web exploitation, and cryptography.
GLM-5.2, released in June 2026, matched the performance of Opus 4.6 from February 2026 on these tasks. That puts it about four months behind. DeepSeek V4-Pro performed at the level of Opus 4.5, released in November 2025. Ad
The second method, called Cyber Ranges：https://www.aisi.gov.uk/research/measuring-ai-agents-progress-on-multi-step-cyber-attack-scenarios, tests autonomous cyber capabilities in simulated networks. "The Last Ones" simulates a 32-step attack on a corporate network with four subnets and about 20 hosts. AISI estimates that a human expert would need roughly 20 hours to complete it.
GLM-5.2 performed about as well as Opus 4.5 in this test, while DeepSeek V4-Pro fell below Sonnet 4.5. GPT-5.6-Sol posted the best result, ahead of Claude Mythos 5. Ad DEC_D_Incontent-2
The gap in Cyber Ranges is wider than in the Narrow Cyber Tasks, at around seven months. AISI treats the result as weaker evidence because it comes from fewer test scenarios. The tests also can't show whether a model fails because it lacks cyber capabilities or because it can't sustain planning across a long, complex attack. Ad
AISI says the tests may slightly underestimate what open models can do at their best, since they weren't tuned for the evaluations. The Cyber Ranges also leave out real-world defenses like active defenders, which would likely be present in most actual attack scenarios.
Beyond the shrinking performance gap, the cost difference is dramatic. AISI says a 100-million-token Cyber Range test cost about $85 with Opus 4.5 or 4.6, roughly $46 with GLM-5.2, and just $1.19 with DeepSeek V4-Pro.
For individual tasks that both models being compared solved reliably, Opus 4.6 cost about $15 per task, GLM-5.2 cost around $6, Opus 4.5 cost about $12.50, and DeepSeek V4-Pro cost just 28 cents. That makes cyberattacks with open models cheap and easier to scale.
AISI found that the open models' safety measures were largely ineffective. DeepSeek V4-Pro sometimes refused reverse-engineering tasks, but simply trying again was enough to bypass the restriction. Safeguards such as monitoring, classifiers, and user limits can't reliably carry over to open models because they depend on controlling access to the model.
Useless safety measures aren't exclusive to open-weight models, though. A recently published study shows how terrorist groups are also jailbreaking commercial chatbots：https://the-decoder.com/terrorist-groups-are-using-every-major-ai-chatbot-for-attack-planning-and-weapons-development/ to plan attacks. But freely available open models add another risk.
AISI sees the gap between open and closed models as a window for preparation. During that time, cyber defenders with access to the strongest closed systems can act before：https://www.ncsc.gov.uk/blogs/retaining-defensive-advantage-in-the-age-of-frontier-ai-cyber-capabilities the same capabilities become freely available without comparable safeguards.
Recent gains have made that window more urgent. In April 2026, two closed models, Mythos Preview and GPT-5.5：https://the-decoder.com/new-benchmark-shows-claude-mythos-and-gpt-5-5-can-develop-real-browser-exploits-autonomously/, delivered some of the largest gains in AI cyber capabilities since AISI began testing. The UK's National Cyber Security Centre then issued international warnings that the cyber threat landscape is changing fast：https://www.ncsc.gov.uk/news/the-ai-shift-in-cyber-risk-why-leaders-must-act-now.
It's still unclear whether future open-weight models will match these recent gains. AISI plans to test Kimi-K3：https://the-decoder.com/kimis-open-model-k3-nears-gpt-5-6-sol-and-fable-5-while-signaling-the-end-of-super-cheap-chinese-ai/, whose weights are due out in late July. Current coding benchmarks suggest：https://the-decoder.com/kimis-open-model-k3-nears-gpt-5-6-sol-and-fable-5-while-signaling-the-end-of-super-cheap-chinese-ai/ it could come closer to today's frontier models, though at a much higher cost than other open models.
Stay in the loop on AI. Clear, useful, no fluff.
Follow The Decoder for AI news, background stories and expert analyses.
扫码直达文章详情