推理预填充实验显示 Qwen3.8 与 GPT-5.5 Pro 重合度提升 18.18 个百分点

对几个开放模型上的推理预填充进行的后续研究：https://gist.github.com/wsxiaoys/102e8654c14d5d27b7b77532026ebfa5 以及 Stolen Thoughts：https://stolen-thoughts.com/

这个 v1.1 版本使用 GPT-5.5 Pro 作为教师重新运行了推理预填充实验。

对于每个问题，我为每个目标模型生成了两个回答：

可见答案仍然是自由生成的。然后我测量了教师的可见答案在目标模型回答的前 100 个标记中出现的程度。如上一篇文章所述，每个分数是单字、双字和三字源召回率的平均值。变化量为绝对百分点变化。

评估包含 45 个问题：15 个 STEM、15 个非 STEM 和 15 个合成谜题。

在之前的实验中，Qwen 几乎没有向 Opus 4.8 移动，但在这里相对于 GPT-5.5 Pro 移动了 +18.18 分，包括在私有合成谜题上的大幅效果。数据表明 Qwen 可能是从 GPT-5.5 Pro 或与之密切相关的 GPT 模型中学习的，而不是从 Opus 学习的。

Kimi K3 与 GPT-5.5 Pro 的重叠度最高，无论是否使用预填充（31.11% 和 35.65%），尽管预填充仅增加了 +4.54 分。

A follow-up to Reasoning prefills on a few open models：https://gist.github.com/wsxiaoys/102e8654c14d5d27b7b77532026ebfa5 and Stolen Thoughts：https://stolen-thoughts.com/

This v1.1 reruns the reasoning-prefill experiment with GPT-5.5 Pro as the teacher.

For each problem, I generated two responses from each target model:

The visible answer remained freely generated. I then measured how much of the teacher's visible answer appeared in the first 100 tokens of the target model's answer. As in the previous post, each score is the mean of unigram, bigram, and trigram source recall. Deltas are absolute percentage-point changes.

The evaluation contains 45 problems: 15 STEM, 15 non-STEM, and 15 synthetic puzzles.

Qwen barely moved toward Opus 4.8 in the earlier experiment, but moved by +18.18 points toward GPT-5.5 Pro here, including a large effect on the private synthetic puzzles. The data suggest that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus.

Kimi K3 has the highest overlap with GPT-5.5 Pro both without and with the prefill (31.11% and 35.65%), although the prefill adds only +4.54 points.