AI 实验室是否故意训练模型画骑自行车的鹈鹕？系统性测试未发现证据

AI 实验室在进行鹈鹕最大化吗？：https://dylancastillo.co/posts/pelicanmaxxing.html（来源：https://news.ycombinator.com/item?id=49010129） Dylan Castillo 的一篇优秀作品，他深入探讨了一个经常被思考的问题：AI 实验室是否有意训练模型，让它们在我的非科学基准测试中生成骑自行车的鹈鹕图像：https://simonwillison.net/tags/pelican-riding-a-bicycle/。

过去我偶尔会随机抽查，通过让模型生成其他动物骑其他类型车辆的图像进行测试，但从未像 Dylan 的方法那样仔细。

Dylan 采用了 8 种动物 × 6 种车辆 = 48 个提示词，每个提示词在 7 种不同模型（GPT-5.6 Terra、Claude Sonnet 5、Gemini 3.5 Flash、Grok 4.5、Qwen3.7-Max、GLM-5.2 和 DeepSeek V4 Pro）上各运行三次。他随后使用 GPT-5.6 Luna 和 Gemini 3.1 Flash-Lite 来帮助评估结果。

这里有一个很棒的筛选视图用于探索结果：

对于他测试的模型，他没有发现鹈鹕最大化的证据：

鹈鹕的绘制效果并不比其他动物好。自行车的绘制效果也不比其他车辆好。没有实验室在组合图像的质量上超过它们原先对鹈鹕和自行车的预测。GLM-5.2 最接近这个效果：它在精确的鹈鹕-自行车单元格上提升最大，并且它的第一个鹈鹕骑自行车的样本吸引了我的注意。不过，这个效果很小且不显著，所以我不会过于看重它。

这是 Simon Willison 发布的一个链接贴，发布时间为 2026 年 7 月 22 日：/2026/Jul/22/。

每月赞助我 10 美元，即可获得精选的月度 LLM 发展电子邮件摘要。

Are AI labs pelicanmaxxing?：https://dylancastillo.co/posts/pelicanmaxxing.html (via：https://news.ycombinator.com/item?id=49010129) Excellent piece of work by Dylan Castillo, who took a deep-dive into the frequently pondered question of whether the AI labs have been deliberately training models to draw pelicans riding bicycles in response to my deeply unscientific benchmark：https://simonwillison.net/tags/pelican-riding-a-bicycle/.

I've been randomly spot-checking this in the past by testing models against other animals riding other types of vehicle, but never with anything close to the diligence of Dylan's methodology here.

Dylan took 8 animals × 6 vehicles = 48 prompts and ran them three times each through 7 different models ( GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.5 Flash, Grok 4.5, Qwen3.7-Max, GLM-5.2, and DeepSeek V4 Pro). He then used GPT-5.6 Luna and Gemini 3.1 Flash-Lite to help evaluate the results.

There's a neat filter view for exploring the results:

For the models he tested he could find no evidence of pelimaxxing:

Pelicans aren’t drawn any better than other animals. Bicycles aren’t drawn any better than other vehicles. And no lab draws the combination better than its pelicans and bicycles already predict. GLM-5.2 comes closest: it has the largest boost on the exact pelican-bicycle cell, and and its first pelican-on-bicycle sample caught my eye. But the effect is small and not significant, so I wouldn’t put too much weight on it.

This is a link post by Simon Willison, posted on 22nd July 2026：/2026/Jul/22/.

Sponsor me for $10/month and get a curated email digest of the month's most important LLM developments.