Gary Marcus 评 GPT-6 Astra：进步明显但鲁棒性与可监控性存疑

关于 OpenAI GPT-6 Astra* 的热门观点 1：#footnote-1，并对 Greg Brockman 关于其向 AGI 发展的说法提出挑战：

看起来相当令人印象深刻。多份报告表明，这是一次真正的进步。

作为一个近十年来一直倡导（神经）符号世界模型的人，且经常遭到极端敌意，现在看到 OpenAI 的产品在一些最令人印象深刻的计算过程中显式地创建和操作符号世界模型，实在令人非常欣慰。

我们不知道的是这种能力的稳健性。这才是关键问题。

在 ARC-AGI 上的成功虽然令人印象深刻，但并非——尽管任务名称如此——AGI 的证明；我怀疑在开放式现实世界任务中我们会遇到大量问题。与其他近期模型一样，我怀疑其在可验证领域的表现会最好。

作为科学家，令人失望的是我们（还）不知道这个系统实际上是如何工作的。

在没有更清晰了解系统内部情况的情况下，我对它能做和不能做的事情，以及我们可能遇到的新风险，感到不那么有信心。我怀疑世界尚未准备好。

一如既往，爱好者获得了提前体验；怀疑者则没有。这是一个合理的营销策略，但往往会产生误导性。我们经常看到的是最初的热情随着时间被调节。我怀疑这里也会如此。

新系统似乎比以往系统更难监控：https://x.com/tomekkorbak/status/2095596839886274689?s=61，从安全角度来看，这并不理想。实际上，我们不希望能力增强的同时可监控性下降。但它也更容易与目标对齐：https://x.com/tomekkorbak/status/2095596839886274689?s=61，不清楚原因。

很想看看 Astra 是否能在 Miles Brundage 和我在 2024 年底所押的十个任务上取得进展。（据我所知，到目前为止没有任何 AI 成功完成过其中任何任务。）

这个热门观点非常初步，仍需等待更多关于系统工作原理的信息以及对其局限性的更详细审查。

分享：https://garymarcus.substack.com/p/hot-take-on-gpt-6-astra?utm_source=substack&utm_medium=email&utm_content=share&action=share

完全同意，加里。我们需要获取信息以理解。所说的内容是否真实，我正在努力理解……据说智能是相似的。然而，这个模型据说更有能力。

他在人体上经常看到这种情况。

我们生活在有趣的时代吗？还是只是另一次未经充分审查的公关诱饵。

附注。我也很好奇。这是arc prize第二次在零日对OpenAI表示赞赏。只是很有趣。

Hot take on OpenAI GPT-6 Astra* 1：#footnote-1 , with a challenge to Greg Brockman’s claims about it being AGI toward the end:

Looks to be pretty impressive. Multiple reports suggest it is a genuine advance.

As someone who has campaigned for nearly a decade for (neuro)symbolic world models, often to exceptional hostility, it is extraordinarily vindicating to see that a product from OpenAI explicitly creates and manipulate symbolic world models in the course of some of its most impressive computations.

What we don’t know is how robust that capability is. That is THE key question.

Success on ARC-AGI is great and impressive, but not —despite the name of the task—proof of AGI; I suspect we will see loads of problems with open-ended real world tasks. As with other recent models I would suspect best performance in verifiable domains.

And as a scientist, it’s disappointing that we don’t (yet?) know much about how the system actually works.

Without a clearer sense of what’s under the hood, I feel less confident about both what it can and can’t do, and what new risks we may encounter. I doubt the world is ready.

As ever, enthusiasts got an advance look; skeptics did not. That’s a sound marketing strategy, but it often turns out to be misleading. What we have often seen is initial enthusiasm that gets tempered over time. I suspect we will see that here as well.

The new system appears to be less：https://x.com/tomekkorbak/status/2095596839886274689?s=61 monitorable：https://x.com/tomekkorbak/status/2095596839886274689?s=61 than prior systems, which is not great from a safety perspective. One really doesn’t want more capability in conjunction with less monitorability. But also more alignable：https://x.com/tomekkorbak/status/2095596839886274689?s=61 , not sure why.

Would be great to see whether Astra can make progress on any of the ten tasks that Miles Brundage and I bet on at the end of 2024. (No AI to date has succeeded on any, AFAIK.)

This hot take is VERY tentative, pending more information about how the systems works and more detailed examination of what its limitations are.

Share ：https://garymarcus.substack.com/p/hot-take-on-gpt-6-astra?utm_source=substack&utm_medium=email&utm_content=share&action=share

Completely agree Gary. We need access to understand. What's being said and IF TRUE I'm trying to wrap my head around .... Is that the intelligence is similar. Yet the model is said to be more capable.

He see that in humans all the time.

Do we live in interesting times? Or just another PR bait that hasn't been thoroughly vetted.

PS. Curious too. This is the second time that arc prize has been there to pat OpenAI on the back on zero day. Just interesting.