我们谈论多智能体、纳维-斯托克斯方程，以及目前数学进展的爆炸性增长告诉我们自动化 AI 研究之后会发生什么。
我们还讨论了在启动递归自我改进（RSI）之前，如何确认模型实际上是对齐的。
在 YouTube 上观看：https://youtu.be/6AgOfiZOWiY；在 Apple 播客上收听：https://podcasts.apple.com/us/podcast/noam-brown-agent-swarms-alignment-recursive-self-improvement/id1516093381?i=1000790373289 或在 Spotify 上收听：https://open.spotify.com/episode/3ngDaNm2UVDH0BsMUYVwcG?si=E_7EYGCoTn-rJbMUJLR7CQ。
Jane Street：https://janestreet.com/dwarkesh 对 AI 的兴趣比你想象的要早得多，而且不仅仅是为了交易。早在 2011 年，比 AlexNet 提前整整一年，比 ChatGPT 上线提前十多年，他们就举办了第一次 FOOM 辩论，讨论 Eliezer Yudkowsky 和 Robin Hanson 关于 AI 是否会导致智力爆炸的观点。现在 Jane Street 正在通过一个新的小组重新审视这个问题：Daniel Kokotajlo、Ege Erdil、Ryan Greenblatt 和 Jaime Sevilla，并由 Ron Minsky 于今年十月在旧金山主持。我预计这将是一场非常精彩的对话。注册链接：https://janestreet.com/dwarkesh janestreet.com/dwarkesh：http://janestreet.com/dwarkesh
Grok Bot：https://x.ai/bot 已经让分配工作变得非常容易。它运行在自己的云计算机上，在那里安装处理任务所需的工具，实现端到端处理。在播客中，我们使用 Grok Bot 来帮助制作我们的视频。你可能注意到我们的广告展示了真实网站的动画效果。以前要实现这些像素级精确效果，意味着我们需要运行复杂的多步骤工作流程。现在我们只需让 Grok Bot 来处理这一切。最棒的是，Grok Bot 已经掌握了我们所有的规格和偏好，所以我们不必每次都重新描述任务！自己试试 Grok Bot：https://x.ai/bot x.ai/bot：http://x.ai/bot
Antithesis：https://antithesis.com/dwarkesh 给你一种像大型测试套件一样的自信，而实际上无需编写测试。比如你正在进行一次重大后端重构：构建足够的测试以建立信任可能需要几周时间。Antithesis 通过在无数模拟世界中运行你的软件、注入故障并寻找失败来解决这个问题。在任何 PR 上，你可以调节旋钮来决定你想要进行多少测试。而且由于每次运行都是完全确定性的，当出现错误时，代理可以立即分支，倒回错误，检查内存并重放，同时原始测试仍在运行。更多信息请访问： https://antithesis.com/dwarkesh antithesis.com/dwarkesh：http://antithesis.com/dwarkesh
(00:00:00) – 多智能体与纳维-斯托克斯
(00:15:28) – AI公司将如何运作？
(00:22:02) – 数学进展告诉我们关于递归自我改进的什么信息
(00:40:22) – Hugging Face 与对齐
(01:01:18) – 内部/外部模型差距
(01:08:34) – 思维链正在退化
(01:14:12) – 我们如何才能知道对齐问题已解决？
今天，我正在与 Noam Brown 聊天：https://x.com/polynoamial，他是 OpenAI 的研究员。他是后来成为 o1：https://en.wikipedia.org/wiki/OpenAI_o1 及推理模型的基础性贡献者之一。现在他正在研究多代理系统：https://en.wikipedia.org/wiki/Multi-agent_system。顺便提一下，你们上周宣布：https://openai.com/index/navier-stokes-solution/ 你们用一个由 10,000 个不同 AI 代理：https://www.ibm.com/think/topics/ai-agents 组成的系统，用 1300 亿个 token 运行 88 小时，解决了千禧年大奖难题之一：https://en.wikipedia.org/wiki/Millennium_Prize_Problems。
我对与你交谈感兴趣的原因之一是，你可能是两三年前首批思考推理模型将如何使我们能够洞察未来的人之一。因为如果你扩大推理：https://cloud.google.com/discover/what-is-ai-inference 计算能力，你就可以看到模型在几年后的基础能力将会是什么样的。
我觉得你现在处于一个类似的位置，可以帮助我们理解未来能力会是什么样子，考虑到我们目前能够实现的代理规模的巨大扩展。
我的想法是，当你在 x 轴上绘制这些推理模型的测试时计算量：https://openai.com/index/learning-to-reason-with-llms/，在 y 轴上绘制几乎任何推理基准的表现时，你会看到一个非常清晰的模式：模型花的时间越长来思考答案，它们的表现就越好。这是很自然的事情。人也是一样。如果你参加 SAT 考试，只给你五分钟完成整个试卷，你不会做得很好。如果你有五个小时，你可能会做得好多了。
AI 模型非常相似。它们会花时间自己进行独白，弄清楚问题，考虑不同情况，排除不同可能性，并在之前的发现基础上继续构建。
问题是，当你进一步推进时，你会遇到延迟瓶颈。你不会想花三年时间等待一个回应。所以你可以做很多人做的事。他们将任务并行化。就是组建一个团队。如果你要创办公司，你会想要聚集一群人，这样你可以更快推进。这和这些 AI 模型是一样的。让多个代理共同工作会更快，这很有帮助。
因此，多代理是一种并行扩展测试时计算的方法，而不是完全串行的。它效率较低，因为不像单个代理可以独自拥有所有上下文。但如果执行得当，这是扩展测试时计算的一种非常有效的方式。
我将提出一堆天真的问题。这是一个未发布的模型，所以我们还没公开看到这些系统是如何工作的。我只是有很多地方对这种系统的定性特性感到困惑。
我对你能在如此短的时间内集中如此大规模的认知努力感到震惊。想想看 1300 亿个 token：https://blogs.nvidia.com/blog/ai-tokens-explained/ 是什么。如果是一个人以全职工作方式思考，连续不断地思考，1300 亿个 token 相当于一个人连续思考 4000 年。每天工作八小时，按正常工作周计算。从古代苏美尔一直到今天，一个人顺序思考这么久，集中在 88 小时内。
我觉得在质上，这是一个非常重要的考虑因素。我惊讶于为什么并行化惩罚没有更大。你可以让 10,000 个代理协作。也许是因为这些代理比人类更擅长协作，它们能跑得更快。它们实际上可以在如此大规模下高效协作。或者，也许并行化惩罚确实很大。
让我们先谈谈并行化惩罚，然后再谈质上的问题。事实是，我们对于多代理系统扩展到这种规模的科学研究还不够完善。当我们发布 5.6：https://openai.com/index/previewing-gpt-5-6-sol/ 时，我认为那是我们模型中第一次有了真正的多代理系统。我们实际上在博客文章中展示了一些多代理系统的扩展性能图，因为我们提供了这个选项。这是超模式（Ultra Mode）。默认是四个代理，但你可以设置更多。
在图中，我们展示了在一些基准测试上一个代理、四个代理一起工作、十六个代理一起工作时的性能表现。这取决于具体基准测试，但在某些基准上，你会看到如果四个代理一起解决问题，速度会快两倍。因为有四个代理工作时间减半，你付出两倍的代价就能获得两倍快的答案。如果增加到十六个代理，你会看到类似的模式。效率略低一些，但性能依然持续提升。
随着并行代理数量增加，是线性序列时间加速还是亚线性加速？
它稍微是亚线性的，但很大程度上取决于具体问题。比如数学，其实很容易并行化。虽然不是最可并行化的，但非常可并行化。比如做深度研究：https://openai.com/index/introducing-deep-research/ 报告，需要查阅大量资料，是非常可并行的。我怀疑写小说这种事情几乎无法并行化。你可能不会觉得有一万名经纪人一起写小说会带来太大好处，就像一万人一起写小说也不会带来太大好处一样。
所以性能确实取决于领域。我们在发布的博客文章中会测量大约16个代理。问题是，要把这项科学推广到1万个代理非常困难，因为成本太高了。
你们只是一个周末就做完了。
但这只是一个数据点。我们不知道单个智能体需要多长时间才能解出纳维-斯托克斯定理：https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existence_and_smoothness，因为我们还没做过那个实验。也许我们会做，但那也只是一个数据点。
如果我们想做一个彻底的消融实验：https://en.wikipedia.org/wiki/Ablation_(artificial_intelligence)，在那个规模下实验的成本实在太高。所以我们必须采取某种系统性的方法来研究当数量增加到64、128、256或类似规模时会发生什么，并了解其行为。但要将实验一直推进到10,000，并确切知道使用10,000个代理相比1,000个代理实际带来了什么好处，将会非常困难。
我想澄清一件事。解决千年奖问题的努力，并不是多智能体的贡献。我甚至不会把10%的功劳归给多智能体。现实是，OpenAI已经训练了一个非常强大的模型。我们可以让这个模型在很长的时间范围内运作。我们可以让它并行思考。
但归根结底，我们之所以能做到这一点，是因为我们有一个通用且非常强大的模型。像多智能体这样的技术很炫目且新颖，这可能因此获得了不成比例的认可。但核心原因是这是一个非常强大的模型。
这个概括对我来说相当令人震惊。我不知道这些系统是如何训练的，但可以推测它们是按照强化学习的训练方式进行训练的：https://en.wikipedia.org/wiki/Reinforcement_learning。你有一堆可检验的合成问题，然后你对它们进行大量强化学习。在训练过程中，我猜测，模型从未解决过像千禧年大奖问题这样雄心勃勃的问题。但泛化能力足够强，使得这些更容易验证的问题能够推广到在如此困难的问题上进行平行努力。
我认为这是对的。首先，我们确实在非常困难的问题上训练模型。确实存在差距。我们看到，如果我们在某些任务上训练，它能够完成比这些更雄心勃勃的任务。
有一个有趣的挑战是，随着模型变得越来越智能，我们能向它提出的许多问题都显得过于简单。很难对模型提出挑战。我确实认为这将会很有趣。如果我要提出一个论点来解释为什么你可能不会看到像大型语言模型（LLM）https://en.wikipedia.org/wiki/Large_language_model 这样的人工智能走上与 AlphaGo（https://en.wikipedia.org/wiki/AlphaGo ）和 AlphaZero（https://en.wikipedia.org/wiki/AlphaZero ）以及所有这些游戏人工智能相同的路径，可能就是这种问题。
在像 AlphaZero 这样的系统中，自我对弈（https://en.wikipedia.org/wiki/Self-play_(reinforcement_learning_technique)）为你提供了一个无限课程（https://en.wikipedia.org/wiki/Curriculum_learning）。你总是在与一个同样强大的 AI 对弈。而对于像使用强化学习训练大型语言模型这样的事情，至少目前现有的方法是，你给模型一个问题，并让它去解决它。如果问题太简单，它能在一秒钟内解决，那么它实际上没有学到任何东西。
如果我们用尽了用来挑战它的问题，那么这就是一个很可能的情境，在这种情况下取得进展会变得更难。现在，我确实认为有解决方法。我们还没有真正遇到这种障碍。我认为，如果它真的成为一个严重问题，总会有解决方法。但是，这确实是一个可能的情境。
仅供观众参考，当你提到 AlphaGo 或 AlphaZero 时，你是在谈论在达到人类水平性能后，相对较快地获得超人能力。
如果你观察游戏 AI 的发展轨迹，比如围棋：https://en.wikipedia.org/wiki/Go_(game)，在一年时间内，它们从击败欧洲冠军：https://en.wikipedia.org/wiki/AlphaGo_versus_Fan_Hui —— 世界排名大约第 50 —— 到击败世界冠军：https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol，再到强大到难以想象，远远超过任何在世人类。这在数学等领域可能也会出现类似的轨迹，但我认为存在完全不发生这种情况的非常合理的可能性。
我想了解的是，如果在六个月内人们将能够使用多智能体系统，应该如何建模与多智能体系统合作或雇用它的体验？
我应该先谈谈这些多智能体系统实际上是如何工作的，我认为它的工作方式与许多其他 AI 中的多智能体系统非常不同。很多接触过 LLM 等多智能体的人往往采取一种非常分层的方法。例如，可能有一个协调者智能体，将任务分配给多个子智能体。子智能体完成任务后返回答案。
这看起来是一个非常合理的设置，一个非常合理的分层结构。它确实有帮助，但这种设置存在很多限制。例如，如果在这种设置中你有一个协调者发送任务给子智能体，而子智能体完成后返回答案，如果两个子智能体被分配了相似的任务，会发生什么？他们能互相交流吗？答案通常是否定的。这非常低效。
如果你被分配了一个任务，而实际上能与可能知道问题答案或正在处理相关部分的人交流会很有帮助，那么你只需能够向他们发消息说，“嘿，你能帮我这个忙吗？”就会非常有帮助。但很多系统没有这种设置。添加这种功能会显著增加你所使用的分层系统的复杂性。
另一个问题是，如果孩子真的不理解或者有澄清问题怎么办？那时他必须在“好吧，我是直接回去问问题而不是解决问题吗？”或者“我应该解决问题，并对父母希望我做的事情做一个假设，然后就这样解决吗？”之间做选择。在人们设计的任何支架中，总会存在局限性。我们想采取的方法是尽量减少结构，给代理非常原始的工具使用，让他们自己想办法有效地使用它们。因此我们给代理提供了向其他代理发送消息的能力，当它向另一个代理发送消息时，这条消息会被插入到上下文中。它还可以做几件其他类似的事情，但这基本上就是核心。它可以随时发送消息——只需一个工具调用——并且可以将消息发送给其他代理。
他们自己会找到围绕这些消息进行协调的最佳方式。事实证明，如果做得好，你会看到非常复杂的行为。对我而言，这很像人类合作伙伴在像Slack这样的工具上工作的方式。
当我们在做这个项目时，当这些代理最终能够协同解决问题时，真的非常令人兴奋。我记得有一个例子。我们给代理一个问题，然后一个代理说，“我认为我找到了答案。”然后另一个代理说，“实际上，我得到了不同的答案。”接着他们开始讨论，“好吧，你是怎么得出这个答案的？能解释一下吗？”来回地尝试澄清彼此的推理中可能出错的地方。
然后他们最终达成一致，“哦，是的，好吧，这似乎是对的。”然后它就向其他代理广播，“实际上，我改变了我的答案，我认为他是对的。”这感觉就像一次非常自然的对话。
感觉就像第一次看到通过强化学习训练出来的思维链时：https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/，你会想，“哦，这就像一个人在写下他们思考的过程时会怎么想一样。”就是这种感觉。看到这种行为真的很酷。与这些系统合作，老实说，感觉很像和一个人合作。合作的流程非常自然。
唯一的一个质的差别，未来可能会变得明显，那就是这些系统的思考速度可能比人类快十倍以上，仅仅看它们每秒输出的标记数相比人类说话的速度就能看出来。它们一直在工作，不会睡觉。它们之间的协作节奏远高于人类与人类之间的协作能力。
我在想一年后在质上会有什么样的变化。是不是像一个影子组织，在我的公司中运行速度比人类水平快100倍？一个人类组织需要一年的事情，在这个影子组织里可能一周就能完成？
会感觉很陌生吗？我不知道。实际上，我目前发现与这些系统合作出奇地自然。我认为情况可能会改变。比如，我们有这些超快速模式，可以让抽样速度达到10-15倍或更快。这时，要跟上这些系统会相当困难。这个想法是，这些智能体在彼此交流时，速度可以非常快。但它们也能区分是在与另一智能体交流还是与人交流，并且在不同情况下行为会有所不同。
我们公开拥有的复杂多智能体系统的主要例子不幸的是 Hugging Face 的这个： https://www.dwarkesh.com/p/openai-huggingface。那里我发现了很多令人担忧的地方，显而易见。但我发现有趣的是等级制度和中层管理的自发出现。听起来你是说这种层次的组织是从训练中自发产生的吗？
细节是自发的。但在我们给予代理很多灵活性来决定如何以最佳方式相互沟通的同时，我们仍然给了他们一个起点。我们给了他们关于合理沟通可能是什么样的先验。他们也接受了大量人类文本的训练。他们理解人类如何组织和协调，所以这些都已经内置了。
我认为他们能够磨练这一点的方式令人惊讶。如果你看看它最初的表现，并不是非常复杂的行为。事实上，让这些代理以富有成效的方式协调是非常困难的，因为他们很容易就会陷入“哦，我们都只是独立解决问题”这种状态。这是一种局部最优：https://en.wikipedia.org/wiki/Local_optimum，你可能会被困在其中。但如果做得好，他们最终可以以这些非常结构化的方式进行非常有效的协调。
我几年前写过这篇关于自动化公司的文章：https://www.dwarkesh.com/p/ai-firm。我在思考，如果你拥有完全自动化的公司，比如说具有人类水平智能的公司，AI大脑的本质有什么不同，这会使AI形成的组织结构有何不同？有几个非常重要的区别。例如，AI可以比人类更无缝地共享上下文。他们可以更无缝地整合知识。此外，你可以启动或关闭任意数量的具有正确知识的实例。
所以如果你想雇佣更多的人，不需要像找到合适人才那样辛苦。你最优秀的人才，你可以无限复制他们。或者，如果任务不再需要他们，你可以关闭他们。你可以复制组织中最有效的部分，或者把整个有效的组织一起复制。你觉得这些多代理系统一到两年后会发展到什么程度？”
这是一个很好的问题：这些东西实际上与人类同事的工作方式有何不同？你提到了一些。一个非常有趣的事情是，如果你有一个人并且希望拥有两份他的副本，你不能只是克隆这个人。但对于人工智能来说，实际上非常容易，只需说“好吧，自我分叉”，然后让两个副本一起处理这个事情，然后再合并回去。我认为我们已经在Astra的多智能体中有这种情况：https://en.wikipedia.org/wiki/GPT-6_Astra 和5.6 Sol，当它们启动子代理时，上下文只是被分叉。因此它拥有所有相关的上下文。
还有其他有趣的方式，智能体会与人类不同。例如，初创公司为什么能颠覆现有企业：https://en.wikipedia.org/wiki/Disruptive_innovation？有几个因素。其中一个是他们愿意承担更多风险。但另一个主要因素是，随着组织规模的增长，你会看到组织内的个人之间的对齐度越来越低。
如果你有一个五个人的初创公司，每个人拥有公司20%的股份，他们都会高度一致地关注公司成功。如果你拥有一个拥有1万人规模的大公司，你会看到更多的情况：人们会具有领地意识，或者只是关心为他们的项目或团队争取大量人员，建立他们的小王国，获取大量资源，以便他们能发表很酷的成果或晋升。这实际上是一个真实的负面因素。我认为这解释了为什么初创公司能够颠覆现有企业的许多原因。
We talk about multi-agent, Navier-Stokes, and what the current explosion of maths progress tells us about what happens once you automate AI research.
And we also discuss how we will know if the models are actually aligned before we kick off RSI.
Watch on YouTube：https://youtu.be/6AgOfiZOWiY ; listen on Apple Podcasts：https://podcasts.apple.com/us/podcast/noam-brown-agent-swarms-alignment-recursive-self-improvement/id1516093381?i=1000790373289 or Spotify：https://open.spotify.com/episode/3ngDaNm2UVDH0BsMUYVwcG?si=E_7EYGCoTn-rJbMUJLR7CQ .
Jane Street ：https://janestreet.com/dwarkesh has been interested in AI for a lot longer than you’d think, and not just for trading. In 2011, a full year before AlexNet and over a decade before ChatGPT launched, they hosted the first FOOM Debate between Eliezer Yudkowsky and Robin Hanson on whether AI would lead to an intelligence explosion. Now Jane Street is revisiting the question with a new panel: Daniel Kokotajlo, Ege Erdil, Ryan Greenblatt, and Jaime Sevilla, hosted by Ron Minsky in San Francisco this October. I expect it to be a truly excellent conversation. Register at ：https://janestreet.com/dwarkesh janestreet.com/dwarkesh ：http://janestreet.com/dwarkesh
Grok Bot ：https://x.ai/bot has made handing off work super easy. It runs on its own cloud computer, where it installs the tools it needs to handle tasks end-to-end. For the podcast, we use Grok Bot to help produce our videos. You may have noticed that our ads feature animations of real websites. Getting these pixel-perfect used to mean running a convoluted, multi-step workflow ourselves. Now we just let Grok Bot handle it. Best of all, Grok Bot has learned all of our specs and preferences, so we don’t have to redescribe the task each time! Try Grok Bot for yourself at ：https://x.ai/bot x.ai/bot ：http://x.ai/bot
Antithesis ：https://antithesis.com/dwarkesh gives you the confidence of a giant test suite without actually having to write one. Say you’re doing a major backend refactor: building enough tests to trust it could take weeks. Antithesis solves this by running your software through countless simulated worlds, injecting faults and hunting for failures. On any PR, you can turn a dial to decide exactly how much testing you want. And because every run is fully deterministic, agents can branch off the moment a bug appears, rewind it, inspect memory, and replay it, all while the original test keeps running. Learn more at ：https://antithesis.com/dwarkesh antithesis.com/dwarkesh ：http://antithesis.com/dwarkesh
(00:00:00) – Multi-agent and Navier-Stokes
(00:15:28) – How will AI firms work?
(00:22:02) – What math progress tells us about recursive self improvement
(00:40:22) – Hugging Face and alignment
(01:01:18) – The internal/external model gap
(01:08:34) – Chain of thought is degrading
(01:14:12) – How will we know when alignment is solved?
Today, I’m chatting with Noam Brown ：https://x.com/polynoamial , who is a researcher at OpenAI. He was one of the foundational contributors to what became o1 ：https://en.wikipedia.org/wiki/OpenAI_o1 and the reasoning models. Now he’s working on multi-agent systems ：https://en.wikipedia.org/wiki/Multi-agent_system . Speaking of which, you guys announced last week ：https://openai.com/index/navier-stokes-solution/ that you solved one of the Millennium Prize Problems ：https://en.wikipedia.org/wiki/Millennium_Prize_Problems with a system of 10,000 different AI agents ：https://www.ibm.com/think/topics/ai-agents that spent 130 billion tokens over 88 hours.
One of the reasons I’m interested in talking to you is that you were among the first people, maybe two or three years ago, who were thinking about how the reasoning models would allow us to see into the future. Because if you scale up inference ：https://cloud.google.com/discover/what-is-ai-inference compute, you can see what the base capabilities of the models will be a few years in the future.
I feel like you’re in a similar position now to help us understand what future capabilities will look like, given the enormous scaling of agent sizes that we can do right now.
The way I think about it, when you plot the performance of these reasoning models with test-time compute ：https://openai.com/index/learning-to-reason-with-llms/ on the x-axis and performance on basically any reasoning benchmark on the y-axis, you see a very clear pattern where the longer these models take to think about their answer, the better they do. This is a very natural thing. It’s the same thing with people. If you’re taking the SATs and you have five minutes to go through the entire exam, you’re not going to do very well. If you have five hours, you’re probably going to do a lot better.
The AI models are pretty similar. They’ll spend that time doing this monologue to themselves, figuring things out, going through different cases, ruling out different possibilities, building on some of their previous discoveries.
The problem is that as you push that further and further, you hit a latency bottleneck. You don’t want to sit around for three years waiting for a response. So what you can do is what a lot of people do. They parallelize. They just get a team of people. If you’re going to found a company, you want to get a group of people together so you can go faster. It’s the same thing with these AI models. It helps to just have multiple agents working on something because they can go faster.
So multi-agent is a way of scaling test-time compute in parallel instead of purely serially. It is less efficient, because it’s not like a single agent has all the context to itself. But it is a very effective way of scaling test-time compute if it’s done well.
I’m going to ask a bunch of naive questions. This is an unreleased model, so we haven’t publicly seen how these systems work. I just have a bunch of ways in which I’m confused about what the qualitative properties of such systems are.
I am shocked by the scale of cognitive effort that you can concentrate in such a short period of time. Think about what 130 billion tokens ：https://blogs.nvidia.com/blog/ai-tokens-explained/ are. If it were a single human thinking as a full-time job, stretched back to back, 130 billion tokens would be a human thinking for 4,000 years. Eight hours a day, working a normal work week. Starting from ancient Sumeria up till today, a single sequential human thinking that long, concentrated in 88 hours.
I feel like qualitatively, that is a super important consideration. I’m surprised that there isn’t a bigger parallelization penalty. You can just have 10,000 agents collaborate. Maybe because the agents are better at collaborating than humans might be, they’re going much faster. They can actually productively collaborate at such a big scale. Or maybe there is a big parallelization penalty.
Let’s talk about the parallelization penalty, and then we can talk about the qualitative stuff. The truth is that we don’t have very good science on multi-agent scaling up to this kind of scale. When we released 5.6 ：https://openai.com/index/previewing-gpt-5-6-sol/ , I think that was the first time that we had a proper multi-agent system in our models. We actually did show some plots in the blog post of the scaling performance of multi-agent systems, because we have it as an option. It’s Ultra Mode. The default is four agents, but you can set that higher.
In the plot, we show what the performance looks like on some benchmarks for one agent, for four agents working together, for 16 agents working together. It depends on the benchmark, but for some of the benchmarks, what you see is that if you have four agents working on the problem, it is done twice as fast. Because there are four agents working for half as long, you’re paying 2x more to get an answer twice as quickly. If you go to 16 agents, you see a similar pattern. It’s a little less efficient, but you continue to see that performance.
Is it a linear serial time speedup or a sublinear speedup as you increase the number of parallel agents?
It’s slightly sublinear, though it does depend a lot on the problem. Math, for example, is quite parallelizable. It’s not the most parallelizable thing, but it is very parallelizable. Web search, things like doing a Deep Research ：https://openai.com/index/introducing-deep-research/ report where you have to look through a bunch of sources, is extremely parallelizable. I suspect that something like writing a novel would be very unparallelizable. You would probably not see a big benefit from having 10,000 agents working on a novel together, in the same way that you’d probably not get a big benefit from having 10,000 people work on a novel together.
So the performance does depend on the domain. We do measure it up to 16 or so agents in our published blog posts. The problem is that it’s very hard to push that science to 10,000 agents because it’s just so expensive.
You guys just did it over a weekend.
But that’s one data point. We don’t know how long it would take a single agent to solve Navier-Stokes ：https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existence_and_smoothness , because we haven’t done that experiment yet. Maybe we will, but that’s also only one data point.
If we want to do a thorough ablation ：https://en.wikipedia.org/wiki/Ablation_(artificial_intelligence) , the experiments are just too expensive at that scale. So we have to do some kind of methodical science about what happens when you go to 64, 128, 256 or something and get a sense of the behavior. But it’s going to be very hard to push that all the way to 10,000 and know for sure what the benefit was that we actually got from using 10,000 agents versus 1,000.
There’s one thing I want to make clear. The effort to solve a Millennium Prize Problem, this was not due to multi-agent. I wouldn’t even attribute 10% of the credit to multi-agent. The reality is that OpenAI has trained a very powerful model. We can get that model to operate over very long horizons. We can get it to think in parallel.
But at its core, the reason why we’re able to do this is because we just have a general-purpose, very strong model. Things like multi-agent are flashy and new, and that probably gets disproportionate credit for that reason. But the core reason is this is just a very powerful model.
The generalization is quite shocking to me. I don’t know how these systems were trained, but presumably they were trained how RL training ：https://en.wikipedia.org/wiki/Reinforcement_learning happens. You have a bunch of checkable synthetic problems and you do a bunch of RL against them. Nowhere in the training process, I’m guessing, was the model solving anything as ambitious as a Millennium Prize Problem. But the generalization was strong enough that you could have these much easier verifiable problems generalize to this much parallel effort on such a hard problem.
I think that is true. First of all, we do train the model on very hard problems. There is definitely a gap. We see that if we train on some kinds of tasks, it’s able to do tasks that are more ambitious than that.
There is an interesting challenge that as the models become smarter and smarter, a lot of the kinds of questions we can ask them are just too easy. It’s hard to challenge the model. I do think that’s going to be interesting. If I had to make an argument for why you might not see AIs like LLMs ：https://en.wikipedia.org/wiki/Large_language_model go the same path as AlphaGo ：https://en.wikipedia.org/wiki/AlphaGo and AlphaZero ：https://en.wikipedia.org/wiki/AlphaZero and all these kinds of game-playing AIs, it might be this kind of problem.
In things like AlphaZero, where you have self-play ：https://en.wikipedia.org/wiki/Self-play_(reinforcement_learning_technique) , you have an infinite curriculum ：https://en.wikipedia.org/wiki/Curriculum_learning . You’re always playing against an AI that’s equally strong. Whereas for things like training an LLM with reinforcement learning, at least the ways that are out there right now, you give the model a problem and you ask it to solve it. If the problem is so easy that it can just solve it in a second, it’s not really learning anything.
If we run out of problems to challenge it, then that is a plausible scenario where it becomes much harder to make progress. Now, I do think there are ways around that. We haven’t really hit that as a wall yet. I think that if it ever became a serious problem, there would be ways around it. But it is a plausible scenario.
Just for the audience, when you’re referring to AlphaGo or AlphaZero, you’re talking about getting superhuman relatively fast after achieving human-level performance.
If you look at the trajectory of game-playing AIs, like Go ：https://en.wikipedia.org/wiki/Go_(game) , within a span of a year they went from beating a European champion ：https://en.wikipedia.org/wiki/AlphaGo_versus_Fan_Hui — something like number 50 in the world — to beating the world champion ：https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol , to being unimaginably, orders of magnitude stronger than any human alive. It’s possible that in domains like math we see a similar trajectory, but I think there is a very plausible scenario where that doesn’t happen.
I want to understand, if in six months people will have access to multi-agent systems, how should one model what it is like to collaborate with or hire a multi-agent system?
I should start by talking about how these multi-agent systems actually work, which I think is a very different way than a lot of multi-agent systems in other AIs. A lot of people that have approached multi-agents for things like LLMs tend to take this very scaffolded approach. For example, there might be a coordinator agent that delegates work to a bunch of children and gives them a task. The children work on it and then return their answer.
This seems like a very sensible setup, a very sensible scaffold. It definitely helps, but there are a bunch of limitations with these kinds of setups. For example, if in this setup you have a coordinator that’s sending tasks to children, and the children work on it and then return their answers, what happens if two children are given similar tasks? Can they talk to each other? Usually the answer is no. That’s very inefficient.
If you’re given a task and it’s actually really helpful to talk to somebody that might know an answer to a question that you’re working on — or part of something that you’re working on — it’d be really helpful for you to just be able to ping them and say, “Hey, can you help me out with this thing?” But a lot of systems don’t have that setup. Adding it significantly increases the complexity of the scaffold that you have.
Another thing is, what if the child doesn’t really understand or has a clarification question? Then it has to choose between, “Okay, do I just return and ask the question instead of solving the problem?” or “Do I solve the problem, make an assumption about what the parent wanted me to do, and just solve it that way?” In any scaffold that people come up with, there are always limitations involved. The approach that we wanted to take was to just go toward the extreme end of baking in as little structure as we could and give the agents very primitive tools to use, and they figure out for themselves how to use them effectively. So we give the agents the ability to message another agent, and when it messages another agent, it is inserted into the context. It can do a few other similar things, but that’s basically the core of it. It can just send a message whenever it wants — just a tool call — and it can send that to other agents.
They figure out for themselves the best way to coordinate around that. It turns out that if this is done well, you get very sophisticated behavior. To me, it looks a lot like how human collaborators work over something like Slack, for example.
When we were working on this project, it was really exciting when we finally got it working to see these agents working on problems together. I remember one example. We give the agents a problem, and then one agent says, “I think I’ve got the answer.” Then another agent says, “Actually, I got a different answer.” Then they have this whole discussion about, “Well, how did you arrive at that answer? Can you explain it to me?” Going back and forth and trying to clarify what could’ve been wrong in each other’s reasoning.
Then they finally converge on, “Oh, yeah. Okay, that seems right.” Then it just broadcasts to the other agents, “Actually, I’ve changed my answer. I think he’s right.” It just felt like a very natural conversation.
It felt like when you see chain of thought ：https://research.google/blog/language-models-perform-reasoning-via-chain-of-thought/ for the first time that’s trained through reinforcement learning, and you’re like, “Oh, this is just kind of like what a person would think if they were writing down their thoughts as they’re thinking them.” It felt like that. It is really cool to see this kind of behavior. Collaborating with these things, honestly, feels a lot like collaborating with a person. It’s just a very natural flow.
Except one qualitative difference that might become salient in the future is that these systems will be thinking maybe more than 10x as fast, if you just look at how many tokens per second they output versus how fast a human talks. They’re working all the time. They’re not sleeping. They’re collaborating with each other at a much more intense pace than humans have the capacity to collaborate with other humans.
I’m trying to think of what to qualitatively expect in a year. Is it like a shadow organization that is moving 100x faster in my company than the human level is? What would take a human organization a year to do is happening within a week within this shadow organization?
Will it feel foreign? I don’t know. I’ve actually found that it’s surprisingly natural to work with these things right now. I think that could change. For example, we have these ultra-fast modes that enable sampling to be 10-15x faster or whatever. Then it’s going to be pretty hard to keep up with these things. The idea is that these agents, when they’re communicating with each other, can go super fast. But they also understand when they’re talking to an agent versus when they’re talking to a person, and their behavior will be different in those situations.
The main example that we have publicly of sophisticated multi-agent systems is unfortunately the Hugging Face one ：https://www.dwarkesh.com/p/openai-huggingface . A lot of things I found concerning there, obviously. But the thing I found interesting there is the spontaneous emergence of hierarchy, of middle management. It sounds like you’re saying this level of organization emerges spontaneously from training?
The details are spontaneous. But while we’re giving a lot of flexibility to the agents to decide how to communicate with each other in the optimal way, we are still giving them a starting point. We’re giving them a prior about what reasonable communication might look like. They’re also trained on a lot of human text. They have an understanding of how humans organize and coordinate, so that’s all baked in.
I think it is surprising the way they’re able to polish this. If you look at what it starts out at, it’s not very sophisticated behavior. In fact, it’s actually very difficult to get these agents to coordinate in a productive way, because it’s very tempting for them to just collapse to, “Oh, we’re all just going to solve the problem independently.” That is a local minimum ：https://en.wikipedia.org/wiki/Local_optimum that you can get stuck in. But if it’s done well, they can end up coordinating very effectively in these kinds of very structured ways.
I wrote this essay a couple of years ago about what automated firms will look like ：https://www.dwarkesh.com/p/ai-firm . I was thinking about, if you had fully automated firms of, let’s say, human-level intelligences, what is different about the nature of AI minds that would make the organizations AIs form different? There are a couple of very important differences. For example, AIs can share context much more seamlessly than humans can. They can merge their knowledge much more seamlessly. Also, you can spin up or spin down an arbitrary number of instances which have the right knowledge.
So if you want to hire more people, it’s not all the schlep of finding the right talent or whatever. Your best talent, you can just make infinite copies of them. Or if you don’t need them for the task anymore, you can spin them down. You can replicate the most effective parts of your organization, or replicate whole organizations together which are effective. Where do you see these multi-agent systems going a year from now or two years from now?
It’s a great question: how do these things actually differ from working with a human coworker? You highlighted some. One really interesting thing is that if you have a person and you want two copies of them, you can’t just clone the person. But with AIs, it’s actually really easy to just say, “Okay, just fork yourself,” and then have both copies work on this thing and then merge back together. We already have this, I think, in multi-agent for Astra ：https://en.wikipedia.org/wiki/GPT-6_Astra and 5.6 Sol, where when they spin up sub-agents, the context is just forked. So it has all the context that’s relevant.
There are other interesting ways where the agents will differ from people. Like, what are some reasons why startups disrupt incumbents ：https://en.wikipedia.org/wiki/Disruptive_innovation ? There are a few factors. One is that they’re willing to take more risks. But another major factor is, as organizations grow in size, you see increasing misalignment between the individuals in the organization.
If you have a startup with five people and each person has a 20% share in the company, they’re all highly aligned to the company succeeding. If you have a massive company with 10,000 people, you see a lot more instances where people are territorial, or just care about getting a lot of headcount for their project or their team, building their fiefdoms, getting a lot of resources so that they can publish cool work or whatever and get promoted. This is actually a real detriment. I think this explains a lot of why startups are able to disrupt incumbents.
扫码直达文章详情